---
title: "KG-MultiMS: a Knowledge Graph-Enhanced Multimodal Framework for Multiple Sclerosis"
authors:
  - "Riccardo Francia"
doi: "10.24963/ijcai.2026/943"
url: "https://ijcai.org/proceedings/2026/943"
pdf: "https://ijcai.org/proceedings/2026/0943.pdf"
venue: "Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence"
year: 2026
pages: "8329-8330"
track: "Doctoral Consortium"
keywords:
  - "Machine Learning: Knowledge-aided learning"
  - "Computer Vision: Biomedical image analysis"
  - "Machine Learning: Multi-modal learning"
  - "Data Mining: Knowledge graphs and knowledge base completion"
---

# KG-MultiMS: a Knowledge Graph-Enhanced Multimodal Framework for Multiple Sclerosis

Riccardo Francia

*Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence*, pages 8329-8330. DOI: [10.24963/ijcai.2026/943](https://doi.org/10.24963/ijcai.2026/943)

## Abstract

Despite advances in the computational analysis of multiple sclerosis, the ability to accurately predict disease progression remains limited due to unimodal approaches in the context of deep learning. KG-MultiMS proposes as a multimodal deep-learning framework that integrates data from three heterogeneous sources: (1) MRI volumes using the Swin-UNETR architecture to learn both semantic embeddings and segmentation masks of brain lesions; (2) tabular clinical variables using the FT-Tokenizer architecture to learn clinical embeddings; (3) medical knowledge graph based on the Harvard-Oxford atlas using a Heterogeneous Graph Transformer to learn contextual embeddings from patient-specific subgraphs. A cross-attention mechanism will allow for leveraging multimodal contributions by dynamically weighting them based on the anatomical-functional concordance represented in the contextual embeddings.