Alzheimer’s Disease Diagnosis Using a Multimodal Approach with 3D MRI and PET

Research Approach

This paper addresses the challenge of early Alzheimer's disease (AD) diagnosis by integrating structural (MRI) and functional (PET) neuroimaging data. While many existing models rely on static concatenation of these modalities, the authors propose an end-to-end deep learning framework that utilizes 3D convolutional neural networks (CNNs) for feature extraction. To improve diagnostic performance and computational efficiency, the model incorporates three distinct fusion strategies—concatenation, Gated Multimodal Units (GMU), and gated self-attention—alongside a sparsely gated Mixture-of-Experts (MoE) classifier. This MoE architecture allows the model to perform input-adaptive routing, activating only the most informative expert sub-networks for each specific patient case.

Key Findings

The proposed model was evaluated on three binary classification tasks: Normal Cognition (NC) vs. Mild Cognitive Impairment (MCI), MCI vs. AD, and NC vs. AD. The Gated Multimodal Unit (GMU) fusion strategy achieved 80.46% accuracy for NC vs. MCI and 95.47% for NC vs. AD. The gated self-attention mechanism performed best on the MCI vs. AD task, reaching 82.08% accuracy. Ablation studies confirmed that the inclusion of the MoE layer was critical, as its removal consistently degraded classification performance across all tasks, validating the benefit of adaptive, multimodal processing.

Interpretability and Clinical Relevance

A significant barrier to clinical adoption of AI in neurology is the black-box nature of deep learning models. To address this, the authors employed Grad-CAM, a visualization technique that highlights the specific brain regions influencing the model's predictions. By identifying pathological markers in the hippocampus and cortex, the model provides clinicians with transparent, interpretable evidence, potentially increasing trust and facilitating its integration into diagnostic workflows.

Early detection of AD at the MCI stage is crucial for timely intervention. By moving beyond static fusion methods and adopting input-adaptive architectures like MoE, this research demonstrates that multimodal models can better handle the inherent heterogeneity of patient data. This approach not only improves diagnostic accuracy but also offers a path toward more personalized and explainable clinical decision support systems.