AI-Powered Research Discovery© 2026 ResearchPod · Berlin, Germany

About Research Blog NotebookLM Speechify Contact Terms Privacy iOS App

OmniMed-FL: A Robust Multimodal Federated Learning Framework for Clinical Diagnosis | ResearchPod

arxiv

OmniMed-FL: A Robust Multimodal Federated Learning Framework for Clinical Diagnosis

Ayush Debnath, Ruelia Saha, Sudip Misra/Sep 9, 2026/Original/PDF

Generate podcast

ResearchPod Summary

Research Question

Approach

Main Findings

Why It Matters

Paper Details

DOI: 10.48550/arxiv.2609.10364
OpenAlex: W7212114717

Topics

Multimodal Federated LearningNon-IID DataClinical Condition ClassificationLabel SkewFusion Strategies

Read next

OmniMed-FL: A Robust Multimodal Federated Learning Framework for Clinical DiagnosisAyush Debnath et al.4 min
Federated Learning for Breast Density Classification: A Real-World ImplementationHolger R. Roth et al.7 min
Towards Robust Personalized Federated Learning: Vulnerability Assessment and Defense Co-DesignMingyuan Fan et al.

As clinical diagnostics increasingly rely on both medical imaging and patient records, a significant challenge arises: how to perform multimodal analysis while adhering to strict data privacy regulations like HIPAA and GDPR. The authors investigate whether federated learning (FL) can effectively fuse visual and textual data across distributed hospital networks without aggregating sensitive raw data into a central repository.

To address this, the authors developed OmniMed-FL, a framework that enables decentralized training of vision-language models. The system uses a proxy corpus of 3,000 chest radiographs paired with class-conditioned synthetic clinical notes. The study evaluates eight different fusion strategies, various initialization methods, and techniques for handling missing data. The framework was tested under diverse conditions, including varying levels of non-IID (non-identically distributed) data skew and different numbers of hospital clients (3 to 20), using metrics such as macro-F1 score, communication volume, and computational overhead.

Multimodal fusion consistently outperformed single-modality baselines, achieving higher classification accuracy by leveraging both radiological patterns and textual context. The researchers found that label skew—the uneven distribution of clinical conditions across different hospitals—is a much stronger driver of performance degradation than the number of participating clients. While federated training incurs additional costs in terms of model state size and communication bandwidth compared to unimodal approaches, it successfully retains nearly all the performance of a centralized model. The study also highlights that warm-start initialization and anti-collapse regularization are critical for maintaining stable performance in decentralized, multimodal settings.

OmniMed-FL provides a practical blueprint for hospitals to collaborate on complex diagnostic models without compromising patient privacy. By demonstrating that multimodal federated learning is both feasible and effective, this work paves the way for more robust, data-efficient clinical AI that can learn from the full breadth of a patient's medical history across multiple institutions.

Key Terms

Federated Learning (FL): A machine learning paradigm where models are trained across multiple decentralized devices or servers holding local data samples, without exchanging the data itself.
Non-IID Data: Data that is not identically and independently distributed, which in this context refers to hospitals having different distributions of clinical conditions, posing a challenge for model convergence.
Multimodal Fusion: The process of combining information from different data types, such as images and text, to improve the predictive performance of a machine learning model.
Label Skew: A form of data heterogeneity where different clients have significantly different proportions of class labels, which can lead to biased or unstable model updates.
Macro-F1 Score: A performance metric that calculates the unweighted mean of the F1 scores for each class, ensuring that the model's performance is evaluated equally across all clinical conditions.

AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.