Asbjørn Munk, Stefano Cerri, Vardan Nersesjan, Christian Hedeager Krag, Jakob Ambsdorf, Pablo Rocamora García, Julia Machnio, Peirong Liu, Suhyun Ahn, Nasrin Akbari, Yasmina Al Khalil, Kimberly Amador, Sina Amirrajab, Tal Arbel, Meritxell Bach Cuadra, Ujjwal Baid, Bhakti Baheti, Jaume Banus, Kamil Barbierik, Christoph Brune, Yansong Bu, Baptiste Callard, Yuhan Chen, Cornelius Crijnen, Corentin Dancette, Peter Drotar, Prasad Dutande, Nils D. Forkert, Saurabh Garg, Jakub Gazda, Matej Gazda, Benoît Gérin, Partha Ghosh, Weikang Gong, Pedro M. Gordaliza, Sam Hashemi, Tobias Heimann, Fucang Jia, Jiexin Jiang, Emily Kaczmarek, Chris Kang, Seung Kwan Kang, Mohammad Khazaei, Julien Khlaut, Petros Koutsouvelis, Jae Sung Lee, Yuchong Li, Mengye Lyu, Mingchen Ma, Anant Madabhushi, Klaus H. Maier-Hein, Pierre Manceron, Andrés Martínez Mora, Moona Mazher, Felix Meister, Nataliia Molchanova, Steven A Niederer, Leonard Nürnberg, Jinah Park, Abdul Qayyum, Jonas Richiardi, Antoine Saporta, Branislav Setlak, Ning Shen, Justin Szeto, Constantin Ulrich, Puru Vaish, Vibujithan Vigneshwaran, Leroy Volmer, Zihao Wang, Siqi Wei, Anthony Winder, Jelmer M. Wolterink, Maxence Wynen, Chang Yang, Si Young Yie, Mostafa Mehdipour Ghazi, Akshay Pai, Espen Jimenez Solem, Sebastian Nørgaard Llambias, Mikael Boesen, Michael Eriksen Benros, Juan Eugenio Iglesias, Mads Nielsen
5 min
The FOMO25 Challenge represents a landmark effort to push brain MRI foundation models toward clinical reality. Organized by an international consortium of over 60 teams from academia and industry, it benchmarked self-supervised learning (SSL) models pretrained on the massive FOMO60K dataset—a heterogeneous collection of 60,529 MRI scans from 11,187 subjects across 16 real-world sources. Unlike toy datasets, FOMO60K captures the noise, variability, and domain shifts of actual clinical data, making it ideal for testing generalization.
The challenge had two tracks: Method (innovate SSL pretraining) and Checkpoint (finetune provided models). Downstream tasks spanned classification (e.g., brain age prediction), segmentation (e.g., white matter hyperintensities), and regression, using few-shot finetuning (e.g., 23 cases for segmentation, 200 for classification) to mimic annotation scarcity in clinics. Key insight: SSL-pretrained models consistently outperformed supervised in-domain baselines, especially on out-of-domain data, proving foundation models' value for label-efficient, robust diagnostics.
FOMO60K is the challenge's secret sauce—a CC BY-NC-SA 4.0 released dataset of diverse, unlabeled clinical MRIs. Pretraining on this 'out-of-domain' mix taught models to handle scanner differences, pathologies, and artifacts better than domain-specific supervised training. Think of it as teaching AI to be a versatile radiologist who adapts across hospitals, not a specialist locked to one scanner.
Results showed out-of-domain SSL > in-domain supervised, with gains up to 20-30% Dice for segmentation. This flips the script: heterogeneous pretraining builds resilience, countering the 'domain shift doom' in medical AI.
SSL shone brightest, but not all flavors equally. Masked Autoencoders (MAE) dominated segmentation by reconstructing local patches, capturing fine-grained anatomy. Hybrid reconstruction-contrastive led classification, blending pixel-level fidelity with global semantics. Regression tasks saw parity across objectives.
Surprisingly, scaling didn't save the day: smaller models (e.g., ViT-S) matched giants, and longer pretraining yielded diminishing returns. Intuition: for MRI's structured data, smart objectives > brute force compute. This democratizes foundation models—clinics don't need supercomputers.
Few-shot finetuning mirrored real constraints, where labeling 1000+ scans is rare. Top models hit clinical viability: e.g., MAE-based segmentation rivaled specialists on unseen data. Yet gaps remain—hybrid tasks need multi-objective pretraining, and scaling laws weaken in heterogeneous regimes.
The challenge exposed that SSL boosts generalization > accuracy on clean data, crucial for deployment across hospitals. Winners integrated domain adaptation and task-specific finetuning, paving the way for plug-and-play clinical AI.
FOMO25 proves foundation models can tame MRI chaos, slashing annotation needs by 10-50x while boosting out-of-domain performance. Released assets (FOMO60K, checkpoints) accelerate progress, but challenges like objective-task mismatch highlight needs for adaptive architectures. Ultimately, this shifts medical AI from siloed models to versatile 'MRI GPTs' enhancing diagnostics, from neurodegeneration to stroke.
Clinical deployment of automated brain MRI analysis faces a fundamental challenge: clinical data is heterogeneous and noisy, and high-quality labels are prohibitively costly to obtain. Self-supervised learning (SSL) can address this by leveraging the vast amounts of unlabeled data produced in clinical workflows to train robust \textit{foundation models} that adapt out-of-domain with minimal supervision. However, the development of foundation models for brain MRI has been limited by small pretraining datasets and in-domain benchmarking focused on high-quality, research-grade data. To address this gap, we organized the FOMO25 challenge as a satellite event at MICCAI 2025. FOMO25 provided participants with a large pretraining dataset, FOMO60K, and evaluated models on data sourced directly from clinical workflows in few-shot and out-of-domain settings. Tasks covered infarct classification, meningioma segmentation, and brain age regression, and considered both models trained on FOMO60K (method track) and any data (open track). Nineteen foundation models from sixteen teams were evaluated using a standardized containerized pipeline. Results show that (a) self-supervised pretraining improves generalization on clinical data under domain shift, with the strongest models trained \textit{out-of-domain} surpassing supervised baselines trained \textit{in-domain}. (b) No single pretraining objective benefits all tasks: MAE favors segmentation, hybrid reconstruction-contrastive objectives favor classification, and (c) strong performance was achieved by small pretrained models, and improvements from scaling model size and training duration did not yield reliable benefits.
Alex: Does that mean clinics could run these on regular computers?
Sam: It suggests so. The Method track required using FOMO60K, and it showed you don't need huge proprietary datasets for strong adaptation. Tailoring the pretraining to the task—like local methods for details or global ones for patterns—made a difference.
Alex: How did they compare directly to supervised baselines?
Sam: They had two baselines. Supervised-OOD trained on the small out-of-domain finetune data. Supervised-ID used a larger set from the Danish hospital.
Alex: So Supervised-ID had a home advantage.
Sam: Most pretrained models ranked higher than Supervised-OOD. Two even beat Supervised-ID reliably, based on statistical tests. For infarct spotting, one gained a notable edge in AUROC—a measure of cleanly separating healthy from damaged—over Supervised-OOD.
Alex: What drove those wins? Did the pretraining type matter per task?
Sam: Yes. Local methods like MAE helped segmentation by focusing on fine details. Hybrids with global contrastive led on classification by capturing broader patterns.
Alex: What about tweaks during training, like changing learning speeds or image adjustments?
Sam: Minimal tweaks often ranked higher on spotting. Heavy ones helped outlining but hurt age prediction.
Alex: And model size or training length?
Sam: Larger models had only a weak edge. More pretraining samples brought no clear gains.
Alex: So a practical path forward, tuned per task.
Alex: Overall, does this suggest foundation models are ready for clinics?
Sam: The results show they can work under real conditions like few labels and scanner differences. Smaller models stayed competitive. But the setup mixed pretraining quality with finetuning choices, which adds real-world relevance but makes it harder to pinpoint why models won. Single tasks per category limit broad claims.
Alex: Still, the outperformance holds up statistically—a meaningful step for diagnostics that adapt across hospitals with minimal local data. Thanks, Sam, for breaking it down so clearly.
Sam: My pleasure, Alex. Thanks for listening to ResearchPod.