Demographically-Conditioned Synthetic Medical Images for Bias Mitigation and Bias Detection in Disease Classifiers

Medical image classifiers often suffer from performance disparities across demographic subgroups (e.g., age or sex). These biases are difficult to mitigate during training due to data scarcity and hard to detect during evaluation because minority subgroups in test sets are often too small to provide statistically stable performance estimates. This paper investigates whether demographically-conditioned synthetic images, generated via a fine-tuned Stable Diffusion 2.1 model, can serve as both a training tool to mitigate bias and an evaluation tool to perform more reliable fairness audits.

The authors compared two ways to incorporate synthetic data: joint augmentation (mixing real and synthetic data in one training pass) and sequential pretraining (pretraining on synthetic data followed by fine-tuning on real data). They found that sequential pretraining with a demographically-balanced synthetic cohort is significantly more effective. Specifically, pretraining on balanced synthetic data followed by fine-tuning on only 1% of the real training set outperformed models trained on the full real dataset, achieving 100x real-data efficiency. In contrast, joint augmentation was only effective when large amounts of real data were already available.

To address the problem of small, unreliable test sets, the authors evaluated whether a synthetic cohort could act as a proxy for a well-powered real-world test set. They found that the synthetic estimator perfectly reproduced the subgroup ranking of a large, well-powered real oracle (Spearman rho = 1.00). While the synthetic proxy is not a perfect substitute for absolute performance measurement, it is highly effective at identifying which subgroups are most impacted by bias, particularly in the minority cells where real-world test data is insufficient.