Posterior Samplings are Missing Modalities Generators for Medical Image Translation

Medical imaging, such as MRI, often requires multiple modalities (e.g., T1, T2, FLAIR) for accurate diagnosis. However, clinical constraints frequently result in incomplete datasets where some modalities are missing. The author addresses the challenge of synthesizing these missing modalities without the need for retraining separate models for every possible combination of input and output.

Instead of using traditional GAN-based or complex hierarchical architectures, the author frames missing-modality generation as a linear inverse problem. The framework utilizes a conditional flow-matching model to learn a joint prior over all modalities. During inference, the model reconstructs missing data by starting from noise and iteratively guiding the sampling trajectory to ensure consistency with the observed modalities. To prevent error accumulation when generating multiple targets, the author implements a 'many-to-one' sampling strategy, ensuring that each target is synthesized using only the available, verified source information.

Experiments on the BraTS and IXI datasets demonstrate that this posterior sampling approach outperforms existing unified frameworks (such as MM-GAN, ResViT, and MMHVAE) across various missing-modality scenarios. The synthesized images show higher fidelity and better preservation of clinically relevant structures, which translates to improved performance in downstream tasks like tumor segmentation. The method's ability to dynamically fuse information from arbitrary observed modalities makes it highly flexible for real-world clinical workflows.

This work provides a robust, unified solution for the common clinical problem of missing data in multimodal imaging. By avoiding the architectural complexity of previous generative models and leveraging the mathematical framework of inverse problems, the proposed method offers a more efficient and accurate way to synthesize missing diagnostic information, potentially reducing the need for repeat scans and improving diagnostic consistency.