MIMFlow: Integrating Masked Image Modeling with Normalizing Flows for End-to-End Image Generation

Normalizing Flows (NFs) are powerful for exact density estimation, but they often struggle to capture high-level semantic structures because they are forced to dedicate their capacity to modeling low-level pixel details. The authors investigate whether integrating Masked Image Modeling (MIM)—a technique typically used for discriminative representation learning—into an end-to-end generative pipeline can resolve this capacity bottleneck and improve image synthesis.

MIMFlow introduces a unified framework that jointly optimizes three components: a masked encoder, a latent normalizing flow, and a generative decoder. The encoder uses learnable query tokens to extract a fixed-dimensional latent representation from masked images, forcing the model to prioritize global structure over local noise. The normalizing flow then models this simplified semantic manifold, while a specialized decoder handles the reconstruction of high-frequency pixel details. This design allows for a principled decoupling of tasks, where the flow model is shielded from redundant pixel-level correlations.

Empirical results on the ImageNet 256x256 benchmark demonstrate that MIMFlow-L achieves an FID of 2.50, representing a 32.8% performance improvement over similar-scale normalizing flow baselines. The model achieves these results using only 128 tokens, which is 50% fewer than standard generative models, highlighting the efficiency of the learned latent space. Furthermore, linear probing evaluations confirm that the latent representations possess significantly higher semantic quality compared to traditional approaches.

This work provides a new recipe for normalizing flows that moves beyond modular, two-stage training pipelines. By demonstrating that MIM can be effectively unified with generative flows, the authors offer a pathway to build more efficient and semantically aware generative models that do not sacrifice high-fidelity synthesis for structural coherence.