Unpaired Joint Distribution Modeling via Multi-Scale Image Representations

How can we accurately model the joint distribution of two domains (e.g., clean and noisy images) when only unpaired marginal observations are available? The paper addresses the inherent ill-posedness of this problem by proposing a framework that balances the need for domain consistency with the preservation of essential information.

The authors introduce LUD-MSR (Latent-variable Unpaired Distribution modeling via Multi-Scale image Representations). The framework operates in two stages:

1. Representation Learning: It learns an auxiliary representation mapping that extracts structural features while suppressing domain-specific noise. This is achieved through a Multi-Scale image Representation (MSR) mapping, which uses wavelet operators and normalizing flows to isolate low-frequency content.
2. Probabilistic Modeling: It uses a latent-variable graphical model to approximate the joint distribution. The model is trained by optimizing evidence lower bounds (ELBOs) derived from the marginal data, with the auxiliary representations fixed to ensure stable inference.

The study provides a theoretical analysis establishing an upper bound on the distribution approximation error. This analysis highlights a fundamental trade-off: while coarser scales improve domain consistency (making the two domains look more similar), they risk discarding fine-grained structural details. The proposed MSR mapping is shown to achieve a superior balance of this trade-off compared to existing methods like linear projections or simple noise-injection mappings. Experiments on standard denoising benchmarks and cryo-EM data demonstrate that LUD-MSR synthesizes more realistic pseudo-paired training samples, leading to improved denoising performance.

Unpaired learning is critical in scientific fields where collecting aligned data is expensive or physically impossible. By providing a mathematically grounded framework that explicitly models the trade-offs in latent representation learning, this work offers a more stable and interpretable alternative to heuristic-based methods like cycle-consistency or standard adversarial training.