Concurrent Image Understanding and Generation: Self-Correcting Coupled Markov Jump Processes

Existing multimodal generation models often treat text and image synthesis as independent or interleaved processes, leading to cross-modal drift where text and images become inconsistent. The authors ask: can we treat understanding and generation as a single, tightly coupled stochastic process where each modality continuously informs and corrects the other in real-time?

To address this, the authors propose Self-Correcting Coupled Markov Jump Processes (SC-CMJP). Unlike standard Masked Diffusion Models (MDMs) that update modalities independently, SC-CMJP uses the model's internal cross-modal attention to make one modality's transition rates a function of the other's confidence.

They implement this via a training-free sampler called CO2Jump. This sampler performs a single forward pass per denoising step, extracting 'Self-Confidence' scores for each modality. It then uses an entropy-gated mixture of these scores and cross-modal attention signals to decide which tokens to keep and which to 'remask' (retract). This allows the model to detect and repair cross-modal contradictions—such as text describing an object that the image branch has not yet rendered—within the same sampling trajectory.

• Joint Performance: CO2Jump achieves state-of-the-art performance on concurrent image understanding and editing, as well as visual reasoning tasks like maze and nonogram solving.
• Monotonic Scaling: The sampler's performance scales monotonically with the number of denoising steps, confirming that the benefits of cross-modal coupling compound throughout the generation process.
• The remasking mechanism successfully retracts early, incorrect commitments, leading to more coherent and logically consistent multimodal outputs compared to standard parallel samplers.