SPAR: Semantic-Pixel Self-Alignment and Adaptive Routing for Unified Multimodal Models

Multimodal Large Language Models (MLLMs) excel at visual understanding but struggle with high-quality image generation. This performance gap stems from a fundamental conflict: semantic encoders are optimized for discriminative tasks (discarding pixel details), while generative models require fine-grained pixel-level reconstruction. The authors investigate whether they can unify these capabilities by shifting generative modeling directly into the semantic representation space without relying on external teachers or suffering from catastrophic forgetting.

The authors propose SPAR (Semantic-Pixel self-alignment and Adaptive Routing), which consists of three main innovations:

1. Asymmetric Dual-Stream Tokenizer: This architecture splits visual processing into two streams. A lightweight semantic stream preserves discriminative features, while a Transformer-augmented pixel stream recovers high-frequency spatial details. These are fused into a compact latent space that allows for high-fidelity reconstruction.
2. Self-Aligned Generation: Instead of using external models to guide the diffusion process, SPAR uses its own optimized tokenizer as an internal teacher, ensuring the generative model stays aligned with the MLLM's native semantic space.
3. Dynamic Token Routing (DTR): Recognizing that different tokens have different semantic needs, DTR allows each token to adaptively aggregate multi-layer features from the MLLM, rather than relying on static, token-agnostic feature extraction.

SPAR achieves state-of-the-art performance for unified multimodal architectures. By explicitly decoupling semantic preservation from pixel reconstruction, the model maintains strong visual understanding capabilities while significantly improving image generation and reconstruction quality compared to existing methods that suffer from blurry artifacts or lossy compression. The DTR mechanism further enhances performance by allowing the model to dynamically leverage hierarchical MLLM features for generative guidance.