Argus-Unified: Towards A Compact and Economical Unified Model for Image Understanding and Generation

How can we unify visual understanding and image generation in a single, compact model without the massive computational costs and data requirements typical of existing unified multimodal models (UMMs)?

Argus-Unified addresses the conflict between the visual features required for understanding (which benefit from rich, continuous semantic representations) and generation (which require discrete tokens for autoregressive modeling). Instead of training from scratch or using separate encoders, the authors leverage a pretrained vision-language model (VLM).

The training pipeline consists of two stages:

1. Unified Vision Tokenizer Training: A quantizer and image decoder are trained on top of a frozen vision encoder. This allows the encoder to output both continuous tokens (for understanding) and discrete tokens (for generation) simultaneously.
2. Unified Multimodal Training: The LLM is initialized from a pretrained VLM, allowing it to retain existing vision-language alignment. The model is then fine-tuned on a significantly smaller dataset (15.6M samples) compared to industry standards, optimizing for both understanding and generation tasks.

Argus-Unified demonstrates that unified models can be trained efficiently, achieving state-of-the-art performance on understanding benchmarks like GQA, POPE, and VQAv2. It achieves this at approximately 1/10th the training cost (~$2,000) and with 1/5th the data of comparable models like Janus or VILA-U. By using a hybrid token design, the model avoids the parameter overhead of using dual encoders while maintaining high-fidelity performance in both domains.

This work lowers the barrier to entry for developing unified multimodal models. By proving that one can effectively repurpose existing, smaller-scale VLMs rather than training massive models from scratch, the authors provide a practical, economical baseline for future research in multimodal AI.