OPD-V: Visual On-Policy Self-Distillation with Modality Balance

Multimodal Large Language Models (MLLMs) often suffer from Modality Imbalance, where the model relies too heavily on textual priors rather than integrating visual information during generation. Existing On-Policy Self-Distillation (OPSD) methods attempt to improve reasoning by providing privileged information to a teacher model, but these methods often fail to address the underlying imbalance between modalities. This paper asks: can Modality Balance itself be used as privileged information to improve MLLM reasoning?

The authors introduce OPD-V, a framework that uses Modality Balance to guide self-distillation. They construct a Positive Teacher (using a zoomed-in image) and a Negative Teacher (using a masked image) to create a contrast in how the model processes visual versus textual information. By comparing the logit outputs of these two teachers, the authors define a Modality-Balance Logits Margin. Tokens that receive higher support from the Positive Teacher than the Negative Teacher are placed in a Modality-Balance Trust Region. The student model is then trained to distill knowledge from the Positive Teacher only for tokens within this trust region.

OPD-V consistently outperforms standard OPSD and other baseline post-training methods across six benchmarks, four MLLM backbones, and five different post-training strategies. For instance, on the Qwen3.5-4B backbone, OPD-V achieved an average accuracy of 80.01%, significantly higher than the 69.26% achieved by standard OPSD. Beyond accuracy, the method also improves computational efficiency; by focusing distillation on a specific trust region and producing more concise responses, OPD-V reduces training step latency by up to 31.8% compared to standard OPSD.

This work provides a novel way to address the persistent issue of Modality Imbalance in MLLMs. By treating the internal allocation of visual and textual attention as a source of privileged information, the authors demonstrate that models can be steered toward better visual reasoning without requiring external, manually curated textual solutions. The resulting framework is both more accurate and more computationally efficient than existing self-distillation approaches.