RETHINKING PRIVILEGED INFORMATION IN ON-POLICY SELF-DISTILLATION

This paper investigates whether performance improvements in on-policy self-distillation (OPSD) stem from the student genuinely learning the privileged reference information provided to the teacher, or if they simply reflect the recovery of reasoning behaviors already latent in the base model. OPSD typically pairs a student generating responses from a problem alone with a teacher evaluating those same responses while conditioned on a reference solution. To isolate the effects of this reference, the authors introduce an analysis framework that decomposes teacher supervision into a reference direction and a recovery direction. Using Qwen3 models ranging from 1.7B to 8B trained on mathematical and scientific reasoning datasets, the study compares student changes against original teacher supervision as well as control conditions where reference solutions are swapped with those from entirely different problems.

The experiments reveal that access to the correct reference solution does not provide a consistent performance benefit across teacher generation modes, model sizes, or training datasets. Notably, students can improve without the correct reference, and solutions sourced from completely different problems can occasionally outperform correct reference solutions on mathematical reasoning benchmarks. Furthermore, distributional analysis shows that the student's predictions align much more strongly with the base model's general thinking behavior than with the specific supervision induced by the reference. Because control problems reproduce much of this alignment and stronger alignment does not reliably track with performance boosts, the authors conclude that standard metrics like performance gains and distributional alignment are insufficient to prove how privileged information contributes to student learning.

These findings challenge the prevailing assumption that performance gains in on-policy self-distillation are driven by the direct transfer of privileged contextual knowledge. By demonstrating that students can thrive under mismatched or absent reference conditions, the work highlights the need for more rigorous evaluation methods in post-training. Researchers must look beyond aggregate performance improvements to understand whether models are actually acquiring new knowledge or merely unlocking existing capabilities.