Is More Privileged Information Better? From Solution Traces to Problem-Solving Structure in Self-Distilled Reasoning

On-policy self-distillation (OPSD) is a powerful technique for training reasoning models, where a student model learns from a privileged teacher that has access to a verified reference solution. However, a common issue is that the teacher often provides token-level targets tied to specific, non-transferable details of the reference solution—such as specific arithmetic paths or phrasing—that the student cannot replicate without that privileged information. The authors investigate whether representing this privileged information as structured problem-solving steps, rather than a full solution trace, improves the student's ability to generalize.

To address this, the authors introduce Problem-Space-Guided OPSD (PS-OPSD). They define a problem-space representation consisting of four key components: the initial state, goal conditions, constraints, and a selected state-transition path. This representation is extracted offline from verified solutions. During training, the teacher is conditioned on this structured guidance instead of the full solution, while the student continues to generate responses based only on the question.

Across three mathematical reasoning benchmarks (AIME24, AIME25, and HMMT25) and model scales ranging from 1.7B to 8B parameters, PS-OPSD consistently achieves the highest aggregate accuracy compared to standard OPSD and other baselines like SFT and GRPO. The authors demonstrate that the performance gains are not merely due to the presence of extra information, but specifically to the relevance and coherence of the structured guidance. Controlled experiments show that breaking the coherence of the state-transition path or using mismatched guidance significantly degrades performance, confirming that the structural representation is the primary driver of the observed improvements.

This work highlights that the design of the privileged context is as critical as the distillation objective itself. By abstracting away instance-specific noise and focusing on the underlying problem-solving structure, researchers can better align the teacher's supervision with the student's inference-time capabilities. This provides a scalable path for improving reasoning models without needing to rely on increasingly complex or opaque teacher-student architectures.