ResearchPod Summary
When distilling reasoning capabilities into Large Language Models (LLMs), a common practice is to use "answer-conditioned" generation—providing the model with the gold answer to help it produce a correct chain of thought when it otherwise fails. This paper investigates whether these answer-conditioned chains, despite being correct, provide a lower-quality training signal than chains generated without the answer (answer-blind).
The researchers conducted a controlled "one-bit" experiment using Qwen3-8B. They kept the generator, the problem set, and the correctness filter constant, varying only whether the model was shown the gold answer during generation. They then performed supervised fine-tuning (SFT) on both sets of chains and compared the resulting students' performance on verifiable-reasoning benchmarks like MATH-500. They also analyzed the internal structure of the chains, specifically tracking the "answer-first rate" (AFR)—the frequency with which the model states the final answer early in its reasoning process.
Training on answer-conditioned chains leads to a sharp decline in verifiable-reasoning accuracy, with the penalty growing as problem difficulty increases (up to 27 points on AIME competition problems). The authors identify this as a form of "off-policy" learning: the model learns to rationalize backward from a known answer rather than deriving it from first principles. This behavior is detectable before training by measuring the AFR in unlabeled generations. The study also found that this "answer-first" habit persists in the student model's own future outputs, even when it is no longer provided with the answer, leading to a "train/test length inversion" where the model produces longer, less effective chains at test time.
This work challenges the assumption that any correct chain of thought is a valid training target for distillation. It demonstrates that the process of generation matters as much as the outcome. For researchers building reasoning models, the practical takeaway is to prioritize answer-blind generation. If answer-conditioning is unavoidable, the authors suggest using a "derive-first" instruction to mitigate the damage, as this recovers a significant portion of the lost accuracy.
AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.