CONSENSUS AS PRIVILEGED CONTEXT FOR LABEL-FREE SELF-DISTILLATION

Large language models often struggle with complex reasoning, and while sampling multiple solutions and taking a majority vote (self-consistency) improves accuracy, it is computationally expensive to perform at inference time. Existing label-free training methods attempt to distill this consensus into model weights, but they typically treat consensus as a simple filter or a scalar reward, discarding the rich information contained in the agreeing solutions. The authors introduce CANON (Consensus-ANchored self-distillatiON), which treats the consensus solution as a privileged context. By conditioning a frozen snapshot of the model on a consensus-reaching solution, the model creates a teacher that provides dense, token-level supervision for the student model to follow during training.

CANON demonstrates that consensus can serve as a powerful, label-free proxy for gold-standard supervision. In transductive experiments on mathematical and scientific reasoning benchmarks, CANON improves pass@1 accuracy by up to 12 points, outperforming label-free reinforcement learning baselines by 6 points while using only one-seventh of the compute. Notably, CANON approaches the performance of an oracle teacher conditioned on gold-verified solutions, suggesting that the model's own consensus captures much of the value of ground-truth labels. Furthermore, the improvements are not merely a result of distribution sharpening; the model gains the ability to solve problems it previously could not, and its majority-vote accuracy improves even on held-out benchmarks.

This work provides a highly efficient, scalable way to improve reasoning capabilities in large language models without requiring expensive human-labeled data or complex reinforcement learning pipelines. By turning the model's internal agreement into a dense training signal, CANON bridges the gap between unsupervised test-time training and supervised fine-tuning, offering a practical path for models to self-improve on unlabeled data.