Reasoning Quality Emerges Early: Data Curation for Reasoning Models

Supervised fine-tuning (SFT) is critical for eliciting reasoning capabilities in Large Language Models (LLMs), but it is highly sensitive to data quality. Current methods for selecting high-quality reasoning data—prioritizing difficulty and diversity—rely on expensive LLM-based filtering or long-trace analysis. This paper investigates whether these attributes can be identified more efficiently by analyzing only the initial segments of reasoning traces.

The authors introduce Token-Efficient Model Perturbation (TEMP), a two-stage data selection framework. First, they identify difficult problems by measuring the loss of the first 100 reasoning tokens at a randomly perturbed checkpoint of the pretrained model. This acts as a stress test, where difficult examples exhibit sharp loss spikes. Second, to ensure diversity, they cluster examples based on their loss patterns over the first 1,000 tokens across several checkpoints extrapolated along the fine-tuning trajectory. This approach leverages the observation that the fine-tuning loss landscape is generally smooth and low-curvature, allowing early-stage loss signals to serve as reliable proxies for full-trace behavior.

TEMP effectively filters and selects training data without needing auxiliary LLM judges. The authors demonstrate that the problem-understanding phase (the first 100 tokens) is a robust indicator of problem difficulty, outperforming simple heuristics like response length. Furthermore, by sampling based on loss patterns at directionally perturbed checkpoints, they curate subsets that induce similar gradients to the full dataset. Experiments on Qwen2.5-7B and Llama3.1-8B across medical and mathematical reasoning datasets show that TEMP outperforms existing baselines by up to 1.7% in accuracy while achieving a 91% reduction in token-processing requirements.