Refining Over Resampling: Test-Time Self-Correction for LLM Reasoning

As test-time scaling becomes a standard approach for improving LLM reasoning, researchers face a trade-off between sampling more candidates (breadth) and improving individual reasoning chains (depth). The authors investigate whether relying solely on wider sampling leads to diminishing returns due to redundant reasoning patterns and whether a structured, iterative refinement process can better utilize inference compute without requiring external verifiers or additional training.

The authors propose a breadth-depth refinement framework that operates entirely at inference time. The process begins by sampling $N$ independent reasoning rollouts from the base model. Each rollout then undergoes $D$ rounds of refinement, where the model acts as a generator, a critic, and a corrector in sequence. The generator continues the reasoning, the critic identifies potential logical or arithmetic errors, and the corrector revises the trace based on that critique. Finally, the refined outputs are aggregated using simple plurality voting. This approach avoids the need for external reward models or specialized training, making it highly portable across different open-weight models.

The framework consistently outperforms standard baselines—including greedy decoding, majority voting, and verifier-based best-of-$N$ selection—across five mathematical benchmarks (AIME24, AIME25, AMC, OlympiadBench, and MATH500). For example, using the Qwen2.5-1.5B model, the method significantly boosts accuracy on MATH500 compared to the strongest verifier-based baseline. The authors demonstrate that while wider sampling often revisits the same semantic clusters, their refinement approach effectively repairs local reasoning errors, leading to a higher net accuracy gain per unit of compute.

This work provides a practical, model-agnostic way to enhance reasoning performance for smaller or mid-sized LLMs. By proving that structured self-correction can surpass the performance of external verifiers, the authors offer a path to improve reasoning in environments where training a dedicated reward model is either too expensive or technically infeasible. It highlights that the "intelligence" of a model can be unlocked by better managing the inference process rather than just increasing the number of raw samples.