CALIBER: Calibrating Confidence Before and After Reasoning in Language Models

As reasoning models become more prevalent, they are increasingly required to provide reliable confidence estimates alongside their answers. Existing methods typically elicit confidence at a single point—either before or after the reasoning process—and apply a uniform supervision target. This paper investigates whether confidence is state-dependent, arguing that the information available to a model changes significantly between the start and end of a reasoning trace, and that supervision targets should be aligned with these distinct information states.

The authors introduce CALIBER, a framework that elicits two verbalized confidence estimates: one before the model begins its reasoning trace (pre-confidence) and one after the answer is generated (post-confidence). CALIBER uses a dual-supervision protocol:

• Pre-confidence is supervised using a group-wise target, which represents the empirical success rate of the model's current policy on a given prompt.
• Post-confidence is supervised using an instance-wise target, which reflects the binary correctness of the specific generated answer.

To ensure stable training, the authors employ a two-stage reinforcement learning process. A warmup stage optimizes the pre-confidence signal, followed by a joint training stage that incorporates the post-confidence and accuracy rewards. This prevents the group-wise pre-confidence signal from being overwhelmed by the higher-variance instance-wise rewards.

CALIBER demonstrates significant improvements in calibration metrics across both 7B and 30B models. On the BigMathDigits benchmark, the 7B model achieved a 52.5% reduction in Expected Calibration Error (ECE) compared to the strongest single-confidence baseline. The model maintained high task performance, remaining within 2.1 points of the best accuracy. Furthermore, the framework showed strong generalization, achieving superior ECE and Brier scores on out-of-distribution benchmarks like GPQA and TriviaQA. The authors conclude that aligning the confidence position with the appropriate supervision target is critical for reliable uncertainty estimation, particularly under distribution shift.