CAT: Confidence-Adaptive Thinking for Efficient Reasoning of Large Reasoning Models

Large Reasoning Models (LRMs) often suffer from overthinking on simple queries, leading to excessive token consumption and increased latency. While existing methods attempt to compress reasoning chains, they often use uniform reduction or coarse-grained difficulty estimation, which can degrade performance on complex tasks. This paper asks: can we use a model's intrinsic confidence to autonomously modulate reasoning depth, balancing efficiency and accuracy?

The authors propose Confidence-Adaptive Thinking (CAT), a framework that leverages self-certainty—a measure derived from the model's internal token distributions—as a fine-grained indicator of reasoning quality and problem difficulty. The framework operates in two stages:

1. Confidence-Aware Preference Labeling (CAPL): The model samples multiple reasoning trajectories and uses self-certainty to construct preference pairs. These pairs are categorized into Conciseness Pairs (CPs) for correct responses and Deliberation Pairs (DPs) for incorrect ones, with preference scores weighted by the margin of self-certainty and length.
2. Confidence-Weighted Preference Optimization (CWPO): The model is fine-tuned using a modified preference optimization objective. Unlike standard methods, CWPO dynamically weights the loss based on a calibration ratio of self-certainty to reasoning length, encouraging the model to favor concise paths when confident and longer, more deliberative paths when uncertain.

CAT consistently outperforms state-of-the-art baselines across multiple benchmarks (MATH-500, AIME24, and GPQA) on various base models. By integrating self-certainty, CAT achieves a superior balance between inference efficiency and reasoning accuracy. Specifically, it effectively reduces token overhead for simple problems while preserving the model's ability to explore complex reasoning paths for harder inputs. Ablation studies confirm that both the confidence-aware labeling and the weighted optimization objective are essential for maintaining performance while achieving compression.