Irina Proskurina, Mayank Kumar, Oyindolapo Komolafe
6 min
As large language models are increasingly deployed in high-stakes domains such as medicine, finance, and law, reliable estimates of model confidence become crucial. While instruction tuning is known to improve general task performance, recent studies show that it often induces verbalized overconfidence. This paper investigates whether these changes in model confidence are accompanied by corresponding shifts in the lexical diversity of the reasoning chains (rationales) generated to support answers. The authors examine whether uncertainty reduction aligns with diversity changes and whether these effects persist when controlling for answer selection and generation length.
To study this relationship, the authors conduct a paired evaluation comparing base language models against their instruction-tuned counterparts across three model families (Qwen2.5-7B, Mistral-7B, and Llama-3.1-8B) and three multiple-choice reasoning benchmarks (ARC-Easy, MMLU, and CommonsenseQA). Model confidence is evaluated through both likelihood-based answer entropy and a two-stage elicited verbalized confidence protocol. Rationale lexical diversity is measured by sampling five chain-of-thought rationales per question using two metrics: the Unique-2 bigram ratio for surface-level richness and 1-SelfBLEU for cross-rationale variability. Furthermore, the authors perform controlled comparisons restricted to instances where base and instruction-tuned variants select the same answer and have matched rationale lengths.
The evaluation reveals that instruction tuning consistently increases model confidence—reflected by lower answer entropy and significantly higher verbalized confidence—despite yielding limited or no improvements in predictive accuracy. In contrast, the effect on lexical diversity is non-uniform: cross-rationale diversity (1-SelfBLEU) consistently decreases across all models and benchmarks, whereas surface-level lexical diversity (Unique-2) varies in both direction and magnitude depending on the model and task. Moreover, controlled analyses confirm that these diversity shifts persist even after filtering for identical answer choices and matching rationale lengths, demonstrating that confidence and lexical diversity capture distinct facets of post-training modification.
These findings highlight a potential risk in downstream applications: instruction-tuned models can exhibit high confidence and lower cross-rationale variability without being more accurate, which may encourage unwarranted reliance on incorrect predictions. The results suggest that practitioners should assess model reliability using multiple complementary metrics rather than relying on confidence proxies alone, and they motivate future research into how alignment techniques shape the semantic and structural variety of model reasoning.
Instruction-tuned language models achieve strong performance across a range of generation tasks, but have also recently been shown to exhibit verbalized overconfidence. In question answering, verbalized model overconfidence may be associated with the consistency of the generated supporting rationales. In this paper, we study whether corresponding changes in the lexical diversity of generated answer rationales accompany changes in model confidence induced by instruction tuning. We evaluate three matched base and instruction-tuned models across question-answering benchmarks and find that instruction tuning consistently alters answer confidence, despite limited changes in predictive accuracy and decreases in likelihood-based calibration. Secondly, we observe a non-uniform effect of instruction tuning on rationale diversity: cross-rationale diversity consistently decreases, whereas surface-level lexical diversity varies in both direction and magnitude across models and benchmarks. Finally, we find that these differences persist after controlling for answer selection and rationale length, confirming that confidence and rationale diversity capture distinct effects of instruction tuning.
Sam: So a narrow spread means the model is very sure, and a wide spread means it's genuinely uncertain — and the paper finds that tuning makes the spread narrow dramatically.
Alex: That's what the data shows. They also test a second kind of confidence: they directly ask the model to state, in words, how likely it thinks it is to be correct. Both measures point the same direction — instruction tuning makes models systematically more confident.
Sam: But here's the key question. Does more confidence actually mean the model is better calibrated? Meaning, does its stated certainty actually match how often it's right?
Alex: This is where the paper's most significant finding comes in. Think of a weather forecaster who says "seventy percent chance of rain" and, over hundreds of forecasts, it actually rains about seventy percent of the time — that's good calibration. A poorly calibrated forecaster might say "ninety percent chance of rain" but only be right half the time. The paper finds that after instruction tuning, that calibration actually gets worse, even as the model sounds more confident.
Sam: So the model gets louder and surer, but the gap between how right it thinks it is and how often it actually is right widens.
Alex: That is what the data indicates. And the researchers go one step further — they also look at whether this confidence shift changes the variety of the model's supporting reasons.
Sam: You mean whether it gives genuinely different explanations, or just repeats the same basic argument in slightly different words?
Alex: Precisely. And the answer is complicated. Lower uncertainty doesn't automatically mean less variety in reasoning — the relationship depends on the specific model being tested. There's no single universal rule.
Sam: So two models could both become more confident after tuning, but one might still offer a range of different justifications while the other just circles back to the same point.
Alex: That's right. The researchers also noticed that the model's explanations got longer after tuning across every model they tested. But longer doesn't necessarily mean richer. More words can just mean more repetition. So they were careful to separate length from actual substance before drawing conclusions about the quality of the reasoning.
Sam: That's an important distinction. You don't want to mistake verbosity for depth.
Alex: One honest limitation the authors flag is that this work focuses strictly on English-language, multiple-choice question formats. It's an open question whether these same patterns would appear in other languages, or in tasks where there isn't a fixed set of answer options to choose from.
Sam: So the findings are meaningful, but they're a starting point rather than a complete picture.
Alex: The authors suggest that future training approaches might try to achieve two things at once: keeping the variety of reasoning broad, and keeping confidence levels honest. Right now, the coaching process that makes models more helpful seems to also make them less aware of their own uncertainty — and that's a tension worth taking seriously.
Sam: It's a bit like training someone to always sound confident in a job interview. It might make them seem more capable, but it doesn't actually make them more capable. And in high-stakes situations, that gap between performance and reality is exactly where things go wrong.
Alex: That's a fitting way to close. The paper doesn't argue that instruction tuning is harmful — it's clearly useful. But it does suggest that the field needs better tools for making sure models know what they don't know. Thanks for listening to ResearchPod.