Irina Proskurina, Mayank Kumar, Oyindolapo Komolafe
6 min
Abstract
Instruction-tuned language models achieve strong performance across a range of generation tasks, but have also recently been shown to exhibit verbalized overconfidence. In question answering, verbalized model overconfidence may be associated with the consistency of the generated supporting rationales. In this paper, we study whether corresponding changes in the lexical diversity of generated answer rationales accompany changes in model confidence induced by instruction tuning. We evaluate three matched base and instruction-tuned models across question-answering benchmarks and find that instruction tuning consistently alters answer confidence, despite limited changes in predictive accuracy and decreases in likelihood-based calibration. Secondly, we observe a non-uniform effect of instruction tuning on rationale diversity: cross-rationale diversity consistently decreases, whereas surface-level lexical diversity varies in both direction and magnitude across models and benchmarks. Finally, we find that these differences persist after controlling for answer selection and rationale length, confirming that confidence and rationale diversity capture distinct effects of instruction tuning.
Sam: So a narrow spread means the model is very sure, and a wide spread means it's genuinely uncertain — and the paper finds that tuning makes the spread narrow dramatically.
Alex: That's what the data shows. They also test a second kind of confidence: they directly ask the model to state, in words, how likely it thinks it is to be correct. Both measures point the same direction — instruction tuning makes models systematically more confident.
Sam: But here's the key question. Does more confidence actually mean the model is better calibrated? Meaning, does its stated certainty actually match how often it's right?
Alex: This is where the paper's most significant finding comes in. Think of a weather forecaster who says "seventy percent chance of rain" and, over hundreds of forecasts, it actually rains about seventy percent of the time — that's good calibration. A poorly calibrated forecaster might say "ninety percent chance of rain" but only be right half the time. The paper finds that after instruction tuning, that calibration actually gets worse, even as the model sounds more confident.
Sam: So the model gets louder and surer, but the gap between how right it thinks it is and how often it actually is right widens.
Alex: That is what the data indicates. And the researchers go one step further — they also look at whether this confidence shift changes the variety of the model's supporting reasons.
Sam: You mean whether it gives genuinely different explanations, or just repeats the same basic argument in slightly different words?
Alex: Precisely. And the answer is complicated. Lower uncertainty doesn't automatically mean less variety in reasoning — the relationship depends on the specific model being tested. There's no single universal rule.
Sam: So two models could both become more confident after tuning, but one might still offer a range of different justifications while the other just circles back to the same point.
Alex: That's right. The researchers also noticed that the model's explanations got longer after tuning across every model they tested. But longer doesn't necessarily mean richer. More words can just mean more repetition. So they were careful to separate length from actual substance before drawing conclusions about the quality of the reasoning.
Sam: That's an important distinction. You don't want to mistake verbosity for depth.
Alex: One honest limitation the authors flag is that this work focuses strictly on English-language, multiple-choice question formats. It's an open question whether these same patterns would appear in other languages, or in tasks where there isn't a fixed set of answer options to choose from.
Sam: So the findings are meaningful, but they're a starting point rather than a complete picture.
Alex: The authors suggest that future training approaches might try to achieve two things at once: keeping the variety of reasoning broad, and keeping confidence levels honest. Right now, the coaching process that makes models more helpful seems to also make them less aware of their own uncertainty — and that's a tension worth taking seriously.
Sam: It's a bit like training someone to always sound confident in a job interview. It might make them seem more capable, but it doesn't actually make them more capable. And in high-stakes situations, that gap between performance and reality is exactly where things go wrong.
Alex: That's a fitting way to close. The paper doesn't argue that instruction tuning is harmful — it's clearly useful. But it does suggest that the field needs better tools for making sure models know what they don't know. Thanks for listening to ResearchPod.