Instruction-tuned language models achieve strong performance across a range of generation tasks, but have also recently been shown to exhibit verbalized overconfidence. In question answering, verbalized model overconfidence may be associated with the consistency of the generated supporting rationales. In this paper, we study whether corresponding changes in the lexical diversity of generated answer rationales accompany changes in model confidence induced by instruction tuning. We evaluate three matched base and instruction-tuned models across question-answering benchmarks and find that instruction tuning consistently alters answer confidence, despite limited changes in predictive accuracy and decreases in likelihood-based calibration. Secondly, we observe a non-uniform effect of instruction tuning on rationale diversity: cross-rationale diversity consistently decreases, whereas surface-level lexical diversity varies in both direction and magnitude across models and benchmarks. Finally, we find that these differences persist after controlling for answer selection and rationale length, confirming that confidence and rationale diversity capture distinct effects of instruction tuning.
Alex: Welcome to another episode of ResearchPod.
Sam: Today we're looking at how training AI models to follow instructions changes how sure they sound of their answers.
Alex: Specifically, the paper investigates whether this training process makes models more confident without actually making them more accurate.
Sam: So the core question is: if an AI sounds completely certain of itself, does that actually mean it's more likely to be right?
Alex: That's exactly it. And the short answer the paper suggests is: not necessarily. The study examines how this training process changes both a model's expressed certainty and the variety of reasoning it offers to back up its answers.
Sam: That matters a lot in fields like medicine or finance, where an AI assistant could give a wrong answer while sounding completely convincing.
Alex: To understand what the researchers found, it helps to know the two types of systems they're comparing. The first is what they call a "base model." Think of it as a system that has read an enormous amount of text from the internet and learned to predict what word comes next. It's powerful, but it doesn't really know how to have a conversation or follow instructions.
Sam: So it's like a very well-read person who has never actually spoken to anyone.
Alex: That's a good way to put it. The second type is what they call an "instruction-tuned model." This is a base model that has been given additional coaching — shown thousands of examples of good, helpful replies — so it learns to behave like a useful assistant.
Sam: And that coaching does help. The models get better at answering questions. But the paper suggests something else is also happening underneath?
Alex: Right. When you coach a model to sound helpful and decisive, you're also changing something more fundamental: the way it distributes its internal confidence across possible answers. Before tuning, a model might assign a moderate probability to one option, a slightly lower one to another, and a small but real chance to the remaining options. After tuning, it tends to pile almost everything onto a single answer and assign near-zero probability to the rest.
Sam: Like the difference between a student who says "I'm pretty sure it's A, but B is also plausible" versus one who says "It's definitely A, full stop."
Alex: Exactly. And to measure that shift, the researchers look at what you might call the "spread" of probabilities across all candidate answers. When the spread is wide, the model is hedging its bets. When it collapses to a single answer, the model is expressing high confidence. They track how this spread changes before and after instruction tuning.
Sam: So a narrow spread means the model is very sure, and a wide spread means it's genuinely uncertain — and the paper finds that tuning makes the spread narrow dramatically.
Alex: That's what the data shows. They also test a second kind of confidence: they directly ask the model to state, in words, how likely it thinks it is to be correct. Both measures point the same direction — instruction tuning makes models systematically more confident.
Sam: But here's the key question. Does more confidence actually mean the model is better calibrated? Meaning, does its stated certainty actually match how often it's right?
Alex: This is where the paper's most significant finding comes in. Think of a weather forecaster who says "seventy percent chance of rain" and, over hundreds of forecasts, it actually rains about seventy percent of the time — that's good calibration. A poorly calibrated forecaster might say "ninety percent chance of rain" but only be right half the time. The paper finds that after instruction tuning, that calibration actually gets worse, even as the model sounds more confident.
Sam: So the model gets louder and surer, but the gap between how right it thinks it is and how often it actually is right widens.
Alex: That is what the data indicates. And the researchers go one step further — they also look at whether this confidence shift changes the variety of the model's supporting reasons.
Sam: You mean whether it gives genuinely different explanations, or just repeats the same basic argument in slightly different words?
Alex: Precisely. And the answer is complicated. Lower uncertainty doesn't automatically mean less variety in reasoning — the relationship depends on the specific model being tested. There's no single universal rule.
Sam: So two models could both become more confident after tuning, but one might still offer a range of different justifications while the other just circles back to the same point.
Alex: That's right. The researchers also noticed that the model's explanations got longer after tuning across every model they tested. But longer doesn't necessarily mean richer. More words can just mean more repetition. So they were careful to separate length from actual substance before drawing conclusions about the quality of the reasoning.
Sam: That's an important distinction. You don't want to mistake verbosity for depth.
Alex: One honest limitation the authors flag is that this work focuses strictly on English-language, multiple-choice question formats. It's an open question whether these same patterns would appear in other languages, or in tasks where there isn't a fixed set of answer options to choose from.
Sam: So the findings are meaningful, but they're a starting point rather than a complete picture.
Alex: The authors suggest that future training approaches might try to achieve two things at once: keeping the variety of reasoning broad, and keeping confidence levels honest. Right now, the coaching process that makes models more helpful seems to also make them less aware of their own uncertainty — and that's a tension worth taking seriously.
Sam: It's a bit like training someone to always sound confident in a job interview. It might make them seem more capable, but it doesn't actually make them more capable. And in high-stakes situations, that gap between performance and reality is exactly where things go wrong.
Alex: That's a fitting way to close. The paper doesn't argue that instruction tuning is harmful — it's clearly useful. But it does suggest that the field needs better tools for making sure models know what they don't know. Thanks for listening to ResearchPod.