Large language models can generate long chain-of-thought (CoT) reasoning, but it remains unclear whether the verbalized steps reflect the models' internal thinking. In this work, we propose a True Thinking Score (TTS) to quantify the causal contribution of each step in CoT to the model's final prediction. Our experiments show that LLMs often interleave between true-thinking steps (which are genuinely used to compute the final output) and decorative-thinking steps (which give the appearance of reasoning but have minimal causal influence). We reveal that only a small subset of the total reasoning steps causally drive the model's prediction: e.g., on AIME, only an average of 2.3% of reasoning steps in CoT have a TTS >= 0.7 (range: 0-1) for Qwen-2.5. Furthermore, we find that LLMs can be steered to internally follow or disregard specific steps in their verbalized CoT using the identified TrueThinking direction. We highlight that self-verification steps in CoT (i.e., aha moments) can be decorative, while steering along the TrueThinking direction can force internal reasoning over these steps. Overall, our work reveals that LLMs often verbalize reasoning steps without performing them internally, challenging the efficiency of LLM reasoning and the trustworthiness of CoT.
Alex: Welcome to another episode of ResearchPod. Sam, we've got an intriguing paper here—walk us through what it's about.
Sam: This paper, titled "Can Aha Moments Be Fake? Identifying True and Decorative Thinking Steps in Chain-of-Thought," comes from researchers including Jiachen Zhao. They tackle a key puzzle in AI: large language models generate long chains of step-by-step reasoning before answering, but do those steps actually shape the final answer, or are many just for show?
Alex: So this is basically asking whether these AI models truly think through detailed reasoning chains like we do, or if they're mostly mimicking the look of human verbosity to seem smarter?
Sam: Exactly. The researchers call these full reasoning sequences "chain-of-thought," where the model writes out intermediate thoughts—like listing steps to solve a math problem before giving the answer. But their work suggests many of those steps don't genuinely influence what the model outputs inside. Imagine a student jotting down elaborate essay paragraphs but skipping the actual calculations needed. Only a tiny portion, around 2.3% on average in their tests, truly drives the prediction.
Alex: That student analogy hits home—like fooling a teacher with words but not the work. So the core problem is figuring out which steps are real thinking versus decorative filler?
Sam: Yes, and that's worth paying attention to because if most steps are superficial, it questions how trustworthy these models are for real reasoning tasks. They introduce a score to measure each step's real causal impact, separating the load-bearing ones from the rest. This opens up whether things like "aha moments"—those self-checks where the model seems to verify itself—are often fake too.
Alex: It's a notable reminder that even smart systems can prioritize appearance over substance. Thanks for breaking it down, Sam. Thanks for listening to ResearchPod.