Valid= Necessary: Diagnosing Latent Inefficiency in Chain-of-Thought

Chain-of-Thought (CoT) prompting has become the standard for enabling complex reasoning in Large Language Models (LLMs). However, these models often exhibit 'over-reasoning'—the generation of verbose, repetitive, or irrelevant steps that increase computational costs without improving the final answer. While existing evaluators are adept at catching logical fallacies or factual errors, they often struggle to penalize 'valid but inefficient' steps, effectively treating redundant fluff as necessary logic.

To quantify this blind spot, the authors introduce RIV-GSM8K, a diagnostic benchmark that injects five specific types of reasoning inefficiencies into standard arithmetic problems. These include simple duplication, paraphrasing, excessive decomposition, circular reasoning, and the inclusion of irrelevant facts. By testing state-of-the-art evaluators against this benchmark, the researchers demonstrate that current methods are largely insensitive to these synthetic inefficiencies, as they prioritize factual correctness over informational utility.

To address this, the authors propose CAID (Context-Aware Information Density), a training-free metric that evaluates the utility of reasoning steps based on four signals: local redundancy, global goal alignment, information density, and semantic progress. Unlike supervised evaluators, CAID does not require a reference trace. The authors validate CAID using PACE (Pruning And Compression for Efficiency), a post-hoc strategy that prunes irrelevant steps and merges verbose ones. Empirical results across GSM8K, StrategyQA, and ARC-Challenge show that PACE reduces token consumption by 31–53% while maintaining accuracy, confirming that much of the reasoning generated by LLMs is indeed 'froth' that can be removed without compromising the final result.