What LLM Forecasters Know but Don’t Say: Probing Internal Representations for Calibration and Faithfulness

This paper investigates whether the internal activations of Large Language Models (LLMs) provide a more reliable window into their forecasting confidence and reasoning fidelity than their explicit, verbalized outputs. The authors focus on the discrepancy between what a model says (its chain-of-thought reasoning and verbalized confidence) and what it internally encodes. Using the Eternis-Forecaster 8B model and several GLM variants, the researchers train lightweight linear probes on intermediate activations to predict forecast correctness and confidence. They also employ causal perturbation techniques—specifically evidence ablation and diversionary injection—to test whether the model's reasoning trace changes when the underlying evidence for a forecast is manipulated.

The study demonstrates that internal probes consistently outperform verbalized confidence in calibration, achieving significantly lower Expected Calibration Error (ECE). The researchers find that chain-of-thought (CoT) reasoning is often unfaithful: when influential evidence is removed from the prompt, the model's forecast frequently changes while the reasoning trace remains static. Crucially, the internal probes act as effective lie detectors; they track these behavioral shifts even when the CoT conceals the influence of the perturbation. Furthermore, the authors show that a model's forecast and confidence are largely determined before the reasoning process begins. By performing a single pre-reasoning pass, they can route questions based on the pre-set answer distribution, saving 30–47% of generated tokens without sacrificing accuracy.

These findings suggest that current methods for auditing LLM reasoning—which rely heavily on reading the generated text—are fundamentally limited. By shifting the focus to internal representations, researchers and developers can build more robust tools for auditing, calibrating, and optimizing reasoning models. The ability to detect when a model is being unfaithful or overconfident, independent of its verbalized output, provides a critical path toward more trustworthy AI systems in high-stakes domains like geopolitical and public health forecasting.