WHY DO LLMS STRUGGLE IN STRATEGIC PLAY? BROKEN LINKS BETWEEN OBSERVATIONS, BELIEFS, AND ACTIONS

This paper investigates why Large Language Models (LLMs) often struggle with strategic decision-making in environments involving incomplete information, such as negotiations or complex games. The authors identify two primary "broken links" in the internal decision-making process of models like Llama 3.1, Qwen3, and gpt-oss.

The authors seek to understand the gap between an LLM's internal reasoning and its external performance. While LLMs are increasingly used for strategic tasks, their failure modes remain poorly understood. By combining game theory with mechanistic interpretability, the researchers aim to determine whether these failures stem from an inability to form accurate beliefs about hidden variables or an inability to act upon those beliefs once formed.

The study uncovers two critical failures in the internal mechanisms of LLMs:

• The Observation-Belief Gap: LLMs form internal beliefs about latent game states (like an opponent's hidden strategy) that are significantly more accurate than what the models report verbally. However, these internal beliefs are brittle. They suffer from primacy and recency biases (over-weighting the beginning or end of an interaction) and lose Bayesian coherence as the interaction length increases.
• The Belief-Action Gap: Even when an LLM holds an accurate internal belief, it often fails to translate that belief into a rational, optimal action. The researchers found that steering the model's internal states toward specific beliefs had a surprisingly weak impact on final actions, suggesting a disconnect between the model's "knowledge" and its "behavior."

The researchers tested these models across three strategic environments: repeated normal-form games, Generalized Kuhn Poker, and The Chameleon (a social deduction game). They used linear probes—a technique from mechanistic interpretability—to "read" the internal representations of the models. By comparing these internal states to the models' verbal outputs and final game actions, they were able to quantify the discrepancy between what the models "know" and what they actually do.