Evaluating Theory of Mind in Reasoning Models: Robustness over Reasoning

This paper investigates whether the improved performance of modern 'reasoning' LLMs (models trained with reinforcement learning to produce chain-of-thought) on Theory of Mind (ToM) tasks represents a genuine advancement in social-cognitive reasoning or simply a byproduct of increased robustness. The authors evaluate several state-of-the-art reasoning models using a battery of established psychological tests, including Sally-Anne tasks, Strange Stories, and Imposing Memory tests. To isolate the effect of robustness, they introduce novel prompt perturbations and compare performance against non-reasoning models and baseline results from previous years.

Reasoning models demonstrate near-perfect performance across most standard ToM benchmarks, significantly outperforming models from 2023. However, the authors argue that these gains are not evidence of a new, specialized ToM ability. Instead, the models show a high degree of stability when faced with task-preserving prompt variations. The analysis suggests that the 'reasoning' process—often characterized by inference-time scaling—functions as a mechanism for maintaining consistency and finding correct solutions under diverse conditions, rather than a fundamental shift in how the model represents mental states. The authors observe that when models do fail, it is often in scenarios requiring complex mental visualizations or nuanced social interpretations that go beyond simple logical deduction.

As LLMs are increasingly deployed in social and agentic settings, understanding whether they possess genuine Theory of Mind is critical for safety and reliability. This research cautions against over-interpreting high benchmark scores as evidence of human-like social intelligence. By framing these improvements as a result of 'robustness over reasoning,' the authors provide a more grounded framework for evaluating AI capabilities, suggesting that current progress is driven by better optimization of existing knowledge rather than the emergence of new cognitive faculties.