Beyond Sally-Anne: Evaluating Theory of Mind in LLMs using Epistemic Schelling Points

Theory of Mind (ToM) is the ability to attribute mental states to others to explain and predict their behavior. While LLMs have shown success on static benchmarks like the Sally-Anne task, these tests are often susceptible to memorization or reliance on linguistic cues rather than genuine social reasoning. This paper introduces the Epistemic Asymmetry Schelling Task (EAST), a two-player, text-based coordination game designed to evaluate whether models can robustly navigate different epistemic states without relying on pre-trained narrative patterns.

In the EAST framework, two LLMs are assigned distinct personas and must select the same word from a list of four options without direct communication. The researchers varied the epistemic conditions to test different cognitive demands:

• Symmetric: Both players know each other's identities.
• Asymmetric: Only one player knows the other's identity.
• Zero Knowledge: Neither player knows the other's identity.

By analyzing both the success rates and the reasoning traces (the step-by-step logic models generate), the authors assessed whether models could successfully model their partner's perspective and adapt their strategy to the specific informational constraints of each condition.

The results reveal a significant capability gap across the model landscape. Only frontier-class models were able to successfully navigate the more complex Asymmetric and Zero-Knowledge conditions. The analysis of reasoning traces identified that the primary bottleneck is not a lack of linguistic ability, but rather "epistemic tracking errors." Models frequently conflated their own private knowledge with their partner's, failing to correctly account for what the other agent did or did not know. Even when prompted to use Chain-of-Thought reasoning, weaker models often produced logic that was incoherent with the provided epistemic constraints.