Flout at Your Own Risk: LLMs Struggle with Pragmatic Cooperativity Under Epistemic Asymmetry

This study investigates the pragmatic reasoning capabilities of Large Language Models (LLMs) in multi-agent collaborative environments characterized by 'collaborative epistemic asymmetry' (CEA). In these scenarios, multiple agents must work together to solve a shared goal, but no single agent possesses all the necessary information. The authors formalize the relationship between Grice’s cooperative maxims and task success, defining an 'actionable' utterance as one that is both informative and goal-oriented. They utilize the CRAFT benchmark, a simulated 3D block-building task, to evaluate how well LLMs function as speakers and listeners when they must rely on partial information from their peers.

The researchers found that LLMs often struggle to act as truly pragmatic collaborators. While they can generate fluent text, their communications frequently suffer from floutings of Gricean maxims—such as providing irrelevant information or failing to be sufficiently clear—which directly correlates with lower task completion rates. The study demonstrates that while prompting strategies (e.g., instructing the model to act as a 'pragmatic listener') and post-training techniques (SFT, DPO, IPO) can elicit better performance, the models remain fragile. They often fail to recognize when an utterance is non-cooperative, leading to a breakdown in common ground that prevents the agents from effectively pooling their private knowledge.

As LLMs are increasingly integrated into agentic pipelines and collaborative workflows, their ability to communicate pragmatically becomes a critical bottleneck. This research highlights that 'fluency' is not equivalent to 'cooperativity.' If an AI agent cannot navigate epistemic asymmetry by adjusting its communication to be actionable, it will fail in complex, real-world collaborative settings where information is distributed among multiple stakeholders. The study provides a framework for measuring these failures, offering a path toward building more robust, cooperative AI systems.