ResearchPod Summary
Large Language Models (LLMs) are constrained by a fixed context window, limiting the amount of information they can process at once. AI agents, which accumulate long histories of user messages, tool outputs, and model responses, must use context compaction to fit this state into the LLM's window. Despite its ubiquity in tools like Claude Code and Codex, this process has lacked formal analysis, often relying on empirical heuristics that discard information without guarantees.
The authors introduce two games to model compaction strategies:
By mapping these games to communication complexity, the authors prove that context generation is theoretically superior to selection. Specifically, they show that for certain query sets, generation requires strictly less budget (fewer tokens) than selection to achieve the same error rate. This equivalence allows researchers to leverage established bounds from communication complexity to determine the theoretical limits of what information can be preserved during compaction.
The study highlights a significant gap between theoretical optimality and current industry practice. In a case study evaluating a production context compaction endpoint, the authors found that the system performed no better than random guessing on set membership queries, whereas a Bloom filter—a near-optimal data structure for this task—would have provided significantly higher accuracy within the same token budget. This suggests that current LLM-based summarization methods may be losing critical information that could be preserved using more structured, algorithmically grounded approaches.
AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.