Transcript: Context Language Models

[[RP_SECTION:self-editing-context-models|Self-Editing Context Models]]

Alex: [curious, leaning in] A model that edits its own context file, rather than appending to an ever-growing history, reportedly reaches over eleven percent higher accuracy on deep-research benchmarks. Sam, why has context stayed an append-only scroll for so long?

Sam: [measured, steady] Mostly because it's the default interface, and management has been external: truncation rules, summarization schedules, fixed strategies. Rulin Shao and colleagues at the University of Washington and Meta ask whether the model should control that itself. They report that intent-driven editing outperforms those fixed strategies.

Alex: [analytical, probing] So the model modifies its own workspace. What does that look like mechanically? [[RP_SECTION:mechanical-implementation-details|Mechanical Implementation Details]]

Sam: [teaching mode, precise] The model gets read-write access to a context file through Bash commands. Think of the difference between a scroll and a word processor. It can delete a dead-end or a typo, or replace a stretch of earlier turns with a summary. The context becomes something the policy acts on, not just something it reads.

Alex: [processing] That raises an infrastructure problem. Editing the middle of the context should invalidate the KV cache, and you'd pay for a re-prefill every time. [[RP_SECTION:suffix-cache-reuse|Suffix Cache Reuse]]

Sam: [nodding] That's the main systems obstacle. Under causal attention, cached states for later tokens were computed conditioned on everything before them. Change something in the middle and the standard assumption is that everything after it has to be recomputed. The authors introduce Suffix Cache Reuse, or SCR, which reuses cached states for the unchanged suffix tokens after an in-the-middle edit. They report it reduces server-side compute by thirty-five percent.

Alex: [slower, for clarity] And without it, the approach wouldn't be practical.

Sam: [grounded] That's my reading. Frequent edits would otherwise erase the efficiency the method is supposed to buy. The question I'd take to the paper is whether that reuse is exact or an approximation, and what it does to output quality. The material I have doesn't say.

Alex: [thoughtful] Is the headline benefit performance or compute? Or is it just a clever way to save compute? [[RP_SECTION:performance-and-accuracy-gains|Performance and Accuracy Gains]]

Sam: [matter-of-fact] They report both. On deep-research benchmarks they see the accuracy gain of over eleven percent, with about twenty-one percent fewer floating-point operations. Those are the load-bearing numbers. The SCR saving is the enabling systems contribution, and it's reported separately.

Context Language Models

Rulin Shao, Shannon Zejiang Shen, Junjie Oscar Yin, Yuetai Li, Minheng Wang, Hamish Ivison, Radha Poovendran, Nathan Lambert, Teng Xiao, Mike Lewis, Wen-tau Yih, Luke Zettlemoyer, Pang Wei Koh

Sep 29, 2026·4:05·arxiv·Original Paper

Context Language Models (CLMs)Context ManagementAgent-Swarm SystemsSuffix Cache ReuseSuffix Cache Reuse (SCR)ContextBench

Listen

4 min

Briefing

Research Approach

Context Language Models (CLMs) shift the paradigm of context management from external, harness-defined rules to intrinsic model behavior. By treating the context as a file, the model gains the ability to perform unrestricted edits—such as compacting information, deleting irrelevant data, or maintaining persistent trackers—using standard programming tools. This approach allows the model to learn and adapt its context-management strategy dynamically, whether through in-context instructions, textual skill evolution, or reinforcement learning.

Key Findings

CLMs demonstrate superior performance and efficiency compared to existing methods like summarization-based compaction or tool-based memory management. On the BrowseComp-Plus deep-research benchmark, CLMs achieved 11.4% higher accuracy while using 21.5% fewer FLOPs. In multi-repository agent-swarm tasks, CLMs provided a 65% greater end-to-end speedup compared to summary-based baselines. Furthermore, the researchers introduced Suffix Cache Reuse (SCR), a serving optimization that reuses cached states for unchanged context segments after an edit, further reducing server-side compute by 35%.

Learning and Adaptation

Because context management is now an intrinsic capability, CLMs can be steered using natural language. Users can provide simple instructions to guide compaction timing or semantic boundaries. Moreover, the authors demonstrate that CLMs can evolve their own context-management skills through a standard optimization loop or internalize these strategies via reinforcement learning. This allows the model to move beyond human-defined heuristics to discover more effective, task-specific ways to manage its own memory.

Why It Matters

This work suggests that context management should not be an external constraint but a core competency of intelligent agents. By letting models decide what to keep and what to discard, we can build agents that are better at long-horizon reasoning and more efficient in their resource usage. This shift aligns with the broader trend of moving away from hand-engineered harnesses toward models that learn to optimize their own execution.

Key Terms and Definitions

Context Language Model (CLM) — A language model that manages its own context by treating it as an editable file, allowing for unrestricted read and write operations.
Prefix-reuse FLOPs — A metric used to measure the computational cost of inference, accounting for the re-prefilling of tokens that occurs when context is modified in the middle of a sequence.
Suffix Cache Reuse (SCR) — A serving optimization that reuses cached key-value states for unchanged text segments following an in-the-middle context edit, avoiding redundant re-prefilling.
Success-gated efficiency advantage — A reinforcement learning objective that rewards successful agent trajectories while simultaneously penalizing higher inference costs, encouraging both accuracy and efficiency.
Context pressure — A measure of the ratio between the total input volume processed over an episode and the model's fixed context window limit.

Abstract

We introduce Context Language Models (CLMs), language models that natively manage their own context. We implement this by treating the context as a file and allowing the model to make unrestricted updates to this file. This allows the model to learn what is most important to maintain in context, and naturally extends to multi-agent systems where multiple agent contexts coexist as files. Building CLMs zero-shot with existing models outperforms SOTA context management strategies across a variety of tasks: 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus, 5% higher scores with 59% fewer FLOPs on 12-hour EdgeBench, and 65% greater improvement with the same compute on a 24-hour multi-repository agent-swarm task. Moreover, by shifting context management from external harness control to intrinsic model behavior, CLMs naturally enable both in-context and parametric learning of context-management strategies. We show that CLMs can be steered with natural-language instructions evolved through a standard skill-optimization loop, improving held-out accuracy by up to 35.9 points on a context-management task while reducing compute. We also introduce an online reinforcement learning method for CLMs, improving Qwen3.5-9B performance on BrowseComp-Plus by 47.6% while using 12% fewer FLOPs. Finally, we co-design Suffix Cache Reuse for CLM serving, further reducing server-side compute by 35% relative to standard SGLang at matched performance.

Context Language Models | Rulin Shao et al. | ResearchPod

Context Language Models: Research Paper Podcast