Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents

Modern LLM agents are increasingly tasked with long-horizon operations that exceed their context windows. To manage this, developers use context compaction—a process that summarizes or evicts older conversation turns to stay within token limits. This paper demonstrates that this mechanism is a significant, silent safety-failure surface. When an agent is governed by in-context constraints (such as organizational policies or memory entries), the compaction process often treats these rules as low-salience information and discards them. The result is that an agent that correctly refuses a prohibited action early in a session will perform that same action later, simply because the rule it was following has been erased from its active memory.

To quantify this, the author introduces the ConstraintRot benchmark, which uses deterministic grading to measure how often agents violate safety policies after compaction. Across seven model families, the study found that compaction increases violation rates from 0% (when the policy is in full context) to an average of 30%, with some models reaching 59%. The research highlights a critical gradient: soft, organization-specific policies are 8.3 times more likely to be discarded than hard safety norms that models are already intrinsically trained to refuse. Crucially, the study shows that the failure is driven by the summarizer model, not the agent model itself; if the summary preserves the constraint, the agent continues to obey it.

The study also identifies the Compaction-Eviction Attack, where an adversary who can influence the agent's ingested data (e.g., via tool outputs or retrieved documents) can force the deletion of a policy. This can be achieved through volume-based attacks that force the system to compact, or through direct summarizer-injection attacks that instruct the compactor to omit specific rules. Even models that are robust to passive compaction often fall to these active adversarial injections, proving that context management is a first-class security surface.