Yifan Wu, Lizhu Zhang, Yuhang Zhou, Mingyi Wang, Bo Peng, Serena Li, Xiangjun Fan, Zhuokai Zhao
5 min
In long-horizon tasks, decision-relevant state is often scattered across an expanding trajectory, while the action agent must surface it and act. As trajectories grow, task requirements, environment facts, prior attempts, diagnoses, and open subgoals can be buried in the context window or pushed beyond it, failing to influence decisions when needed. We call this failure mode "behavioral state decay". We study memory as an active intervention mechanism rather than passive retrieval. A separate memory agent runs alongside an unmodified action agent, updating a structured memory bank from the recent trajectory and deciding whether to inject a memory-grounded reminder or remain silent. The module is plug-and-play with frontier action agents and existing agent harnesses. Across Terminal-Bench 2.0 and $τ^2$-Bench, it improves pass@1 for both weaker and stronger action agents, with gains of +8.3 pp on Terminal-Bench and +6.8 pp on $τ^2$-Bench. Ablations show that selective intervention outperforms passive bank exposure, always-on injection, advisor-only guidance, and general retrieval. As an early step toward open-weight memory policies, we train Qwen3.5-27B on SETA using SFT and GRPO, improving validation reward and achieving partial transfer to Terminal-Bench.
As language agents tackle increasingly long-horizon tasks, they often suffer from "behavioral state decay," where critical information—such as task requirements, environment facts, or previous diagnostic findings—is lost or ignored as the trajectory grows. The authors investigate whether memory should be treated as an active intervention mechanism rather than a passive retrieval system, asking: can a separate memory agent effectively decide when to inject specific, grounded reminders into an action agent's decision loop?
The researchers propose a two-phase memory intervention architecture. A memory agent runs in parallel with an unmodified action agent. At regular intervals, the memory agent performs two tasks:
This design decouples memory maintenance from action selection, allowing the system to avoid the pitfalls of "always-on" context injection, which can distract the agent or consume unnecessary tokens.
Experiments on Terminal-Bench 2.0 and τ²-Bench demonstrate that this proactive intervention consistently improves pass@1 performance for both weaker and stronger action agents. Specifically, the method achieved gains of +8.3 percentage points on Terminal-Bench and +6.8 percentage points on τ²-Bench. Ablation studies confirm that selective, proactive intervention outperforms passive memory exposure, general retrieval, and advisor-only guidance models. The authors also show that this intervention policy can be partially distilled into an open-weight model (Qwen3.5-27B) using supervised fine-tuning and reinforcement learning.
This work shifts the focus of agent memory from "what to store" to "when to intervene." By treating memory as a control-loop intervention, the authors provide a scalable way to improve long-horizon performance without needing to retrain the underlying action agent. This plug-and-play architecture offers a practical path for enhancing the reliability of autonomous agents in complex, multi-step environments.
Alex: That's exactly the right concern, and it's where the training method becomes important. They use an approach called Group Relative Policy Optimization—GRPO for short. Here's the intuition: instead of just telling the memory agent what the right answer is, you show it a whole group of different choices it could have made, and you let it compare the outcomes. Over time, it learns to recognize the specific moments when an intervention actually helps, versus when it would just be in the way. It's learning the timing, not just the content.
Sam: Like learning when to speak up in a group project. Jumping in every five seconds is annoying, but staying silent when someone's about to make a big error is also a problem.
Alex: That's a good way to put it. And the two-phase workflow is what makes this practical. First, the memory agent updates its bank—making sure its information is current. Then, and only then, it decides whether to intervene. Those two steps are kept separate, which means the agent isn't trying to refresh its memory and make a judgment call at the same time.
Sam: So the filing cabinet gets updated before anyone consults it. That way the advice is never stale.
Alex: Correct. And because the categories are structured—status, knowledge, procedures—the agent doesn't have to sort through a messy pile of data to find what's relevant. It knows exactly where to look.
Sam: Are there downsides? Does the agent ever get confused by unnecessary advice?
Alex: The authors are candid about this. The system does sometimes misjudge—raising a concern that wasn't actually needed, which triggers redundant verification steps. It can be a bit too eager to help.
Sam: So the calibration isn't perfect yet.
Alex: Not yet. The system currently checks its memory on a fixed schedule. The authors see the next step as teaching the agent to decide when to check, rather than checking at set intervals. Moving from a fixed routine to a genuine "check only when something feels off" judgment would make these agents considerably more efficient.
Sam: It's a disciplined approach to a real problem. Instead of letting the agent muddle through, you give it a dedicated supervisor that knows when to step in—and crucially, when to stay quiet.
Alex: And that distinction matters more than it might seem. For long-running tasks where staying on target is the hardest part, the paper suggests that memory is really an intervention problem—not a storage problem. Thanks for listening to ResearchPod.