Shiding Zhu, Yudi Qi, Yajie Wang, Jiaze Li, Chao Song, Yaorui Shi, Yibo Miao, Hanqi Gao, Kai Zhang
4 min
How can large language model (LLM) agents reliably learn from past experiences in open-world environments without falling into the Self-Confirmation Trap? The authors investigate why current single-agent memory systems often fail: they allow agents to evaluate their own performance, leading to the storage and reinforcement of internally consistent but objectively incorrect reasoning trajectories.
The authors propose the Execute-Distill-Verify (EDV) framework, which decouples the experience learning process into three distinct stages:
This design shifts experience construction from an isolated, error-prone loop to a collaborative filtering process. The system also maintains an 'Ability Matrix' to route new tasks to the most suitable model based on historical performance.
Experimental results across three long-horizon benchmarks (tau2-bench, Mind2Web, and MMTB) demonstrate that EDV consistently outperforms strong baselines, including single-agent memory methods and ensemble-based routing strategies. The study confirms that the Self-Confirmation Trap is a significant source of error in existing agentic systems and that decoupling execution from validation is essential for robust, long-term self-evolution. By suppressing noisy or incorrect content before it enters long-term memory, EDV enables more reliable and accurate decision-making in complex, multi-step tasks.
As LLM agents are increasingly deployed in persistent, open-world settings, their ability to learn from experience is critical. This paper provides a practical, scalable solution to the 'garbage-in, garbage-out' problem in agentic memory, offering a blueprint for building more reliable, self-improving autonomous systems.
Experience-driven self-evolution is critical for large language model (LLM) agents to improve through open-world interaction. However, existing experience learning methods mostly rely on single-agent loops, where the same agent executes tasks, summarizes outcomes, and determines memory content. This setup makes agents vulnerable to the Self-Confirmation Trap: wrong-but-self-consistent trajectories are misidentified as successful experience, leading to cumulative errors during retrieval and reuse. To address this issue, we propose EDV, an Execute-Distill-Verify framework for reliable experience learning. In the Execute stage, multiple heterogeneous agents explore the same task space in parallel to generate diverse candidate trajectories. In the Distill stage, a dedicated third-party agent comparatively analyzes these trajectories to produce candidate experiences, reducing executor-centric summarization bias. In the Verify stage, the execution group validates candidates via a consensus mechanism, and only approved experiences are written into shared or private memory. By decoupling the three stages, EDV transforms experience learning from isolated self-reflection into collaborative construction, filtering erroneous and noisy content before memory insertion. We evaluate EDV on three challenging long-horizon benchmarks: tau2-bench, Mind2Web and MMTB. Results show EDV consistently outperforms strong baselines, validating that reliable experience construction is essential for robust agent self-evolution. Our code is available at https://github.com/shidingz/EDV.
Sam: That's a fair description. And it addresses something important: the quality of what an agent remembers matters as much as how much it remembers. A memory full of confidently-held wrong answers is worse than a smaller memory of verified correct ones.
Alex: But if the filter is that strict — requiring group agreement — doesn't it risk throwing out useful lessons that just happen to be a bit unusual?
Sam: That's a reasonable concern. The researchers' argument is that the diversity of the agent pool is the safeguard. Because the agents think differently from one another, something genuinely correct is more likely to earn broad agreement than something that's merely a consistent error shared by agents with similar blind spots. Agreement across a diverse group is a much stronger signal than agreement within a uniform one.
Alex: So the diversity isn't just a nice feature — it's what makes the verification step meaningful in the first place.
Sam: Right. It's not about having more data; it's about having cleaner data. The collaborative filtering approach consistently outperformed the solo approach across the tasks they tested. The paper's suggestion is that how you construct experience is just as important as how much of it you accumulate.
Alex: That's a principle that probably applies well beyond AI. The review process turns out to be just as important as the practice itself.
Sam: It does. More practice doesn't automatically mean better performance. If the feedback loop is flawed, practice just makes the errors more persistent.
Alex: Thanks for listening to ResearchPod.