Self-Evolving Agents with Anytime-Valid Certificates

How can we enable LLM agents to self-evolve—improving their own prompts, tools, and libraries—without falling into the failure modes of self-referential training, such as catastrophic forgetting, reward hacking, or distribution collapse? The paper addresses the endogenous-loop failure mode, where the agent produces the very data and evaluations it uses to update its own policy, effectively voiding the guarantees of classical learning theory.

SEA (Self-Evolving Agents) introduces a four-layer architecture that separates the frozen base model from the mutable components. The system uses five loop controllers (ALG 1–ALG 5) that act as anytime-valid gates, ensuring that every self-modification is statistically justified against a global error budget. To overcome the lack of dense signals in sparse, binary-reward environments, the authors implement a verifier-in-the-loop engine (ALG 6–ALG 10). This engine includes self-authored reproduction oracles that generate grader-free feedback from the issue text alone, allowing the agent to debug and refine its patches mid-episode rather than relying on a single terminal evaluation.

Testing on a 52-instance subset of SWE-bench Verified across four base models, the authors found that base model capability is the primary driver of performance. However, the SEA suite consistently provided a measurable lift across all models. By using a no-op composite control to isolate the algorithm suite from the scaffolding, the authors demonstrated a deconfounded improvement of +4 to +5 resolved instances. Event logs confirm that the mechanisms successfully fire and prevent regressions, with the directive-learning controllers (ALG 2 and ALG 3) and the verified search engine (ALG 7 and ALG 8) providing the most significant performance gains.

This work provides a rigorous framework for agentic self-improvement that prioritizes safety and auditability. By replacing anecdotal self-debugging with statistically gated, verifier-in-the-loop mechanisms, SEA offers a path toward agents that can reliably grow their own skill libraries and repair their own harnesses without requiring constant human intervention or risking the stability of the underlying model.