HarnessEvolve: Learning from Reference Trajectories for Reliable Agent Self-Evolution

How can autonomous agents reliably improve their own internal logic—including prompts, skills, tools, and execution workflows—without human intervention, while avoiding common pitfalls like credit assignment failure, shortcut learning, and catastrophic forgetting?

HarnessEvolve introduces a decoupled, multi-agent architecture that separates the execution agent from the evolutionary pipeline. The framework utilizes four distinct modules:

• Execution Agent: Performs tasks and generates trajectories.
• Evaluation Agent: Assesses accuracy and verifies the quality of reference trajectories.
• Optimization Agent: Compares failed execution paths against verified reference trajectories to identify the first point of divergence, allowing for root-cause diagnosis. It then clusters these errors to generate systematic harness updates.
• Gate Agent: Enforces two critical safeguards: a quality gate that filters for data leakage and prompt bloat, and a performance gate that ensures updates improve current batch performance without degrading historical competence.

By comparing failed executions against reference trajectories (paths generated using ground-truth answers), the system overcomes the ambiguity of sparse, terminal feedback. The framework further stabilizes the evolution process by maintaining a pool of agent snapshots and selecting the best performer on a held-out validation set at the end of each epoch.

HarnessEvolve consistently outperforms state-of-the-art baselines across five diverse benchmarks, including open-domain tasks like SearchQA and specialized enterprise tasks like CloudCoreNetwork-QA. On the latter, the framework achieved a 21.6 percentage point improvement over the strongest baseline. Ablation studies confirm that reference-guided error diagnosis is the most significant contributor to performance gains, followed by error clustering and the quality gate. The optimized harnesses demonstrate broad improvements across prompts, skills, and execution logic, rather than just isolated component tweaks.