REWARD-FREE EVOLVING AGENTS VIA PAIRWISE VALIDATOR

Self-evolving agentic loops—systems that iteratively refine prompts or code by proposing changes and accepting them based on a quality signal—typically rely on scalar rewards. Designing these rewards is expensive, requiring domain expertise and labeled examples. This paper asks whether a frozen, training-free LLM can act as a 'pairwise validator' to replace these scalar rewards, judging whether a child candidate is better than its parent without needing explicit numerical scores.

The authors introduce a pairwise validator that compares a parent and child candidate and returns a binary verdict. They integrate this into three existing self-evolving frameworks (GEPA, ADRS, and ShinkaEvolve) using two primary configurations:

• Adaptive Focus: Replaces the gate's reward signal with the validator while retaining the engine's original parent-selection logic.
• Soft Elo: Replaces both the gate's reward and the parent-selection logic with an Elo-based rating system driven entirely by the validator's pairwise verdicts, making the entire evolution process reward-free.

The validator is prompted with a small number of in-context examples (or a periodically refreshed rubric) to guide its judgments, avoiding the need for training or fine-tuning.

The study finds that the pairwise validator is a viable, drop-in replacement for traditional scalar rewards. Across multiple agent configurations and tasks (including retrieval, instruction-following, and code evolution), the proposed methods consistently match or exceed the performance of full-reward baselines. The authors observe that the pairwise approach often exhibits a smaller validation-to-test gap than traditional methods, particularly in low-data regimes where validation sets are small and prone to noise.