GDPevo: Evaluating Agent Self-Evolution on Real Business Tasks

Agent self-evolution—the process by which an agent updates its persistent state (e.g., skills, memory, or prompts) based on experience to improve performance on future tasks—is difficult to measure reliably. Existing benchmarks often suffer from data contamination, lack of task diversity in economically valuable domains, or ambiguous train-test splits that make it unclear if performance gains are due to genuine learning or memorization.

To address this, the authors introduce GDPevo, an evolution-native benchmark focused on enterprise workflows (CRM, ERP, finance, healthcare, legal, and data-centric tasks). The core innovation is "rule hybridization," a process that decomposes complex business workflows into atomic, domain-specific rules. These rules are distributed across training tasks and recombined in held-out test tasks. This design forces agents to learn and generalize these rules rather than relying on pre-existing world knowledge.

The benchmark is generated by a fully automated pipeline that enables rapid scaling and resistance to data contamination. The pipeline follows three stages:

1. Scenario Discovery: Using existing benchmarks to identify complex, rule-based business scenarios.
2. Task Group Generation: Creating a shared environment and 10 related tasks (5 training, 5 test) per group using the rule hybridization method.
3. Calibration and Review: Using independent agents to ensure tasks are neither too easy (solvable via base knowledge) nor too hard, and that they provide measurable headroom for improvement.

This automation allowed the authors to scale from 120 tasks (V1) to 240 tasks (V2) in just two days.

The authors evaluated four agents under four supervision types (no-evolution, few-shot, reflect, and self). Key findings include:

• Consistent Improvement: Self-evolution consistently boosts accuracy, with gains of up to 16.44 percentage points compared to non-evolving baselines.
• Performance Ceiling: Despite these gains, the best agents remain significantly below the 91.6% oracle ceiling, suggesting that current self-evolution capabilities are far from fully realized.
• Transfer Dynamics: Supervision types like few-shot learning tend to overfit to source domains, whereas reflective methods (similar to reinforcement learning) demonstrate more robust transfer across different task groups.
• Efficiency: The study treats tokens, turns, and costs as first-class metrics, finding that effective evolution can improve both accuracy and resource efficiency.