Sihang Jiang, Lipeng Ma, Zhonghua Hong, Keyi Wang, Zhiyu Lu, Shisong Chen, Jinghao Zhang, Tianjun Pan, Weijia Zhou, Jiaqing Liang, Yanghua Xiao
11 min
Traditional LLM-based agents excel at single, isolated tasks—like solving a math problem or writing code—but they hit a wall in real-world scenarios. They suffer from episodic amnesia, resetting their memory after each task, and rely on static toolsets that can't adapt. Enter Self-Evolving Agents (SEAs): a new paradigm for agents that operate in open-ended environments with complex interactions (e.g., raw GUIs, dynamic web pages, system interfaces) and evolve continuously across tasks via persistent memory.
SEAs treat tasks as part of a continuous multi-turn evolution, turning past experiences into cognitive priors that improve future performance. This mimics human learning: you don't forget how to use a tool after one use; you refine it over time.
SEAs are built around the Evolutionary Flywheel, a minimal closed-loop system with two interdependent hubs:
These hubs form a cycle: execution generates data → distillation (reflection) extracts insights → augmented execution applies them. When it spins well, execution efficiency (token consumption) converges monotonically—getting cheaper with experience. When it fails, you get pseudo-evolution: the agent fakes progress via zero-shot reasoning, with costs scaling linearly.
This dual-hub architecture recognizes their symbiosis—memory needs reflection from execution, execution needs memory to be smart. It's the minimal condition for true evolution.
Existing agent benchmarks use episodic assessment: isolated tasks with memory resets. This hides key SEA traits like cross-task learning and efficiency gains. Result? A capability illusion—two agents with similar success rates (SR) might differ wildly: one truly evolves (costs drop), the other pseudo-evolves (costs rise).
Issues include evolutionary bottlenecks (e.g., unhandled states where tools fail, needing autonomous tool synthesis), episodic amnesia, and failure to measure implicit user alignment (encoding user preferences into memory over interactions). Benchmarks miss stability, evolutionary gain, and convergence.
SEA-Eval is the first benchmark tailored for SEAs, using sequential task streams (chains of related tasks) to measure:
From these, derive evolutionary gain (improving SR/T), stability (consistent across streams), and alignment convergence (adapting to user via stratified memory and feedback).
Empirically, token use varies 31.2× at same SR across frameworks, exposing pseudo-evolution. SEA-Eval demands closed-loop evolution for monotonic T convergence—true growth, not illusion.
SEAs push toward digital embodiment: agents as functional actors in digital worlds, enabling industrial tasks like software engineering or workflow orchestration. SEA-Eval ensures we build agents that genuinely learn, not just brute-force. It formalizes evolution, guides architecture (e.g., digital embodiment, continuous evolution), and reveals failures like unhandled states requiring new tools.
Current LLM-based agents demonstrate strong performance in episodic task execution but remain constrained by static toolsets and episodic amnesia, failing to accumulate experience or optimize strategies across task boundaries. While the Self-Evolving Agent (SEA) paradigm has been previously proposed, this paper contributes a new formal definition of SEA grounded in digital embodiment and continuous cross-task evolution, and introduces SEA-Eval, the first benchmark designed to evaluate SEA characteristics across two dimensions, intra-task execution reliability and long-term evolutionary performance. By organizing tasks into sequential streams and analyzing Success Rate and Token Consumption over time, SEA-Eval quantifies evolutionary gain and structural stability in ways that existing episodic benchmarks cannot. Empirical evaluations reveal a significant evolutionary bottleneck in current state-of-the-art frameworks, where identical success rates mask up to 31.2 times differences in token consumption and divergent evolutionary trajectories under sequential analysis. SEA-Eval provides a rigorous scientific foundation for advancing agents from mere task executors toward genuinely self-evolving digital entities.
Alex: So one part is like the eyes, hands, and body moving in the digital world... and the other is the planning brain that learns from it all?
Sam: Yes. The action side—they name it the Execution Hub—also builds brand-new tools on the fly when it hits a roadblock, like a situation where none of the current options fit what needs doing. It makes a fresh tool, tests it, and adds it to the toolkit for later. The thinking side, the Cognition Hub, sorts memories into layers: step-by-step how-tos from past successes, stories of what went wrong before, and notes on rules or what a specific user likes. This setup creates a loop where doing stuff feeds learning, and learning makes doing easier next time.
Alex: Wait—building new tools automatically? That sounds key for open-ended stuff, but what keeps the memory from getting cluttered with junk?
Sam: Good question. After each task, the Cognition Hub boils down the full record of what happened—what it saw, did, and the results—into clean takeaways, then picks only the broadly useful ones to store long-term, ditching one-off noise. Over many back-and-forth turns with the same user, it also picks up on their habits and fixes from feedback, quietly tuning its style to match without special training. This ongoing tweak, called implicit user alignment, happens naturally as part of the loop.
Alex: Huh. So it's not just repeating tasks better—it's growing its own toolbox and getting personalized through real use.
Sam: Exactly. These traits aren't add-ons; they're what any agent must do to truly evolve in messy, changing digital spaces, turning raw trial-and-error into lasting smarts. The paper lays this out as the bare minimum structure needed.
Alex: So that dual-hub loop sounds solid for building smarts over time. But to spot real evolution from fakes, what exactly do they track in these task chains?
Sam: They break evaluation into two main layers. First is reliability within a single task—how steadily the agent handles surprises like website changes or errors without freezing up or needing human fixes. The second layer checks long-term growth over many tasks: does performance get better and more efficient as repeats happen, and does it stay steady without messing up old skills?
Alex: Okay, single-task steadiness makes sense for basics. But that long-term part—how do they measure if it's truly improving across jobs?
Sam: For long-term, they look at three signs. Evolutionary gain means costs keep dropping steadily as the agent reuses lessons—like pedaling a bike where each push builds speed without restarting. Stability checks if new learning doesn't erase old know-how, keeping success even on mixed tasks. And implicit alignment convergence tracks if user fixes drop over time, as the agent quietly picks up personal habits from ongoing use.
Alex: Wait, dropping user fixes naturally? So it's learning your style without being told directly.
Sam: Yes. They use simple trends over task sequences for this. One is the efficiency evolution rate—basically, how much the computing cost per task shrinks from one to the next. If it steadily heads toward zero, that's real progress; flat or jumpy lines mean no flywheel effect.
Alex: Right, so trends reveal the fakes. What kinds of task chains test that?
Sam: They build from basic building blocks called atomic tasks—simple jobs like web searches or file tweaks, grouped by hardness: easy ones for baselines, normal for real tests, hard for limits. These slot in variables, like swapping file names, so repeats feel fresh but use the same logic. Then they chain them into sequences: correlated ones repeat similar structures to build skills, orthogonal mix unrelated ones to check stability, and implicit intent ones hide goals to test adaptation—all with added noise like fake data to mimic real mess.
Alex: Huh. So by watching those trends in noisy chains, you see if the loop is really turning experience into reusable steps.
Sam: Exactly. Auxiliary checks like step count or self-fix frequency add details within tasks, but the trends confirm if evolution is genuine—steady drops in cost and fixes, without ups and downs that signal pseudo-progress. The paper suggests this setup clearly separates agents that consolidate experience from those stuck redoing everything.
Alex: So those trends in noisy chains paint a clear picture of real versus fake progress. But to make it concrete, what do the actual tests look like—say, the different kinds of task chains?
Sam: They set up chains in three styles to probe different angles. One style links similar tasks, like booking flights with slight changes each time, to watch if costs drop as skills build. Another mixes totally different jobs, such as jumping from web searches to file edits, to check if learning one doesn't wipe out know-how from before—that risk is called catastrophic forgetting, where new stuff crowds out the old. A third uses vague instructions, like "handle this booking" without details, so the agent guesses the user's real goal from trial fixes over time.
Alex: Mixing unrelated tasks tests for forgetting, and vague ones force style-matching. They also add mess to mimic real life?
Sam: Yes, they inject noise two ways. In the clean setup, the agent starts each chain with a blank slate, no past skills carried over, to see pure learning from zero. The noisy one loads it with 20 unrelated skills first, like stuffing a toolbox with wrong tools, to test if junk clogs retrieval and stalls progress.
Alex: Right, so noise checks if memory holds up under clutter. With that, how did real agents stack up against others in practice?
Sam: They tested two setups: OpenClaw and GenericAgent, both using the same strong language model in safe, boxed environments. On single tasks, both nailed 100% success. But over chains, one showed token use dropping steadily—like from 520 thousand to 117 thousand on repeats—by pulling reusable steps from memory. The other jumped around with high costs every time, treating each variant as brand new, which the paper flags as no real evolution.
Alex: So even from errors, it builds something reusable. Makes sense why steady trends matter over single wins.
Sam: Exactly. Overall, the paper shows agents can match perfect success records yet differ by wide margins in ongoing costs, with only the evolving ones showing that smooth drop-off path.
Alex: Right, so it shifts testing from one-off checks to watching growth over time. But are there limits to what this benchmark covers?
Sam: Yes, a few key ones. It focuses on single agents working alone, so it doesn't test groups of them sharing lessons or competing—which could speed up progress but add new complications. It also assumes humans kick off the first tasks; a fully self-starting agent would need to spot its own weak spots and make practice runs without prompts, something not measured here.
Alex: Huh. Groups and self-starts sound like natural next steps, especially for real-world mess.
Sam: Another gap is deeper user matching. The tests use fewer fixes over time as a stand-in for getting personal tastes right, like style or look of outputs, but true likes are harder to pin down without direct feedback methods. On the thinking side, big models slow things down for quick jobs, so blending fast small ones for actions with heavy ones for planning could help, though figuring the split is tricky.
Alex: Fair points—keeps it balanced, not overclaiming. And safety? Running code and files sounds risky.
Sam: They ran everything in sealed test boxes to block real-world harm, with no personal data touched. For wider use, matching powers to growth stage and outer controls would guard against bad drifts, beyond just agent smarts.
Alex: So solid foundation, but room to grow on teams, starts, tastes, speed, and safeguards.
Sam: In the end, this work defines what self-evolving agents need at core and gives a tool to spot real progress from look-alikes. It points toward agents that sharpen up over months of use, matching human workflow speeds for everyday help without huge costs.
Alex: That's a clear step forward in measuring true smarts. Thanks for breaking it down, Sam.
Sam: My pleasure, Alex.
Sam: Thanks for listening to ResearchPod.