Coding agents can solve a task well before they stop working on it. We characterize \emph{termination lag}--the number of steps between an agent's first correct patch and the end of its run--across five models, two agent frameworks, and three SWE-bench datasets by recording and testing every distinct repository patch generated during execution. Our analysis reveals that 38--82% of successful runs achieve a correct solution on their initial source-code edit. However, agents continue to expend substantial effort after correct patches, with post-solution steps accounting for a median of 35--37% for Qwen3-Coder and 54--60% for Laguna-XS.2 in successful runs on SWE-bench Verified and Multilingual. Agents spend most of these additional steps verifying their work and exploring the repository. In a small fraction of cases (1--8%), this continued activity leads agents to undo their earlier correct patch, leaving the final solution incorrect.
Sam: Coding agents can fix a bug, keep working, and break the fix before submitting it. This study finds a broader problem: even successful agents often spend substantial effort after their solution already works.
Alex: So the issue isn't just whether software can solve a coding task. It's whether it recognizes that the task is finished.
Sam: That's the distinction in an empirical preprint by Musa Cim, Burak Topcu, and Mahmut Taylan Kandemir at Penn State. It's called “Termination Lag: Coding Agents Solve Early but Stop Late.” They track working fixes through an agent's execution, rather than judging only what it submits.
Alex: For researchers building or comparing these systems, that could change what counts as good performance. Let's unpack how they locate the first working fix, what happens afterward, and whether the extra work is actually avoidable.
Sam: First, a coding agent here is software that can inspect a repository, change files, and run commands or tests. It receives an issue description and tries to produce a patch: the accumulated changes to the original code. The usual evaluation asks whether its final patch passes the benchmark's tests.
Alex: That final score sounds reasonable if you only care about the submitted code. What does it miss that earlier work hadn't already captured?
Sam: It misses the route to that result. Earlier studies examined excess generation, agent costs, and predictions for stopping unsuccessful runs. Some also found that agents could lose a correct intermediate fix. This study systematically measures the interval between producing a correct repository patch and ending the run.
Alex: So two agents could earn identical scores, even though one kept working long after the other would have finished. But locating that moment sounds difficult. How do you know the earlier version really worked?
Sam: The key is to save the patch after every agent step. After the run ends, the researchers test each distinct patch using the benchmark's official evaluation. They do that separately, in isolated containers. The agent never sees those retrospective results.
Alex: That separation matters. If the researchers told the agent its patch had passed, they'd be changing the stopping behavior they wanted to observe. What counts as a step in their measurement?
Sam: A step includes the model's reasoning, its selected action, and any resulting tool execution. The researchers locate the earliest step whose patch passes the official tests. Everything between that point and the run's end is what they call termination lag.
Alex: There's an immediate objection, though. A patch passing hidden benchmark tests doesn't mean the agent knows it has passed. Isn't more checking a reasonable response to uncertainty?
Sam: Yes, and the authors explicitly acknowledge that. Agents need some verification because they cannot see the benchmark tests. Termination lag describes what happened after correctness was reached; it doesn't establish that every later action was unreasonable. That's the central distinction between measuring an opportunity and delivering a usable stopping rule.
Alex: With that distinction in place, how broad was the evaluation? I'd be less persuaded if this were one model getting stuck in one particular workflow.
Sam: They studied five models, mainly through a framework called OpenHands, which supplies tools for inspecting and modifying code. They also checked a second framework. Their tasks came from three SWE-bench datasets: collections of software issues with tests for proposed fixes.
Alex: Were those all similar coding problems, or did the study stretch beyond one kind of repository?
Sam: The main evaluation combined Qwen's coding model and Laguna with both Python tasks and a multilingual benchmark. Additional evaluations used commercial models on the Python benchmark and Seed-OSS on the harder SWE-bench Pro tasks. The pattern appeared across those settings, although the evidence isn't equally extensive in each.
Alex: Let's start with when a solution appears. Do agents usually need a long sequence of revisions before they get something correct?
Sam: Not always. For Qwen on SWE-bench Verified, the first source-code edit was already correct in roughly four out of five runs that ever reached correctness. That denominator matters: this is not the success rate across all attempted tasks. It's a description of runs that produced a correct patch at some point.
Alex: And “first edit” doesn't mean “first action.” The agent might already have spent time reading the repository before changing anything.
Sam: The study tracks that distinction. Agents often inspect code before their first source change. What stands out is that, once they make that change, many have already produced the benchmark-correct solution. Yet they do not immediately submit it.
Alex: How much execution separates those moments? A quick final check would mean something different from spending most of the run after the fix.
Sam: Among successful Qwen runs on Verified, the median share of steps after the first correct patch was thirty-five percent. For Laguna on the same benchmark, it was sixty percent. These are medians of each successful run's post-solution fraction, not percentages of all tasks or direct measurements of money saved.
Alex: So the headline isn't just a few unusually long runs pulling up an average. For successful runs in those settings, the gap is a substantial part of execution. What fills it?
Sam: Mostly verification: running repository tests, writing extra checks, or building the project. Exploration comes next, including reading files, searching code, and reviewing changes. Further source edits account for relatively little activity in most settings, though editing is more common for Seed-OSS on the harder benchmark.
Alex: That sounds more like caution than pointless wandering. Can the study tell us whether the checks improved anything beyond passing the benchmark?
Sam: It doesn't establish that. Correctness here means passing the official task tests. Once a patch meets that criterion, further work cannot improve that binary benchmark outcome. But the measurement doesn't show that later checks have no value for confidence or for properties outside those tests.
Alex: There's still a downside if continued work changes the code. How often did agents actually lose a solution they had already found?
Sam: Across the reported settings, between one and eight percent of runs that reached a correct patch ended with an incorrect one. Other runs temporarily broke the fix and repaired it before finishing. Most retained correctness, but the final score alone would hide those losses and recoveries.
Alex: Suppose someone wants to turn this into an efficiency claim. Have the authors demonstrated an agent that stops earlier without sacrificing success?
Sam: No. They consider an oracle: a hypothetical stopping decision with access to the first correct moment. It would preserve the tasks already solved and avoid later regressions. But the real agent lacks that information. These results measure the potential benefit, not a demonstrated way to obtain it.
Alex: And steps aren't necessarily interchangeable units of cost. A search and a test run can involve different amounts of work. What other limits should shape our confidence?
Sam: The authors say steps approximate computational cost rather than measuring it directly. Each task was run once, so the study doesn't characterize repeated-run variability. The harder-benchmark model solved few tasks, making its statistics less secure. I'd trust the recurring descriptive pattern more than any precise forecast of deployment savings.
Alex: Who should read the full document, and where should they begin?
Sam: Researchers building coding agents, evaluating them, or designing stopping policies should read it. Start with the methodology on recording patches after each step. Then read “Understanding Agent Behavior During Termination Lag” to separate checking, editing, and actual regressions. Keep the distinction between retrospective correctness and available evidence in view.
Alex: And for researchers who only need the central lesson?
Sam: Finding a working fix and knowing when to stop are different abilities. A final success score measures the destination, but it can hide both wasted effort and a solution lost along the way.
Alex: Thanks for listening.