Ziyu Ma, Hailang Huang, Shun Zou, Yong Wang, Shidong Yang, Yiming Hu, Fei Wei, Xiangxiang Chu
5 min
Abstract
Large language model (LLM) agents increasingly undertake long-horizon tasks that require sustained reasoning, tool use, and revision across many interdependent steps. However, existing agent harnesses maintain task execution, task state, and completion assessment within a growing context, making the state difficult to track and allowing incorrect self-assessments to propagate into later decisions. We reformulate long-horizon execution as a task-state management problem and propose LongHorizon-Harness, which maintains the task state explicitly outside execution and updates it only with facts independently verified from the environment. Its Manage-Execute-Audit(MEA) loop uses a manager to maintain the task state and determine the next subtask, a fresh-context executor to perform it, and a read-only auditor to verify the resulting environment state before the next round. A lightweight AgentAdapter supports interchangeable model and harness backends without modifying their native agent loops. LongHorizon-Harness improves Qwen~3.7-Plus from 51.8% to 80.7% on WeaveBench, from 69.7% to 77.2% on Terminal-Bench~2.1, and from 2.8% to 8.3% on OSWorld~2.0. It also raises Claude Opus~4.7 from 20.0% to 34.3% on an OSWorld2.0 subset, demonstrating consistent gains across models, harnesses, and interaction domains.
Alex: So you could take the same AI brain and get substantially better results just by organizing how it works?
Sam: That's what the paper argues. And it holds across different models and task types — the gains aren't specific to one particular AI system. The implication is that a lot of the reliability problems people attribute to weak models might actually be problems with weak organization.
Alex: If the system is running all this extra auditing on top of the normal work, does it become slow or expensive to run?
Sam: That's a reasonable concern, but the researchers found the opposite tends to be true. The Manager role is relatively lightweight. The Auditor does more work, yes — but the overall system often uses fewer total resources than the old approach. The reason is that unstructured agents frequently get stuck in loops, repeating failed steps over and over. By catching errors early and cleanly, the MEA loop actually cuts down on wasted effort.
Alex: So the overhead of auditing is less costly than the waste of spinning your wheels on mistakes you don't even know you've made.
Sam: Exactly. And there's a deeper point here. The system turns execution progress into a verifiable record — a series of confirmed facts — rather than a long, tangled transcript of actions that the agent has to interpret. That's a much more stable foundation for a long-running task.
Alex: It's almost like the difference between a project with clear milestones and sign-offs versus one where everyone just keeps working and hopes it comes together at the end.
Sam: That's a fair comparison. The researchers are essentially arguing that the discipline we apply to human project management — break it down, verify each step, don't move on until you're sure — is exactly what AI agents need too. The intelligence is still in the model. But the reliability comes from the structure around it.
Alex: That's a genuinely useful reframe — that improving AI performance isn't always about building a smarter system, but sometimes about building a better-organized one. Thanks for walking us through it, and thanks to everyone listening to ResearchPod.