Yaxin Luo, Haobin Jiang, Jialv Zou, Xu Huang, Wenhao Yan, Haodong Li, Zhengrong Yue, Jing Li, Xiaofu Chen, Xiaohan Zhao, Jiacheng Liu, Jiacheng Cui, Zhiqiang Shen, Xiaotong Li
4 min
Can an agentic system recursively improve its own design workflow—referred to as a design harness—rather than merely editing individual outputs, in order to transform complex multimodal inputs into structured, human-aligned media artifacts?
AutoDesign establishes a nested two-loop architecture. The inner loop executes a design harness that ingests source documents, generates an editable artifact (such as an HTML-based academic poster), and refines it iteratively using rule-based validators and model-based visual critics. The outer loop functions as a meta-harness optimizer: it evaluates rollout trajectories across multiple tasks, identifies recurrent failure modes, proposes bounded updates to a single functional component of the harness, and passes candidate updates through an acceptance gate to prevent overfitting and performance regression. To rigorously evaluate this framework, the authors introduce PosterBench, a 100-paper benchmark spanning five disciplines equipped with a seven-dimensional evaluation rubric.
On the PosterBench Main Track, AutoDesign achieves an overall score of 78.32, outperforming the closed-source commercial system Claude Design by 7.45 points. Across seven distinct code-agent-model configurations, integrating the optimized DesignHarness increases the average PosterBench score from 54.99 to 67.39, representing a 12.4% relative performance improvement. In a fully autonomous long-horizon run, the system executes 253 tool calls and 11 editing turns within 40 minutes for under $3, producing human-level conference posters that achieve the highest preference in system-blind human evaluations.
This work shifts the paradigm of agentic design from static, prompt-engineered pipelines to self-improving production systems. By systematically converting empirical exploration and evaluation feedback into persistent harness updates, AutoDesign demonstrates how multimodal agents can accumulate reusable design experience and achieve human-competitive performance autonomously.
Transforming multimodal sources into condensed and structured media outputs can be fundamentally conceptualized as a long-horizon agentic process centered on a model-harness system. While an ideal harness system should align with human design priors and accumulate reusable experience through empirical exploration to drive recursive self-improvement, existing paradigms remain static and fall short of this capability. In this paper, we present AutoDesign, a framework that aligns with human design priors, where a meta-harness optimizer guides a code agent to recursively improve harness based on rollout feedback. To instantiate and evaluate this framework, we focus on the academic paper-to-poster generation task and introduce PosterBench, comprising a 100-paper Main Track spanning five disciplines and PosterBench-mini, a shared 10-paper subset for controlled evaluation. On the PosterBench Main Track, AutoDesign achieves the highest score of 78.32, surpassing the closed-source commercial system Claude Design by 7.45 points. Across seven controlled code-agent-model configurations, integrating the learned DesignHarness consistently improves performance, increasing the average PosterBench Score from 54.99 to 67.39 (+12.4%). In a fully autonomous long-horizon loop, it executes 253 tool calls and 11 editing turns within 40 minutes for under $3, reaching average conference-poster quality in human evaluation. A system-blind human study further demonstrates that AutoDesign achieves the highest human preference among evaluated systems.
Sam: That's a substantial gap. Could this same approach work beyond posters — slide decks, websites, that kind of thing?
Alex: The paper suggests that's exactly where this is heading. The core idea is that the underlying research content stays the same, but the harness can be adapted to different presentation formats. The same paper could, in principle, be reformatted for a video script, a webpage, or a conference slide deck — with the system learning the specific conventions of each format over time.
Sam: So the argument isn't really about making the AI model itself more capable. It's about building smarter scaffolding around it.
Alex: That's the central claim. And there's one more design choice worth understanding. Rather than letting the system rewrite the entire harness at once, updates are restricted to one component at a time. The researchers call this "credit assignment" — the idea being that if you change ten things simultaneously and the result improves, you have no idea which change actually helped.
Sam: Like debugging code. You isolate one variable so you know exactly what caused the change.
Alex: Exactly. It keeps the system from accidentally overwriting something that was already working well. The result is a more deliberate, traceable kind of improvement — one where every change has a clear reason behind it.
Sam: So the broader insight is that reliable AI systems might depend less on raw model intelligence and more on the quality of the workflows built around them.
Alex: That's what this research points toward. The harness accumulates design knowledge that persists across tasks — knowledge that would otherwise be lost every time a new request comes in. Whether that approach generalises beyond academic posters is still an open question, but as a proof of concept, the results are worth paying attention to. Thanks for listening to ResearchPod.