Dingjie Song, Hanrong Zhang, Dawei Liu, Yixin Liu, Zongxia Li, Zhengqing Yuan, Siqi Zhang, Henry Peng Zou, Zhiling Yan, Yuxuan Zhang, Yanfang Ye, Philip S. Yu, Lichao Sun
6 min
Abstract
Command-line coding agents (e.g., Claude Code, Gemini CLI) can already read and write files and sustain long sessions, yet end-to-end research still fragments across chat tools, IDEs, terminals, and writing environments, and the decisions that make it auditable are rarely preserved. We present Dr. Claw, an open-source workspace that wraps existing coding-agent executors in a controllable and auditable human-in-the-loop workflow rather than introducing another autonomous agent. Persistent state objects, a reusable skill library, and multi-executor coordination link human decisions to AI execution, turning planning, execution, and writing into one traceable, recoverable loop. We demonstrate Dr. Claw through an interactive three-view scenario and a failure-recovery walkthrough, and evaluate it against a bare command-line agent sharing the same backend executor, so the comparison contrasts the whole orchestration layer (task graph, state objects, and skill library) with the agent it wraps. Holding the executor fixed, Dr. Claw scores higher on research completeness while persisting an auditable, recoverable process trail. Demo access: repository https://github.com/OpenLAIR/dr-claw, released under AGPL-3.0 with GPL-3.0 upstream components.
Alex: Let me make sure I have the logic right. The system doesn't generate the research—it constrains the execution to enforce standard hygiene. Is that the primary value-add?
Sam: That's the core of it. In their pilot evaluation, holding the backend model constant, the version with the Dr. Claw orchestration layer consistently included hygiene elements—limitations sections, real citations—that the bare agent simply skipped. The model was capable of producing them, but without the orchestration layer enforcing the structure, it didn't know it should.
Alex: That's a subtle but important distinction. The capability was always there; the process wasn't. But is there a cost? If every action has to pass through a state machine, doesn't that add meaningful overhead?
Sam: It does—it's slower than a bare CLI agent. But the authors argue that for a researcher, a modest orchestration cost is a reasonable trade for a project that is actually auditable and recoverable. The system is optimized for reliability over throughput.
Alex: You mentioned this was a pilot study. How much weight should I put on these findings?
Sam: I'd be careful. The evaluation is small-scale—one replicate per task. It's directional evidence, not a causal proof. And because the task graph, state objects, and skill library are all bundled together, we can't isolate which component is doing the heavy lifting. We don't know whether the gains come from the state management itself or simply from the better prompting that the skill library provides.
Alex: So it's a bundle effect. A careful referee would push back on that immediately—you'd want an ablation to know whether the state objects are actually necessary, or whether a well-structured prompt would get you most of the way there.
Sam: Exactly. The authors are transparent about this, though. They frame it as an exploratory demonstration of a new paradigm rather than a benchmark result. They're testing the feasibility of the workflow, not the ceiling of the model.
Alex: What about the "Vibe Research" label? Is there a technical reason for it, or is it mostly framing?
Sam: A bit of both. It's meant to emphasize the human-in-the-loop dynamic—the researcher sets the high-level goals and constraints, the AI handles execution. The human is still the conductor; the AI is playing the instruments. Whether that framing does useful conceptual work or just makes the paper more memorable is a fair question.
Alex: Looking ahead—if you can track the process this precisely, does that open the door to automated verification? Not just logging what happened, but checking that the reported results actually match the execution trace?
Sam: That's the logical next step the authors gesture toward. A persistent artifact store and a complete execution trace would, in principle, let you build a system that doesn't just track the research but formally verifies that reported results match what was actually run. That would be a meaningful advance for reproducibility—though it's speculative at this stage, not something the paper demonstrates.
Alex: So the takeaway isn't that we have a new AI scientist, but a new way to organize the scientist we already have. Infrastructure, not just intelligence.
Sam: That's the right read. We've spent years optimizing model capability and largely neglected the infrastructure around it. Dr. Claw is a step toward treating the research process itself as a first-class, auditable scientific artifact—and that's a different kind of contribution than a benchmark improvement, but potentially a durable one. Thanks for listening to ResearchPod.