Haowen Liu, Xirui Li, Shaoxiong Yao, Peng Shi, Tianyi Zhou, Jia-Bin Huang, Furong Huang, Jiayuan Mao
5 min
Abstract
Language models trained on large-scale vision-language data have demonstrated strong potential for embodied agents. Harnessing models through embodied tools use offers a promising alternative to end-to-end vision-language-action systems by combining high-level reasoning with external modules for perception, planning, and control. However, it remains unclear what makes an effective harness for embodied manipulation, and to what extent such a harness can unlock embodied capabilities in a wide range of reasoning models. In this work, we present Guava, a harness framework for embodied tool use developed through systematic exploration of the design space of agent workflows, action spaces, and observation spaces. Our study identifies three key ingredients for effective embodied agents: iterative perception-reasoning-action loops, semantic action abstractions, and multimodal observations. To understand whether these design principles are universal even to small models, we develop an end-to-end training pipeline that distills embodied manipulation capabilities into a 4B open-source model using fewer than 2K trajectories collected entirely in simulation. Experimental results in both simulation and real-world environments show performance comparable to frontier proprietary models while exhibiting strong generalization to unseen objects, novel instructions, and long-horizon tasks. Results suggest that a well-designed harness can serve as a scalable, model-agnostic interface for embodied manipulation, enabling strong emergent embodied capabilities in compact open-source models with minimal training data.
Alex: Squeezing that intelligence into a smaller container. How much data did they need to pull that off?
Sam: Fewer than two thousand practice runs, all conducted inside a computer simulation. That's a notably small dataset for this kind of task. The resulting model—which is meaningfully smaller than the large proprietary systems—performed at a comparable level, and it even handled objects it had never encountered during training.
Alex: So the "harness" itself is doing a lot of the heavy lifting, not just the raw size of the model.
Sam: That is the central argument. By structuring the interaction properly, you can make a smaller, cheaper system behave with the same robustness as a much larger one. It suggests that the *interface* between AI and the physical world deserves as much attention as the AI model itself.
Alex: You mentioned a two-stage training process. How does that help the model learn to recover from mistakes?
Sam: The first stage is supervised fine-tuning—essentially, the model reads through a large set of worked examples, the way a student studies a textbook before an exam. That gives it a solid foundation. The second stage is where recovery behaviour gets reinforced.
Alex: What happens in that second stage?
Sam: They use a technique called Group Relative Policy Optimization—GRPO for short. Here's a useful way to picture it: imagine a student who's given the same maths problem five times and tries a different approach each time. The teacher doesn't just mark them right or wrong—they compare all five attempts and give the highest grade to the one that reached the correct answer most efficiently. GRPO works the same way. The model generates several different sequences of actions for a given task, compares them, and reinforces whichever path actually led to success.
Alex: So it's learning from its own trial and error, rather than just copying examples.
Sam: Exactly. And crucially, it learns *what to do when things go wrong*. If a grasp fails, the model learns that the right response is to reset and try again—not to continue the failed motion. That's the recovery behaviour the earlier approach was missing.
Alex: That explains how it gets so capable with so little training data. It's not just memorising steps—it's learning a strategy.
Sam: That's a good way to put it. And it points to something worth sitting with: the most significant gains here didn't come from scaling up the model or collecting more data. They came from rethinking how the AI is connected to the physical world in the first place. That's a meaningful shift in how researchers are approaching this problem.
Alex: Thanks for walking us through it. And thanks to our listeners for joining us on ResearchPod.