Precise grounding matters. It specifies which object is the target and where that object is, even in clutter and for tiny objects, and it has to be fast enough for closed-loop control. Yet vision-language-action (VLA) and world-action models (WAMs) take perception from general-purpose vision-language and video-generation backbones, which still fail in these settings. We introduce GroundingPI, a 4B grounding foundation model that generates points and boxes as quantized coordinates in a shared vocabulary. Training combines multimodal and spatial pretraining, supervised fine-tuning, and reinforcement learning with GRPO, using supervision from public datasets and dedicated data engines. Against 44 baselines across 34 grounding benchmarks spanning 11 perceptual capabilities, GroundingPI establishes a new state of the art, averaging 73.68%, above the larger GPT-6 Astra (71.54%). As a downstream visual backbone, GroundingPI improves performance on robotic manipulation and autonomous driving. On RoboTwin 2.0, it outperforms every mainstream backbone we evaluate in all four out-of-distribution settings, by up to 24.8% relative to the strongest backbone. On RoboCasa-GR1, GroundingPI trained with 50% of the demonstrations outperforms those baselines trained with 75%. On nuScenes, used as the visual backbone, GroundingPI attains an average open-loop L2 error of 0.296 m. We systematically analyze GroundingPI's pretraining in scale and data composition. Downstream autonomous driving and robotic manipulation improve as the pretraining is scaled. Analyzing the data recipe across these 11 perceptual capabilities shows dense grounding's substantial benefits for both, and OCR's potential as a catalyst for perceptual learning. These results support grounding as a perceptual foundation, and dedicated perceptual pretraining as a promising direction for foundation models of physical intelligence.
Alex: Teaching a model where to act can make learning how to act easier. This study finds that precise target-finding improves simulated robot learning, including when demonstrations are scarce.
Sam: That sounds like an argument about what models should learn before they ever see robot demonstrations. Whose work is it?
Alex: It’s an arXiv method preprint by Qize Yu and colleagues, introducing GroundingPI. Their four-billion-parameter model learns to connect instructions with specific objects and locations. They call that capability grounding.
Sam: So the audience isn’t just people improving object detection. It’s anyone deciding what pretrained model to put inside an action-learning system.
Alex: Especially researchers who have limited robot data and need policies that survive unfamiliar scenes. The claim is that stronger perception can be a better starting point than generality alone. We’ll unpack what they changed, the controlled comparisons, and why these results stop short of real-world readiness.
Sam: Start with the problem they’re trying to solve. What does a general visual model miss that matters for action?
Alex: The authors use a cluttered desk. Before moving a mug, you need to find the mug and its handle. Recognizing the scene broadly doesn’t guarantee that precision. Dense scenes and tiny objects expose weaknesses even in strong general-purpose models.
Sam: And if perception is weak, the robot demonstrations have a double job. They must teach the model what to attend to, as well as how to move.
Alex: That’s the authors’ concern. Vision-language-action models, which turn visual observations and instructions into actions, often inherit general-purpose visual-language models. Other systems inherit video-generation models. Those foundations bring useful knowledge, but their training doesn’t necessarily prioritize precise interaction targets.
Sam: Perception supervision already exists in robot learning, though. What is actually new here?
Alex: The authors point to the progression from pi-zero to pi-zero-point-five, which adds bounding-box prediction within a broader training recipe. Other approaches add perception modules or auxiliary objectives. GroundingPI instead learns broad grounding before action adaptation, making it a native capability rather than a task-specific addition.
Sam: Then explain the mechanism through the mug example. What does learning “where” actually look like?
Alex: Give the model an image and an instruction such as “Locate the cup.” It generates the target label and coordinates describing a box. Points and boxes are its visual primitives: simple spatial outputs. Different perception tasks share that output language.
Sam: So the unifying idea is not a separate prediction head for every task. It’s a common way to express locations.
Alex: Coordinates become dedicated output tokens, discrete symbols in the model’s vocabulary. The model generates them sequentially, alongside labels and protocol markers. That same interface supports finding objects, pointing, reading text, and locating interface elements or document regions. It can also say that a queried category is absent.
Sam: A shared format makes the tasks compatible. What makes the locations accurate rather than merely well formatted?
Alex: Training combines general visual-language learning with supervised grounding and reinforcement learning. Public datasets supply supervision, supplemented by a data engine that combines annotations from multiple teacher models and validates them. Reinforcement rewards then assess whether targets were covered and localized precisely, while penalizing duplicate or oversized boxes.
Sam: Let’s separate perception performance from action performance. How broad was the first test?
Alex: They evaluated across thirty-four grounding benchmarks against forty-four baselines. The suite includes crowded scenes, referring expressions, spatial pointing, text recognition, and document layouts. GroundingPI’s average score was about seventy-four percent, versus about seventy-two for the larger GPT-six Astra.
Sam: That average mixes different tasks and metrics. I wouldn’t hear it as “this model gets nearly three quarters of everything right.”
Alex: Nor should we. It summarizes this benchmark suite, not a universal accuracy rate. The task-level results matter: GroundingPI is strong on dense-object boxes, but Astra retains substantial advantages on several spatial and interface tasks. Better box boundaries also don’t automatically mean better point selection.
Sam: Now the consequential test: could the robot gains come from changing the action learner rather than improving its perceptual foundation?
Alex: The downstream comparison holds that learner fixed within each manipulation benchmark. The authors keep the robot data, action representation, training budget, and inference procedure unchanged. They vary the backbone—the pretrained model supplying representations to the action learner.
Sam: That isolates the choice of foundation under one adaptation recipe. It doesn’t mean the foundations had identical pretraining histories.
Alex: Correct; their pretraining sources and objectives differ. The comparison is useful for choosing a foundation under matched downstream conditions. It does not isolate every architectural cause. GroundingPI supplies intermediate features to an action model that generates continuous robot controls.
Sam: What kinds of robots and unfamiliar conditions did they test? A single setting could make this look broader than it is.
Alex: They use simulated tabletop manipulation with paired robot arms, and simulated humanoid manipulation with dexterous hands. Out-of-distribution testing means conditions held out from training. The tabletop test changes clutter, lighting, backgrounds, and other scene properties. The humanoid tests change object appearance, object categories, and object–container pairings.
Sam: Those distinguish scene robustness from recognizing or handling unfamiliar entities. Did GroundingPI help on both?
Alex: It ranked first in every held-out setting they evaluated. The strongest relative improvement over the best competing backbone was about twenty-five percent, on the tabletop scene-shift test. That is a relative gain, not percentage points. Absolute success there remained low, so leading the comparison doesn’t mean reliable execution.
Sam: And it wasn’t the winner everywhere. Where does the pattern break?
Alex: On the humanoid benchmark’s full, in-distribution evaluation, RynnBrain did better. GroundingPI’s advantage was more consistent under distribution shift. It also led every reduced-data condition: using half the demonstrations, it beat every compared baseline trained with three quarters.
Sam: That makes the practical argument stronger. A better foundation may save demonstrations, rather than just raise performance when data are plentiful.
Alex: Within that benchmark and training recipe, yes. Driving transfer was more modest. On recorded driving data, average trajectory error was roughly thirty centimetres, slightly lower than the compared compact foundations. This was open-loop prediction: predicting future positions without testing feedback-controlled driving.
Sam: Can they identify what perception training transfers? Otherwise, “train a better backbone” leaves researchers without much guidance.
Alex: Their scaling study links more grounding training with better grounding scores and improved manipulation. Their data-removal experiments also point to dense grounding as a strong spatial foundation. Removing it hurts manipulation and driving more than removing referring or robot-pointing supervision in the available comparisons.
Sam: The text-reading result seems less obvious. Why would reading words help manipulation?
Alex: Removing the group containing optical character recognition—reading and locating text—causes the largest manipulation losses. Yet that group alone transfers poorly. The authors hypothesize that precise character boundaries and linking regions to words strengthen perception. That mechanism remains a hypothesis, and complementary supervision performs best overall.
Sam: So there is evidence for the training mixture, but not proof of why text reading helps. What’s the biggest boundary on the whole story?
Alex: No real-world robots or vehicles were deployed. Manipulation was simulated, and driving was evaluated offline. The study doesn’t establish closed-loop driving performance, deployed controller latency, or safety. Compact coordinate outputs reduce serialization costs, but generating them sequentially still takes time.
Sam: Given those boundaries, who should read the full document, and where should they start?
Alex: Robot-learning researchers choosing a pretrained foundation should start with “Robot Manipulation Performance” and “Data Efficiency.” Then read “What Transfers from Grounding to Action?” for the training-mixture evidence. Driving researchers should start with “Autonomous Driving Performance,” keeping its open-loop scope in view. For everyone else: knowing what’s there is not the same as knowing precisely where to act.
Sam: Thanks for listening.