Generalist robot policies must follow user instructions while reasoning about how objects, cameras, and robot actions interact in the 3D physical world. Recent vision-language-action models (VLAs) and video world-action models (WAMs) inherit strong semantic or temporal priors from large-scale foundation models, but they still operate primarily on 2D image frames or 2D-derived latent spaces, leaving implicit the 3D geometry required for contact-rich manipulation. We propose the Geometric Action Model (GAM), a language-conditioned manipulation policy that directly repurposes a pretrained geometric foundation model (GFM) as a shared substrate for perception, temporal prediction, and action decoding. GAM splits the GFM at an intermediate layer: the shallow layers serve as an observation encoder, and a causal future predictor inserted at the split layer forecasts future latent tokens conditioned on language, proprioception, and action history. The predicted future tokens are then routed through the remaining GFM blocks for feature propagation and decoding, allowing a single backbone to produce both future geometry and actions. This design equips the GFM with language-conditioned temporal world modeling through minimal architectural modification while preserving its rich geometric priors. Across a broad suite of simulation and real-robot manipulation benchmarks, GAM is more accurate, more robust, faster, and lighter than current foundation-model-scale baselines.
Alex: Welcome to another episode of ResearchPod. Today, we're discussing a new approach to robot control called the Geometric Action Model, or GAM. The central claim of this research is that robots can perform better if they're built to explicitly understand 3D space, rather than just interpreting flat, 2D images.
Sam: So the problem this paper is tackling is that most robots today are essentially working from a photograph of the world, rather than a real sense of depth and space?
Alex: That's a good way to put it. Imagine trying to catch a ball using only a single photo taken a moment ago. You'd have no sense of how far away it is, or how fast it's moving toward you. Most robot control systems work a bit like that—they process what the camera sees as a flat image, and when the view shifts even slightly, they lose track of where things actually are in space.
Sam: So something as simple as the camera getting nudged, or the lighting changing, can throw the whole system off?
Alex: Exactly. The robot hasn't built a mental model of the 3D world—it's just pattern-matching against what it's seen before. Change the angle, and those patterns no longer match. GAM addresses this by using what the researchers call a Geometric Foundation Model, or GFM, at its core.
Sam: What is a Geometric Foundation Model, exactly?
Alex: Think of it as a pre-trained brain that has already spent a lot of time learning how to convert flat images into 3D structures. It's the difference between a map that just shows roads, and a GPS that understands the actual terrain—hills, depth, distance. The GFM has that spatial understanding baked in before the robot ever starts learning a task.
Sam: So they're not building 3D understanding from scratch—they're borrowing it from something that already knows how to do it. How do they then turn that into actual arm movements?
Alex: They split the GFM into two functional parts. The first half acts as an encoder—it takes in what the camera sees and translates it into a rich 3D representation of the scene. Then, between the two halves, they insert what's called a causal transformer. That's a component that looks at the current state of the world and predicts what's likely to come next.
So it's not just reacting to what it sees right now—it's actively forecasting?
Alex: Right. And that forecasting happens inside the model's own 3D understanding of the scene, not just in terms of pixels. So the robot isn't guessing "the cup will probably be a bit to the left in the image"—it's reasoning about where the cup will be in actual space. That grounding in physical reality is what makes the actions more reliable.
Sam: I'd have assumed all that 3D reasoning would make it slower. Is there a computational cost?
Alex: You'd expect that, but the research reports the opposite. Because the same backbone handles both perceiving the scene and planning the action—rather than running separate systems—it's substantially faster than approaches that use a technique called diffusion to generate movements. Diffusion models are powerful, but they work by iteratively refining a noisy guess into a clean answer, which takes time. GAM sidesteps that entirely.
Sam: So the speed comes from the fact that understanding the scene and deciding what to do are happening inside the same structure, rather than being two separate, expensive processes?
Alex: That's the key insight. The joint prediction—forecasting the future geometry of the scene at the same time as the robot's own next actions—is what the researchers identify as the central mechanism behind the performance gains. The model stays oriented even when the camera is moved, because it's reasoning about a 3D world, not just a particular camera angle.
Sam: It's a meaningful shift from just training a robot to copy motions it's seen before. This approach is asking the robot to actually understand the space it's operating in.
Alex: That's the core argument of the paper. By integrating geometric awareness directly into how the robot makes decisions, the system no longer has to infer depth and position from flat pixels—that understanding is already there. The evidence the researchers present suggests this is a clear improvement, particularly when conditions change in ways the robot hasn't seen before. Thanks for listening to ResearchPod.