Haoxiang Ma, Junhao Cai, Xiaoxu Xu, Hao Li, Yuyin Yang, Yang Tian, Jiafei Cao, Hongrui Zhu, Zherui Qiu, Zhaxizhuoma, Yuqiang Yang, Jiaqi Peng, Xueyuan Wei, Yangkun Zhu, Jiahao Jiang, Xing Gao, Hanqing Wang, Feng Yuan, Kailin Li, Xueyue Zhu, Tai Wang, Yan Ding, Jiangmiao Pang, Jia Zeng, Jingjing Zhang, Bowen Zhou, Yao Mu, Chunhua Shen, Weinan Zhang
5 min
Abstract
Unified models for robot manipulation aim to equip one policy with both the semantic priors of pretrained VLMs and the physical dynamics learned through future prediction. In practice, existing designs tend to erode the semantics of the pretrained backbone, suffer interference among heterogeneous objectives, and learn future prediction from scratch in pixel space, leaving the dynamics priors of pretrained video generators unexploited. We present InternVLA-A1.5, which builds the policy on a native VLM backbone that keeps training on VQA and subtask prediction, and attaches a lightweight unified expert for continuous action generation. Future prediction is recast as a latent-querying problem, where a small set of learnable foresight tokens condenses the task-relevant future into a compact latent code under the supervision of a frozen pretrained video generation model, so the policy inherits world-model dynamics priors without ever learning pixel-level generation. The video branch is discarded at inference, keeping real-time control. Pretrained on 1.2M robot episodes and 3M multimodal samples, InternVLA-A1.5 achieves the best overall results on all six simulation benchmarks. In the real world, the preserved semantics deliver the strongest compositional generalization on held-out instruction bindings, and the two designs together sustain long-horizon execution.
Alex: So it's like a kitchen with two stations—one chef reading the recipe, one chef doing the chopping—and they're talking to each other, but neither one is doing the other's job.
Sam: Exactly. And because those functions stay distinct, the robot can follow a complex, multi-step instruction and navigate the physical world with foresight, without either skill degrading the other.
Alex: So how does the robot actually turn all of that internal knowledge into a physical movement? There's still a gap between "understanding what should happen" and "moving an arm."
Sam: Right, and that's where the second half of the architecture comes in. The foresight tokens—that compressed fingerprint of the expected outcome—pass their information to what the paper calls "action query tokens." Think of these as a translation layer. They take the abstract knowledge about what should happen and convert it into a concrete plan for what the robot's joints need to do.
Alex: And then there's something called a "flow matching" head that handles the actual movement calculation?
Sam: Yes. Imagine you're trying to draw a smooth curve, but you start with a scribble. "Flow matching" is a mathematical technique that learns how to gradually transform that messy starting point into a precise, smooth path. Applied to robotics, it teaches the system to "flow" from the robot's current position toward its goal in one fluid motion.
Alex: So rather than calculating one tiny step, then the next, then the next—it plans the whole arc of the movement at once?
Sam: Exactly. It predicts a short sequence of actions as a single chunk, rather than one isolated twitch at a time. That's what gives the robot's movements their smoothness—it's thinking about the whole gesture, not just the next millisecond.
Alex: So to put the whole pipeline together: the language side sets the goal, the foresight tokens provide a sense of what a good outcome looks like, and the flow matching system translates all of that into actual physical motion.
Sam: That's it. And the training process is careful to protect each layer. They first train the model to be a strong instruction-follower. Only after it's solid on that do they introduce the foresight tokens. So the physical common sense is built on top of the language ability, not in competition with it.
Alex: Are there limits to what this approach can handle?
Sam: Yes, and the researchers are upfront about it. The foresight supervision currently covers only a short window of time—it's good at handling the immediate physical mechanics of a single movement, but it isn't doing long-term planning across a full, multi-step task. It understands how to make a particular motion well, but it isn't thinking five moves ahead like a chess player.
Alex: So it's physical intuition for the near term, not strategic planning for the long term.
Sam: That's a fair summary. The paper frames it as a meaningful step toward robots that can handle real-world physical tasks reliably—but the researchers are clear that longer-horizon reasoning remains an open problem. This is progress in one important direction, not a complete solution.
Alex: That's a useful distinction to keep in mind. Thanks for walking through the architecture so clearly, Sam. And thanks to everyone listening—this has been ResearchPod.