Angen Ye, Angyuan Ma, Boyuan Wang, Chaojun Ni, Fangzheng Ye, Guan Huang, Guo Li, Guosheng Zhao, Haodong Yan, Hengtao Li, Jiwen Lu, Kai Wang, Mingming Yu, Qitang Hu, Qiuping Deng, Songling Liu, Xiaoyu Tian, Xiaofeng Wang, Xinyu Zhou, Xiuwei Xu, Xinze Chen, Yang Wang, Yejun Zeng, Yifan Chang, Yun Ye, Zhenyu Wu, Zhanqian Wu, Zheng Zhu
5 min
Abstract
World Action Models (WAMs) improve robot policy learning by jointly modeling actions and future visual observations, using future scene evolution as dense supervision for physically grounded action generation. However, a common design in existing WAMs is to explicitly generate future videos at inference time, incurring substantial computational overhead and hindering real-time closed-loop deployment. GigaWorld-Policy addresses this issue with an action-centered formulation, where future visual dynamics are used during training while action-only decoding is used at inference time. Building upon this framework, we present GigaWorld-Policy-0.5, an enhanced action-centered WAM designed for more efficient robot control. During pretraining, GigaWorld-Policy-0.5 adopts a mixed Action-Conditioned World Modeling (AC-WM) and WAM training strategy. This strengthens the coupling between visual dynamics and robot actions and improves the transferability of action representations for downstream policy learning. For efficient inference, GigaWorld-Policy-0.5 introduces a Mixture-of-Transformers architecture that separates visual dynamics modeling and action generation into specialized experts, reducing active computation during action-only inference and achieving 85 ms inference latency on a local RTX 4090 setup. In addition, we employ an agent-based AutoResearch pipeline to systematically search training configurations, enabling more efficient identification of optimal experimental setups while reducing the time and manual intervention required for hyperparameter tuning. Experiments and ablations show that GigaWorld-Policy-0.5 preserves the training benefits of future visual dynamics while improving inference efficiency for robot control.
Sam: So the robot isn't carrying dead weight when it's working. It's only running the part of its brain it actually needs.
Alex: Precisely. And that efficiency has a real effect on speed. By bypassing the visual-dynamics component during operation, the system responds fast enough for fluid, real-time movement on standard hardware.
Sam: But there's a catch, right? If the movement specialist learned everything it knows from working alongside the visual specialist during training, what happens if it encounters something genuinely unexpected in the real world?
Alex: That's the right question to ask, and the paper is candid about it. The quality of the action policy depends entirely on how much physical understanding transferred during training. If the training data didn't cover a particular type of situation, the movement expert won't have the instincts to handle it. The separation of training and deployment is a deliberate trade-off: you gain speed, but you're betting that training was thorough enough to cover what the robot will face.
Sam: So the robustness of the whole system lives or dies in the training phase.
Alex: Which is why the paper also describes a component they call "AutoResearch." Think of it as an automated lab assistant. Rather than having human researchers manually test thousands of different training configurations — adjusting settings, running experiments, checking results — AutoResearch does that systematically and at scale. The goal is to find the combination of settings that produces the most capable action policy, without relying on guesswork.
Sam: That's a practical necessity. If the training phase is doing all the heavy lifting, you need to be confident you've trained it well.
Alex: Exactly. The broader principle the paper is demonstrating is this: a robot doesn't need to simulate the future in order to act wisely in the present, provided it has already internalized enough of the world's physics during learning. The researchers describe this as distilling a kind of physical intuition — that internal sense of how things move and interact — into a lightweight action policy that can run quickly and independently.
Sam: So instead of forcing the robot to dream about the future every time it acts, you teach it during training until the knowledge becomes instinct. Then you let it work without the overhead.
Alex: That's the core claim. It's a meaningful step toward robots that can operate with physical intelligence at practical speeds — though, as with any early-stage system, how well that transfers across varied real-world conditions remains an open question. Thanks for listening to ResearchPod.