Junhan Sun, Hao Zhao, Guofeng Zhang
4 min
Traditional latent world models often separate representation learning from control, requiring expensive test-time search (like CEM) to invert the model for goal-directed tasks. This paper asks whether it is possible to unify these processes by learning an explicit, search-free interface that maps desired latent motion intents directly to effective action sequences.
INTACT (INtent-To-ACTion) introduces an isomorphic architecture that treats both physical transitions and future goals as motion-intent requests. The model uses a shared predictor to map these intents—defined as latent displacements—to action distributions. By using asymmetric gradient routing, the model grounds physical successors while treating future goals as anchors. This allows the system to learn a unified action-law semantics where the conditional mean of the learned distribution serves as a direct, zero-search policy. The architecture is trained end-to-end using action-labeled, reward-free trajectories, ensuring that the latent space is shaped specifically for control.
INTACT achieves state-of-the-art performance on the LeWM benchmark, reaching up to 100% success on individual tasks with zero-search inference. The model reduces planner latency by approximately 300x compared to traditional CEM-based approaches. When optional local verification is used, INTACT achieves 96.86% macro success while reducing the number of sampled action sequences by over 23x. Furthermore, the authors demonstrate that the joint intent supervision improves the underlying visual representation, as evidenced by improved performance of frozen probes and actor-disabled CEM compared to baseline models.
This work bridges the gap between representation learning and control by giving latent spaces an operational semantics. By moving away from expensive, iterative search methods toward direct, learned intent-to-action mappings, INTACT provides a scalable path for high-frequency robotic control in multi-task settings without requiring reward labels or complex, post-hoc policy training.
Forward latent world models predict how actions change a scene, but recover actions for a desired change only through expensive test-time search. We introduce INTACT (INtent-To-ACTion), an end-to-end JEPA that turns action-labeled, reward-free trajectories into a deployable intent-to-action interface. Each transition supplies physical intent $z_{t+1}-z_t$, while a future goal supplies deployment intent $\operatorname{sg}(z_g)-z_t$. The architecture is isomorphic between the local and goal motion-intent backbone-input graphs through an identical four-slot grammar and shared parameters, and between supported local and goal motion-intent families through action-law semantics induced by the same predictor rather than pointwise latent equality. INTACT also provides intact transfer from RGB evidence to action-effective latent intent coordinates and from intent families to their corresponding action-law families. Asymmetric endpoint gradients ground physical successors and fix future goals as anchors, joining representation learning and control without pointwise latent matching or globally linear dynamics. The resulting coordinates support a robust distributional action law: its conditional mean serves directly as a search-free policy, while sampling remains available for diversity or optional verification. On the four official LeWM tasks, one-epoch, zero-search models reach 85.78\%, 100.00\%, 97.67\%, and 97.89\% success. Optional local CEM centered on the Direct plan reaches 96.86\% macro success using 384 instead of 9,000 candidate sequences, reducing sampling by $23.44\times$ while improving pure CEM by 16.00 points. One shared four-task encoder reaches 89.39\% E5 Direct macro and improves every task over jointly trained LeWM, while predicted--expert action-family kNN tracks Direct success at $r=0.954$. Direct inference takes 2.9--5.5 ms.
Alex: There's something else that's been nagging at me, though. If the system is trained so aggressively to find shortcuts, doesn't it risk becoming too specialized — learning to handle the training examples perfectly but struggling with anything genuinely new?
Sam: That's a legitimate concern, and one the researchers took seriously. Their solution is a technique they call Asymmetric Gradient Routing. Here's the intuition: think of training an athlete. Some drills are about building physical instincts — muscle memory, grounded in the real world. Other drills are about strategy and anticipation — thinking ahead. If you mix those up and train them the same way, you get a confused athlete. By keeping these two roles distinct during training, the model learns to generalize rather than just memorize.
Alex: So it's learning the underlying rule of how movement works, not just a catalogue of right answers.
Sam: That's the right way to think about it. The model learns how a desired change in position translates into a specific motor command, and because the same rules govern that translation consistently, its internal map stays reliable. That consistency is what allows it to handle new situations, rather than falling apart the moment something unfamiliar comes along.
Alex: So INTACT is essentially teaching robots to think about movement the way a skilled person does — not by running through every option, but by having a deep enough understanding of the rules that the right action becomes almost immediate.
Sam: That's a fair summary. It's a shift from brute-force search to principled understanding. And if the results hold up in broader testing, it could make real-time robot decision-making significantly more practical than it currently is.
Alex: Thanks for walking us through that, Sam. And thanks to everyone listening to ResearchPod.