Towards Predictive, Aligned, and Scalable Robot Learning

How can robots move beyond simple memorization of training data to perform complex, long-horizon manipulation tasks? The authors investigate how to better align heterogeneous modalities—specifically visual observations, natural language instructions, and continuous control signals—to enable predictive reasoning in embodied agents.

The authors introduce Lumo-2, a latent world-action model that treats action generation as a reasoning process over latent world dynamics. Unlike previous models that rely on explicit textual reasoning or simple reconstruction-based action tokenization, Lumo-2 employs a three-stage progressive training paradigm:

1. Action-Dynamics Alignment: The model learns to extract physically grounded latent world dynamics from visual transitions, using these to provide global context for action reconstruction.
2. Semantic Alignment: Action representations are aligned with vision and language to ensure semantic interpretability.
3. Joint Modeling: The model performs large-scale co-training to integrate future projection with action generation, using a temporal context buffer to resolve ambiguities inherent in single-timestep observations.

By conditioning action generation on a latent representation of future physical possibilities, the model aims to capture the causal structure of the environment rather than just mapping pixels to motor commands.

Lumo-2 consistently outperforms state-of-the-art vision-language-action (VLA) and world-action model (WAM) baselines across diverse real-world tasks. The authors demonstrate that the geometry of the latent space is critical; by moving away from standard reconstruction-based objectives—which often prioritize low-level signal fidelity over control performance—and toward a structured, multi-modal alignment, the model achieves better generalization and scaling. The system shows particular strength in tasks requiring temporal reasoning, physical understanding, and high-complexity dexterous manipulation.