The Moving Eye: Enhancing VLA Spatial Generalization via Hybrid Dynamic Data Collection

Vision-Language-Action (VLA) models are powerful but notoriously fragile when faced with changes in camera viewpoint or object configuration. The authors investigate why these models struggle with spatial generalization, identifying "shortcut learning"—where models latch onto fixed, spurious correlations (like the static relationship between a camera and the robot base)—as the primary culprit. They ask whether a data-centric approach, rather than architectural changes, can force models to learn more robust, view-invariant spatial representations.

The researchers implement a dual-arm robotic setup where one arm performs manipulation while the other acts as a mobile environmental camera. They categorize data collection into three patterns: Fixed (static), Multi-Fixed (static within an episode but varied across episodes), and Moving (continuous camera motion). They propose a hybrid strategy that mixes Multi-Fixed and Moving data to balance training stability with the need to break spurious correlations. They also inject multi-dimensional diversity by randomizing object configurations and relative poses during data collection.

The study reveals that simply adding more static viewpoints is insufficient to prevent shortcut learning. Instead, the hybrid strategy—specifically a mix of Moving and Multi-Fixed data—effectively mitigates spurious correlations. The authors identify an empirically optimal "Golden Ratio" (Moving:Multi-Fixed = 1:3) for their primary model, Gr00t. Crucially, they show that this susceptibility to shortcut learning is a universal issue across architectures, including ACT, Diffusion, and various VLA models, all of which see significant performance gains when trained on this hybrid dataset. Furthermore, they demonstrate that spatial knowledge learned from a simple, low-cost auxiliary task (like pen pick-and-place) can be transferred to improve the robustness of policies on entirely new, more complex tasks.