Local Motion Matters: A Deconstruct–Recompose Paradigm for Reinforcement Learning Pre-training from Videos

How can we improve the sample efficiency of reinforcement learning (RL) by leveraging large-scale, unlabeled video data for pre-training, while overcoming the morphological mismatch between different agents?

The authors introduce the Deconstruct-Recompose Paradigm (DRP), which shifts from global motion modeling to a local modeling approach. The process consists of two phases:

1. Deconstruct Phase: The system extracts 'Atomic Actions' by tracking motion-salient keypoints across video frames and cropping local optical flow patches. A Dual-Attention Encoder (DAE) then learns spatiotemporal relationships among these patches using both Intra-Frame and Inter-Frame attention mechanisms, trained via a Masked Autoencoder (MAE) objective.
2. Recompose Phase: A learnable Motion Aggregation Token ([MAT]) aggregates these local representations. This is integrated into a latent dynamics model that learns to predict future states, effectively imbuing the local representations with dynamic semantics. Finally, an adapter bridges these representations to the specific action space of the downstream agent.

DRP significantly outperforms existing global-modeling pre-training methods (like IPV and PreLAR) across diverse robotic control and manipulation tasks. Experiments on the DMControl Remastered and Meta-World benchmarks demonstrate that DRP achieves superior sample efficiency and asymptotic performance. The ablation studies confirm that deconstructing global motion into local components is essential for effective cross-domain transfer, and that both the Intra-Frame and Inter-Frame attention mechanisms are critical for capturing robust motion patterns.

Traditional RL pre-training often fails to generalize because global motion patterns are tightly coupled to an agent's specific body structure (morphology). By focusing on local motion components—which are shared across many different agents—this paradigm enables more effective knowledge transfer from large-scale, 'in-the-wild' video datasets to specific robotic tasks, drastically reducing the amount of online interaction required for an agent to learn a new skill.