SKooP: Symmetric Koopman Predictions for Faster and More Generalizable Legged Robot Locomotion with Reinforcement Learning

Reinforcement learning (RL) for high-dimensional robotic systems often suffers from poor sample efficiency and limited generalization. The authors investigate whether combining physics-informed priors—specifically morphological symmetries and Koopman-based linearized dynamics—can enhance the training speed and performance of policies for complex bipedal locomotion on quadruped robots.

The authors introduce SKooP (Symmetric Koopman Predictions), a modular framework that learns a Koopman model of the system dynamics concurrently with the RL policy. The framework employs an Equivariant Controlled Dynamics Auto-Encoder (ecDAE) to map the robot's state into a latent space where dynamics are globally linear. This latent representation is used as a privileged observation for the critic, providing it with a predictive prior that simplifies the value function learning process. Additionally, the architecture enforces group symmetries across the actor, critic, and encoder networks, ensuring the policy is highly equivariant.

SKooP consistently outperforms standard PPO and existing symmetric RL baselines in both convergence speed and final reward across multiple bipedal locomotion tasks (e.g., standing, walking on slopes, and pushing doors). The ablation studies demonstrate that the combination of Koopman predictions and symmetry constraints is critical for achieving high success rates on mirrored, out-of-distribution tasks. By providing the critic with future-state information in the lifted Koopman space, the agent learns more robust and informative features, leading to more stable and symmetric motions.

This work bridges the gap between theoretical model-based control (Koopman theory) and data-driven model-free RL. By embedding physical priors directly into the network architecture, the authors demonstrate that robots can learn complex, high-dimensional behaviors more efficiently without requiring extensive manual tuning or massive amounts of training data. This approach is particularly valuable for legged robotics, where exploiting structural symmetry is essential for robust locomotion.