PolicyTrim: Boosting Intrinsic Policy Efficiency of Vision-Language-Action Models

Modern Vision-Language-Action (VLA) models are often bottlenecked by deployment efficiency. While most research focuses on reducing per-step computational latency (e.g., through quantization or pruning), the authors argue that the intrinsic policy efficiency—determined by how many actions a model can reliably execute before needing a new observation and how many physical steps it takes to complete a task—remains largely ignored. This paper investigates how to optimize these two factors to reduce the total number of forward inference calls required during robotic manipulation.

The authors introduce PolicyTrim, a two-stage reinforcement learning (RL) post-training framework. The first stage, Reliable Chunk Extension, uses a dynamic exploration strategy where trajectories are assigned varying execution window lengths. By rewarding successful completions that utilize longer chunks, the model learns to push its trustworthy prediction horizon toward its empirical limit. The second stage, Redundancy-Aware Step Reduction, introduces a reward function that favors shorter, successful trajectories while applying a group-anchored stability penalty to prevent the policy from collapsing onto unreliable or 'shortcut' behaviors. The framework uses Group Relative Policy Optimization (GRPO) to update the policy without requiring additional expert data or architectural changes.

PolicyTrim significantly improves the operational efficiency of VLA models across three distinct benchmarks. By systematically addressing planning unreliability and action redundancy, the framework improves action chunk utilization by 3x and reduces the total number of physical execution steps by 51.4%. These improvements culminate in an end-to-end deployment speedup of up to 5.83x, demonstrating that optimizing the policy's intrinsic decision-making process is a highly effective, orthogonal strategy to traditional computational acceleration methods.