A Motion-Aware Vector Quantization Framework with Centroid Reuse for Efficient VLA Inference

As Vision-Language-Action (VLA) models grow in size to support embodied AI, their high inference latency on GPUs hinders real-time robotic control. This paper addresses the challenge of deploying these massive models on resource-constrained edge devices by identifying and exploiting redundancies in both memory access and computational operations.

The authors introduce VQVLA, a co-design framework consisting of two primary innovations. First, they propose MotionVQ, a quantization scheme that classifies robot movement into execution states (requiring high precision for fine-grained tasks) and transition states (tolerating lower precision). By monitoring the robot's motion magnitude, the system dynamically switches between high- and low-precision codebooks. Second, they implement a Merged-Centroid Vectorized GEMM paradigm. Instead of dequantizing weights back to full precision, this method performs matrix multiplication directly on the compressed index-codebook representation. It uses spatial aggregation to sum inputs mapped to the same centroid and temporal reuse to cache intermediate results across columns, significantly reducing the total number of multiply-accumulate operations.

Experimental results demonstrate that VQVLA achieves significant speedups compared to existing accelerators, including 6.5x over an A100 GPU and 2.8x over the Dadu-Corki accelerator. The authors show that these performance gains are achieved with negligible degradation in task success rates. The study confirms that the transformer backbone is the primary latency bottleneck in VLA models and that centroid access patterns in vector-quantized weights exhibit high locality, which is effectively exploited by their custom hardware architecture.

This work provides a practical path for deploying large-scale VLA models in real-world robotic systems where low latency and power efficiency are critical. By moving beyond treating models as black-box full-precision workloads, the authors demonstrate that hardware-aware algorithmic optimizations can unlock substantial performance improvements for embodied AI.