Jetson-PI: Towards Onboard Real-Time Robot Control via Foresight-Aligned Asynchronous Inference

Deploying Vision-Language-Action (VLA) models on low-power edge devices like the NVIDIA Jetson Orin is hindered by high computational complexity, leading to significant inference latency and low control frequencies. While asynchronous inference can mask this latency, it creates two primary issues: perception-execution misalignment (where the environment changes before the action is executed) and long reaction times. This paper investigates how to enable real-time, efficient VLA deployment on edge hardware without sacrificing task success rates.

The authors introduce Jetson-PI, a framework designed to optimize VLA deployment through two main innovations. First, they implement Foresight-Aligned Asynchronous Correction, which uses a lightweight (40M parameter) module to predict future environment representations conditioned on committed actions. This allows the action expert to predict actions based on the future state rather than outdated observations. Second, they introduce Confidence-based Scheduling Optimization, which dynamically balances the invocation frequency of the VLM and the action expert based on the predicted confidence of the environment state. The system is further accelerated through hardware-specific optimizations, including CUDA graph reuse, GPU-resident intermediate buffering, and flow unrolling, adapted from the llama.cpp framework.

Jetson-PI significantly outperforms existing methods on edge hardware. On the NVIDIA Jetson Orin, the framework achieves 8.66x and 5.41x improvements in control frequency compared to naive PyTorch and vla.cpp implementations, respectively. Furthermore, it demonstrates a 14.8% improvement in average success rate on the LIBERO benchmark compared to the VLASH baseline. The authors also demonstrate that, unlike high-end GPUs where parallelizing VLM and action expert tasks is beneficial, edge devices are primarily bandwidth-bound, making sequential, optimized scheduling more effective than parallel execution.