Deltoris: Enabling Real-time VLA Inference in Embodied AI via Bit-level Sparsity and Speculative Inference

Diffusion-based Vision-Language-Action (VLA) models are essential for high-quality robotic control but are computationally intensive, often failing to meet the 50–200 Hz real-time requirements of embodied AI. This paper addresses the challenge of enabling real-time, energy-efficient inference for these models on edge devices.

The authors propose Deltoris, an algorithm-hardware co-design framework. The algorithm component, temporal-aware bit-level sparsity, exploits the high similarity between consecutive robot observations. Instead of recomputing the entire model, it calculates only the differences (deltas) between consecutive inputs, significantly reducing redundant bit-level operations. To manage the increased off-chip memory traffic caused by this differential approach, the authors introduce speculative inference, where a lightweight model predicts future actions that a larger model validates, effectively amortizing data loading costs. Finally, the hardware component features a custom 1D systolic bit-serial processing element (PE) array designed to eliminate workload imbalances common in bit-serial architectures.

Deltoris achieves substantial performance improvements over existing platforms. Experimental results demonstrate up to 34.2× speedup compared to mobile GPUs and 6.1× speedup over prior dedicated accelerators, while maintaining comparable task success rates. The framework also achieves significant energy savings, making it highly suitable for resource-constrained robotic platforms.

As embodied AI moves toward more complex, real-world tasks, the demand for high-frequency, low-latency control increases. By proving that temporal redundancy can be effectively exploited in diffusion-based VLA models, this work provides a scalable path for deploying sophisticated generative models on edge hardware, bridging the gap between high-performance research models and real-world robotic deployment.