ResearchPod Summary
Autonomous systems often rely on large foundation models for Human Mesh Recovery (HMR) to understand their environment. However, these models are typically too computationally expensive and power-hungry to run on edge platforms, especially for active-perception tasks where a robot must continuously plan its next-best view (NBV) without cloud connectivity. This paper investigates how to compress these large HMR models to enable real-time, on-board execution on resource-constrained edge hardware.
The authors introduce LEAP-NBV, a framework that combines knowledge distillation and post-training quantization. First, they distill a family of large HMR teacher models into a compact 32M-parameter student model using an offline mesh-matching objective. This process allows the student to learn from the teacher's performance without requiring ground-truth labels during the distillation phase. Second, they apply FP16 quantization to the student model to further reduce its memory footprint and latency. The authors systematically evaluate the accuracy-latency trade-offs of these compressed models on an NVIDIA Jetson Xavier NX, specifically testing them within an occlusion-aware active-perception loop.
The distillation process successfully recovers 6–7 mm of Procrustes-aligned mean per-vertex position error (PA-MPVPE) compared to an undistilled student of the same size. The authors identify a critical numerical stability issue: naive FP16 quantization causes overflow in the final normalization layer for wider models (like ViT-H), leading to catastrophic mesh drift. Because the distilled student has a narrower embedding width, it remains numerically safe under FP16, allowing for a uniform, high-performance deployment. On the Jetson Xavier NX, the final pipeline runs at 3.6 FPS and 2.6 J per frame, significantly outperforming the uncompressed baseline in both speed and energy efficiency while preserving downstream task quality.
This research demonstrates that foundation-model-driven perception can be effectively deployed on edge devices by carefully characterizing compression effects directly on the target hardware. By validating the model within a closed-loop active-perception system, the authors show that the accuracy-latency trade-offs are manageable, making sophisticated human-aware autonomy feasible for tactical and field-deployed mobile robots.
AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.