ACT WITH INTENT: DISTILLING BEHAVIOR INTENT FOR VISION-LANGUAGE-ACTION MODELS

Vision-Language-Action (VLA) models excel at translating multimodal inputs into continuous robot actions, but their action decoders are typically trained entirely through behavior cloning. While behavior cloning successfully teaches a model which motor command to execute, it leaves the local objective or intent of that behavior completely implicit. Consequently, similar physical movements that serve different semantic objectives under a given instruction are treated without structural guidance. This paper asks whether explicitly modeling the semantic objective of forthcoming behavior can improve action decoding and generalization in robotic policies.

The authors propose Intention Distillation (INDI), a training framework that injects behavior-level intent into pretrained VLA action decoders. During training, a frozen teacher VLM observes the current observation, language instruction, coarse action summary, and execution video to interpret the demonstrated segment. The student VLA decoder is then trained to recover this multimodal intent representation at an intermediate layer using learnable intent queries. To ensure the recovered latent acts as a true functional state rather than an auxiliary loss, the architecture uses asymmetric attention and an intent-mismatch penalty that forces downstream action and grounding predictions to depend specifically on the correct intent.

Experiments across multiple simulation benchmarks and real-world environments demonstrate substantial performance gains. On SimplerEnv-Bridge, INDI improves the baseline GR00T-N1.7 success rate from 64.3% to 84.7%, with particularly large gains on challenging tasks like placing an object into a basket. On the RoboCasa Kitchen benchmark, INDI improves the controlled baseline from 64.1% to 70.3% across 24 tasks, and shows consistent improvements when applied to the pi_0.5 architecture. In real-world evaluations, INDI increases average success from 62.0% to 68.7%, exhibiting robust generalization to held-out objects and distracting items.