VOiLA: Vectorized Online Planning with Learned Diffusion Model for POMDP Agents

How can autonomous robots perform effective planning under uncertainty in complex, real-world environments without requiring massive amounts of task-specific training data or computationally prohibitive online models? The authors address the challenge of obtaining faithful POMDP models by integrating learned generative models with massively parallel online planning.

VOiLA (Vectorized Online planning wIth Learned diffusion model for POMDP Agents) introduces a framework that learns transition and observation samplers offline using conditional diffusion models. To overcome the high computational cost of iterative diffusion sampling, the authors distill these models into compact, feedforward generators. These generators are then integrated into the Vectorized Online POMDP Planner (VOPP), which leverages GPU parallelization to perform belief-space planning. The system also learns a contrastive likelihood model to facilitate particle-based belief updates, allowing the agent to maintain a belief state even in continuous observation spaces.

The distillation strategy reduces sampling costs by nearly three orders of magnitude, making learned generative models viable for real-time online planning. In benchmark evaluations, VOiLA achieved performance comparable to or better than Recurrent Soft Actor Critic (SAC) while utilizing less than 10% of the training data. Furthermore, VOiLA demonstrated superior generalization to unseen environment configurations. Physical robot experiments with a quadruped confirmed that models trained solely in simulation could be deployed directly to successfully complete tasks in 10 out of 10 trials.