ResearchPod Summary
Embodied AI agents often struggle with navigation in complex indoor environments due to ambiguous instructions. The DialNav framework addresses this by enabling a dialog-based loop where a Navigator agent can ask a remote Guide for help. However, the original DialNav benchmark was limited by a severe scarcity of training data (only 2K episodes), making it difficult to train robust models. This paper investigates how to overcome this data bottleneck through automatic augmentation and improved training strategies.
The authors propose a three-stage pipeline to generate the RAINbow dataset, which contains 238K episodes—an increase of over two orders of magnitude compared to the original data. First, they concatenate existing Vision-and-Language Navigation (VLN) paths to create longer, multi-turn trajectories. Second, they use a vision-language model to generate scene captions at specific dialog points. Finally, they use an LLM to reformat these captions and existing instructions into natural, multi-turn dialogs.
To effectively utilize this large-scale data, the authors introduce two key improvements:
This work demonstrates that high-quality, synthetic data can effectively bridge the gap in embodied AI tasks where human annotation is prohibitively expensive. By scaling the training data and aligning the training scheme with the dynamic nature of dialog-based navigation, the authors achieve a new state-of-the-art performance, with success rates increasing by 89% on seen environments and 100% on unseen environments. This provides a scalable path forward for training more reliable and communicative embodied agents.
AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.