Anticipate Before Acting: Future-State-Conditioned Vision-Language Navigation

Standard end-to-end vision-language navigation (VLN) models typically rely on behavior cloning, which supervises only the immediate next action. This approach often fails to capture the long-term visual consequences of navigation decisions, leading to accumulated errors over long trajectories. The authors investigate whether explicitly training a policy to anticipate future visual states—without requiring future observations during deployment—can improve navigation accuracy.

The authors propose Future-State-Conditioned VLN (FSC-VLN), a lightweight modification to causal vision-language models. During training, the model is augmented with a learnable future-query token and an action-query token. The future-query token is trained to align its hidden state with a frozen visual embedding of an observation 32 steps ahead in the expert trajectory. This future-target branch is strictly for training; it is discarded at inference. Because the future query precedes the action query in the causal sequence, the action-decoding process is implicitly conditioned on the model's learned anticipation of future visual outcomes.

Experiments on the R2R benchmark demonstrate that FSC-VLN consistently outperforms the StreamVLN baseline in Success Rate (SR), Oracle Success Rate (OSR), and Success weighted by Path Length (SPL). The performance gains are particularly significant for long-horizon episodes, where the ability to anticipate future states helps the agent maintain trajectory consistency. Ablation studies confirm that separating the future-query and action-query roles is critical for performance, as it allows the model to decouple predictive state representation from immediate action selection.