FlowMaps: Modeling Long-Term Multimodal Object Dynamics with Flow Matching

As robots become more integrated into household environments, they face the challenge of "object permanence"—objects are frequently moved by humans, making previous observations unreliable. The authors ask whether a generative model can learn the latent, repetitive patterns of human-induced object displacement to predict where objects will be located in the future, thereby improving robotic navigation and retrieval tasks.

The authors introduce FlowMaps, a generative model that treats object displacement as a multimodal distribution in continuous 3D space. The architecture consists of two primary components:

1. Variational Autoencoder (VAE): Encodes object geometry and semantic labels into a compact latent space.
2. Latent Flow Matching (FM) Network: A Transformer-based model that learns to transport a simple Gaussian distribution to the target distribution of future object locations. By using a Conditional Diffusion Transformer (CDiT) structure, the model conditions its predictions on the current scene layout, the object's semantic label, and the desired time horizon.

The model is trained on procedurally generated household environments (ProcTHOR) where objects follow specific human-like routines. This allows the model to learn generalizable priors about how objects move in relation to room layouts and other objects.

FlowMaps demonstrates superior performance in predicting future object locations compared to baselines that rely on static frequency priors or large language models. In over 600 simulated Object Navigation (ObjNav) episodes, FlowMaps achieved a 7.6% improvement in minimum Final Displacement Error (minFDE) over the strongest baseline. Crucially, the model successfully generalizes to previously unseen household layouts, proving that it learns transferable spatial-temporal patterns rather than just memorizing specific training environments. The authors also successfully validated the approach on a real-world robotic platform.