Better Slots, Better Worlds: Representation Quality & Robustness in Object-Centric World Models

This paper investigates the relationship between object-centric representation quality and downstream planning performance in world models. While object-centric (OC) models are theoretically appealing for their compositional structure, prior research has often conflated the benefits of the OC inductive bias with auxiliary training techniques (like slot masking or proprioception) and has not rigorously tested whether better-bound slots actually lead to better planning.

The authors conduct a controlled study using visual model-predictive control in 2D (PushT) and 3D (OGBench-Cube) environments. They decouple the slot encoder from the dynamics model, training world models on various checkpoints from the SlotContrast encoder to measure how planning success scales with unsupervised slot-quality metrics (FG-ARI and mBO). They further compare their object-centric world model (SlotContrast-WM) against two scene-centric baselines: DINO-WM (using frozen DINOv2/v3 patch features) and LeWM (an end-to-end trained model using a global CLS token).

1. Quality Correlation: Planning success is positively correlated with unsupervised slot-quality metrics. However, these performance gains saturate once the slots are sufficiently well-bound, suggesting that extreme precision in segmentation is not required for effective planning.
2. Redundancy of Auxiliary Objectives: With high-quality slots, the auxiliary proprioception inputs and slot-masking inductive biases—previously considered essential in models like C-JEPA—become unnecessary. These techniques primarily serve to compensate for poor-quality slot representations.
3. Robustness: Object-centric models and DINO-WM (which uses frozen pretrained features) exhibit significantly higher robustness to distribution shifts (such as appearance or background changes) compared to the end-to-end trained LeWM. This indicates that the use of frozen, pretrained visual foundations is a primary driver of robustness in modern world models.