Slot-RAE: Streamlining Object-Centric Learning via Direct Representation Auto-Encoders

Traditional object-centric learning (OCL) models often struggle to balance accurate scene decomposition with high-quality image generation. While recent diffusion-based models have improved visual fidelity, they typically rely on heavy, pre-trained generative priors (like Stable Diffusion) and external VAE latent spaces, which introduce architectural complexity and task-agnostic biases. This paper asks whether generative object-centric modeling can be achieved entirely within the semantic feature space of a vision foundation model (VFM), bypassing these bottlenecks while maintaining strong performance.

The authors introduce Slot-RAE, a framework that integrates slot-based scene decomposition with feature-space diffusion. Instead of mapping images to VAE latents, Slot-RAE extracts dense features using a frozen DINO encoder. These features are processed by a Slot Attention module to generate object slots, which then condition a Diffusion Transformer (DiT) trained from scratch to denoise the semantic feature map. To ensure stability and semantic consistency, the authors incorporate a Representation Alignment (REPA) head that aligns the DiT's intermediate outputs with the target DINO features. Finally, a frozen RAE decoder is used to map the generated features back to pixel space for visualization and evaluation.

Slot-RAE demonstrates that foundation-model feature spaces are naturally suited for both unsupervised object discovery and high-fidelity generative modeling. Experiments on the COCO and Pascal VOC datasets show that the model achieves state-of-the-art results in unsupervised object discovery, matching or exceeding existing deterministic baselines. Furthermore, because the model operates directly in the semantic feature space, it supports robust zero-shot compositionality—allowing users to swap object slots between images to create coherent new scenes—without the computational overhead or structural artifacts associated with traditional VAE-based diffusion pipelines.