Humanoid-OmniOcc: Stereo-Based Full-View Occupancy Dataset for Embodied AI

As humanoid robots increasingly operate in human-centered indoor spaces, they require reliable, near-field 3D perception to navigate and interact safely. Existing occupancy datasets are heavily biased toward autonomous driving (forward-facing, long-range, static road scenes) or rely on expensive, cumbersome LiDAR sensors. This paper addresses the lack of a panoramic, stereo-based occupancy benchmark specifically tailored for the egocentric, 360-degree sensing needs of humanoid robots.

The authors present Humanoid-OmniOcc, a dataset featuring 155K+ samples across 15 simulated and 5 real-world indoor environments. The dataset is built on a Real2Sim2Real closed-loop paradigm: physical sensor specifications (intrinsics, extrinsics, baseline) from a Unitree G1 robot are replicated in NVIDIA Isaac Sim to generate high-fidelity, annotated training data. This simulation-first approach allows for centimeter-level occupancy ground truth.

To process this data, the authors propose the Humanoid Surround Stereo-guided Occupancy model (HS2Occ). Unlike monocular methods that suffer from depth ambiguity, HS2Occ uses stereo matching to construct depth-oriented cost volumes. These depth priors are then used to lift 2D image features into a 3D voxel grid via a transformer-based projection, which is subsequently refined by a 3D convolutional decoder to predict occupancy states.

Experiments demonstrate that the HS2Occ model consistently outperforms monocular baselines in occupancy prediction accuracy. By leveraging stereo-derived depth priors, the model exhibits stronger geometric reliability and better generalization to unseen simulated scenes and real-world environments. The study validates that the Real2Sim2Real design effectively bridges the domain gap, allowing models trained in simulation to perform robustly on physical hardware without requiring expensive LiDAR sensors.