A Unified Stereo Geometry Estimation Framework for Disparity and Surface Normal

Stereo matching and surface normal estimation are fundamental to 3D scene understanding, yet existing feed-forward methods often struggle in challenging environments such as low-light, highly reflective, or transparent surfaces. The authors investigate whether leveraging powerful diffusion priors can overcome these limitations by creating a unified framework where disparity and surface normals provide complementary geometric guidance to one another.

The authors propose GeoStereo, which integrates a stereo matching branch with a diffusion-based normal estimation branch. The stereo branch uses a hybrid cost-volume approach to predict initial and refined disparity. This disparity is then converted into a coarse surface-normal map (disparity-to-normal initialization) and used to warp the right-view image into the left-view coordinate system. These outputs serve as geometry-aware priors and cross-view conditions for a denoising U-Net, which produces the final surface normal map. The framework is trained using a consistency strategy where gradients from the normal objective propagate back to the stereo branch, allowing the two tasks to refine each other.

GeoStereo demonstrates robust performance in challenging real-world scenarios where traditional stereo methods typically fail. In zero-shot experiments, the model achieves state-of-the-art results on standard benchmarks, including Rank-1 performance for disparity estimation on KITTI and NYUv2, and superior normal estimation accuracy on datasets like iBims-1 and ScanNet. The results confirm that the coupling mechanism effectively uses diffusion priors to enhance structural integrity in ill-posed regions while using stereo geometry to guide normal prediction.

By bridging the gap between traditional stereo matching and generative diffusion models, this work provides a more reliable way to reconstruct 3D geometry in complex, real-world environments. This is particularly valuable for downstream robotics and autonomous driving applications that require precise depth and surface orientation in non-ideal lighting or material conditions.