Beyond Visual Ambiguity: Guiding Robust Monocular Depth Estimation in Challenging Scenarios via Detailed Long Captions

Monocular depth estimation (MDE) often struggles with visual ambiguities caused by non-Lambertian surfaces (e.g., glass, mirrors) and adverse weather conditions (e.g., rain, fog). While general MDE models have advanced, they remain limited by the inherent ill-posed nature of single-image depth prediction. This paper investigates whether incorporating detailed, long-form language descriptions can provide the explicit spatial guidance necessary to resolve these ambiguities and improve robustness.

The authors propose CapDepth, a framework that integrates language guidance through three primary innovations:

1. Detailed Long Caption Template: Unlike previous methods that use short, generic labels, this template uses structured atom sentences to explicitly define spatial relationships between objects in the scene.
2. Dynamic Caption Encoder: This component uses progressive masked attention to extract fine-grained, depth-relevant features from the text, focusing on spatial descriptors while suppressing irrelevant tokens.
3. Text-Adaptive Decoder: To bridge the gap between noise-prediction features (typical of diffusion-based models) and the geometric precision required for depth, the decoder uses stable adaptive layer normalization (SAdaLN) to inject text-derived spatial cues directly into the depth decoding process.

CapDepth demonstrates superior performance compared to existing state-of-the-art MDE methods. Experimental results on benchmarks such as Booster, ClearGrasp, and nuScenes show that the model achieves a 25.0% reduction in depth error on non-Lambertian surfaces and a 22.0% reduction under adverse weather conditions. The study confirms that the density of spatial information in the input text is a critical factor in the model's ability to resolve visual ambiguity.

This work highlights that language is not merely a label for classification but a powerful modality for geometric reasoning. By moving beyond simple text inputs to detailed spatial descriptions, the authors provide a unified, robust solution for MDE that performs well across diverse, challenging environments without requiring scenario-specific augmentations or disjointed training strategies.