DepthART: Scaling Foundation Monocular Depth to Tiny Models

As monocular depth estimation (MDE) shifts toward foundation models, these advancements have largely bypassed tiny, resource-constrained models. The authors investigate why naively shrinking large foundation models fails for on-device deployment, specifically identifying two bottlenecks: overfitting to dataset-specific distribution biases and the loss of transferable geometric knowledge during metric fine-tuning.

To address these limitations, the authors introduce DepthART, a lightweight MDE framework. The approach consists of two main strategies:

1. Bias-Resistant Data Sampling (BRDS): To mitigate overfitting, the authors use a density-aware stratified sampling scheme. By projecting images into a feature space using a VAE and applying PCA, they identify and down-sample overrepresented, dominant visual patterns, ensuring the model trains on a more diverse, balanced dataset.
2. Camera-Conditioned Fine-tuning (CamFT): To prevent catastrophic forgetting during metric adaptation, the authors freeze the distilled encoder. They inject lightweight camera adapters that process pyramid camera prompts (derived from intrinsics) and employ a multi-query scale head. This allows the model to adapt to specific metric scales without overwriting the robust, generalized depth priors learned during the initial distillation phase.

DepthART consistently outperforms previous tiny baselines in both zero-shot generalization and metric accuracy. For example, the DepthART-S variant achieves a zero-shot $\delta_1$ accuracy of 0.964 on the NYUD v2 dataset. The model demonstrates high efficiency, achieving over 100 FPS on an Orin NX and maintaining real-time performance on a Jetson Nano. The authors show that their CamFT protocol significantly reduces the geometric damage typically caused by full fine-tuning, leading to more stable scale estimation across different camera settings.