Lite Any Stereo V2: Faster and Stronger Efficient Zero-Shot Stereo Matching

As deep learning-based stereo matching models grow in size and complexity, they become increasingly difficult to deploy on resource-constrained platforms like robotics or mobile devices. While efficient stereo models exist, they are often perceived as having poor zero-shot generalization capabilities compared to large-scale foundation models. This paper investigates whether lightweight stereo architectures can achieve competitive zero-shot performance without sacrificing real-world deployment efficiency.

The authors introduce Lite Any Stereo V2 (LAS2), a series of models developed through two primary lenses: architecture and training. Architecturally, the team moves away from traditional 3D cost aggregation—which is computationally expensive—in favor of a 2D-only convolution-based framework. This design is specifically optimized for real-world inference latency rather than theoretical Multiply-Accumulate (MAC) counts.

To ensure strong zero-shot generalization, the authors employ a three-stage training strategy:

1. Supervised Training: Initial training on a large-scale synthetic dataset.
2. Self-Distillation: A teacher-student setup where the student learns to handle perturbed inputs, improving robustness.
3. Real-World Knowledge Distillation: Fine-tuning on unlabeled real-world data using a teacher model. To prevent the model from learning from noisy pseudo-labels, the authors introduce a filtering mechanism (using left-right consistency checks and edge/sky masks) and an error-clamping operation to ensure stable synthetic-to-real transfer.

Findings

LAS2 demonstrates that lightweight models can indeed bridge the gap between efficiency and accuracy. The model family includes feed-forward variants (S, M, L) for different budgets and an iterative variant (H) for higher precision. Experimental results show that LAS2-M outperforms previous state-of-the-art efficient methods while running significantly faster on hardware like the H200 and Orin NX. Furthermore, the iterative LAS2-H variant achieves superior zero-shot performance compared to existing iterative foundation-based methods while maintaining a substantial speed advantage.