DDStereo: Efficient Dual Decoder Transformers for Stereo 3D Road Anomaly Detection

Stereo-based 3D object detection is essential for autonomous driving, offering dense depth information at a lower cost than LiDAR. However, existing methods struggle to balance two critical requirements: real-time inference speed and the ability to detect novel, unseen objects (open-set detection). Current stereo approaches are often too slow for real-time deployment, while existing open-set methods typically rely on computationally expensive language models or complex, non-real-time pipelines.

The authors propose DDStereo, an end-to-end Transformer-based framework that decouples 3D object detection from foreground anomaly detection. The architecture consists of three primary branches:

1. Main 3D Detection Branch: Uses visual features from the left image to regress 3D attributes (size, orientation, and category) for known objects.
2. Foreground Detection Branch: Operates on disparity-aware features derived from stereo correlation volumes to perform binary, category-agnostic localization of potential obstacles.
3. Depth Prediction Branch: A lightweight U-shaped network that reconstructs a high-resolution depth map, allowing for precise object-centric depth sampling.

To ensure efficiency and spatial consistency, the two detection branches share object-level queries. This design eliminates the need for traditional, slow post-processing steps like Non-Maximum Suppression (NMS). An anomaly score is then calculated by comparing the foreground confidence with the classification confidence of known categories.

DDStereo achieves state-of-the-art results on the KITTI benchmark, particularly on moderate and hard difficulty levels. Most notably, it achieves an inference speed of 23.5 ms per frame, making it the first stereo 3D detector to reach performance levels comparable to monocular approaches while maintaining superior accuracy. The dual-decoder design effectively enables the detection of out-of-distribution (OoD) objects without the need for additional text prompts or complex language-based supervision.