UniDrive: A Unified Vision-Language and Grounding Framework for Interpretable Risk Understanding in Autonomous Driving

Autonomous driving systems must navigate complex, safety-critical environments where identifying hazards requires both temporal reasoning (how a situation evolves) and spatial precision (where a small or distant object is located). Existing multimodal large language models (MLLMs) often struggle to balance these two needs, frequently providing either coarse scene descriptions without precise grounding or high-level reasoning that misses fine-grained visual evidence. This paper asks: can a unified framework that explicitly fuses temporal semantics with high-resolution spatial perception improve the interpretability and safety-oriented grounding of autonomous driving systems?

The authors propose UniDrive, a framework consisting of two synergistic branches. The Temporal Reasoning Branch (T-RB) processes a sequence of video frames to capture scene dynamics and infer risk-related intentions. Simultaneously, the High-Resolution Perception Branch (P-B) analyzes the most recent frame at a higher resolution to preserve fine-grained spatial details. These two streams are integrated via a gated cross-attention fusion module, which aligns dynamic context with precise spatial evidence. The fused representation is then fed into a large language model to jointly generate natural-language risk descriptions and grounded bounding-box outputs for identified hazards.

UniDrive demonstrates superior performance on the DRAMA-Reasoning benchmark compared to representative image-based and video-based baselines. The model excels in both captioning accuracy and risk-object localization. Notably, it shows significant advantages in localizing small objects—a common failure point for standard MLLMs—and maintains strong performance in zero-shot generalization tests on the NuScenes and BDD100K datasets. Human-centered evaluations further indicate that UniDrive provides more interpretable and trustworthy outputs, suggesting that the dual-branch fusion strategy effectively bridges the gap between semantic reasoning and visual grounding.