CoRE: Weakly Supervised Coarse-to-Fine Risk Evidence Learning in Driving Videos

Driving risk is inherently dynamic and tied to specific scene entities, yet most datasets only provide coarse, video-level risk labels. This paper addresses the challenge of recovering fine-grained temporal and entity-level support for these risk predictions without the need for costly, manual fine-grained annotations.

The authors introduce CoRE, a framework that uses a teacher-student distillation strategy. First, a coarse video-level predictor is trained and frozen. The researchers then perform structured interventions—systematically masking or perturbing temporal regions and entity tracks—to measure how each candidate influences the coarse prediction. These measured prediction effects are converted into graded targets, which serve as supervision for a student model. During training, the student learns to directly predict these fine-grained support scores from the original video. At inference, the student provides both the coarse risk assessment and the fine-grained support in a single, efficient forward pass.

CoRE demonstrates that coarse video supervision is sufficient to recover meaningful fine-grained evidence. On the RISEE dataset, CoRE successfully identifies risk-supporting temporal regions and entities using only clip-level subjective ratings. On the DoTA benchmark, it achieves state-of-the-art temporal localization performance compared to existing weakly supervised methods, despite never seeing temporal event labels during training. Furthermore, the framework generalizes well to non-driving tasks, showing competitive performance on the UCF-Crime anomaly detection benchmark. Ablation studies confirm that the prediction-effect supervision is the primary driver of these performance gains.

This work bridges the gap between coarse video-level labels and the need for explainable, fine-grained risk understanding in autonomous driving. By automating the discovery of supporting evidence, CoRE reduces the reliance on expensive human annotations, making it easier to train models that are both accurate and interpretable in safety-critical environments.