HarmVideoBench: Benchmarking Harmful Video Understanding in Large Multimodal Models

Large vision-language models (LVLMs) are increasingly deployed for automated content moderation on video-sharing platforms. However, existing benchmarks often reduce the complex task of harmful content detection to a simple binary classification (harmful vs. non-harmful). This approach ignores the multi-layered nature of harmful content—where malice may be implicit or dependent on cultural context—and fails to provide explanatory rationales, effectively treating the model as a black box that may rely on superficial shortcuts rather than genuine understanding.

The authors introduce HarmVideoBench, a diagnostic benchmark consisting of 1,379 videos and 4,137 multiple-choice questions. It moves beyond binary classification by evaluating models across three hierarchical dimensions:

• Observable Evidence: Detecting explicit visual, auditory, or textual cues.
• Clip-Internal Meaning: Inferring implicit harmful messages or intent within the video content.
• Beyond-Clip Reasoning: Synthesizing external cultural, historical, or social context to judge harmfulness.

This structure allows researchers to pinpoint exactly where a model fails, distinguishing between simple perception errors and deeper reasoning deficits.

Evaluating 19 leading models revealed a consistent performance gap: while models are generally proficient at identifying observable evidence, their accuracy drops sharply as tasks require deeper semantic and contextual reasoning. To address this, the authors propose Boundary-Constrained Reasoning (BCR). BCR is a lightweight method that predicts the reasoning scope of a question and retrieves context only when necessary. By implementing this boundary-aware approach, the authors improved the macro-average performance of their base model from 61.7% to 84.4%, demonstrating that explicit reasoning control is essential for robust harmful-video understanding.