QCA: Query- and Content-Aware Keyframe Selection for Long Video Understanding

Existing Multimodal Large Language Models (MLLMs) often struggle with long-form videos due to severe temporal redundancy and limited token budgets. While uniform sampling is a common approach to compress video input, it frequently misses semantically critical moments. This paper addresses the challenge of selecting a compact, information-rich subset of frames that maximizes reasoning performance for long videos without requiring additional model training.

The authors propose the Query- and Content-Aware (QCA) framework, which operates in two main stages. First, it performs inter-segment keyframe allocation: the video is partitioned into temporal segments, and each segment is assigned a frame budget based on a combination of semantic matching (how relevant the segment is to the user query) and content deviation (how visually distinct or diverse the segment is compared to the rest of the video).

Second, it performs intra-segment keyframe selection: within each segment, the model anchors on the most query-relevant frame and then iteratively selects additional frames that maximize semantic diversity. This greedy approach ensures that the selected frames are not only relevant to the query but also provide a comprehensive overview of the video content, avoiding redundant information.

QCA provides a training-free, model-agnostic solution that can be integrated into existing Video-LLMs. By prioritizing informative frames over uniform sampling, the framework significantly enhances performance on long video benchmarks like LongVideoBench and Video-MME. It demonstrates that intelligent frame selection is a highly efficient way to improve reasoning capabilities, allowing models to achieve state-of-the-art results even under strict frame constraints.