Video-Deepresearch Team
4 min
As multimodal agents transition from static images to continuous video streams, they face significant challenges in long-horizon reasoning. Current models often suffer from two primary failures: modality bias, where they ignore visual data in favor of text-based search, and parametric knowledge leakage, where they rely on internal training data rather than performing real-time, tool-augmented research. This paper introduces Video-DeepResearch (Video-DR) to address these bottlenecks in complex, multi-hop video question answering.
The authors propose a decoupled perception-exploration pipeline. Instead of allowing the model to choose its tools freely, the agent is forced to follow a stage-wise process: it must first use visual tools (Select_Keyframe and Crop_Search) to ground entities across frames before it is permitted to use web-based search tools. The training process uses a two-stage recipe: Supervised Fine-Tuning (SFT) on 7,000 high-quality, tool-use-heavy trajectories, followed by Group Relative Policy Optimization (GRPO) to refine the agent's autonomous exploration capabilities. To evaluate this, the authors created VideoDR-Bench, a new benchmark of 200 complex, multi-hop VQA instances that require both visual grounding and external knowledge.
Video-DeepResearch-35B-A3B achieves a state-of-the-art 64.0% accuracy on the new benchmark, outperforming proprietary models like Claude-4.5-Sonnet (59.0%), Gemini 2.5 Pro (57.5%), and GPT-5 (52.5%). The results demonstrate that forcing a decoupled, stage-wise tool-unlocking strategy effectively mitigates the tendency of models to bypass visual evidence. Even the smaller 30B-A3B variant remains competitive with proprietary baselines, suggesting that the training paradigm is highly effective for compact models.
This work shifts the focus of multimodal agents from simple image captioning to active, strategic research in video environments. By proving that agents can be trained to prioritize visual evidence over internal memory, the authors provide a scalable blueprint for building agents that can reliably synthesize information from complex, long-form video content.
We introduce Video-DeepResearch (Video-DR), extending multimodal agents from static images to continuous video streams, a setting that demands dense spatiotemporal grounding coupled with open-web exploration. Preliminary evaluations reveal two critical bottlenecks in current models: (1) modality bias, where agents bypass visual tools in favor of textual search, and (2) parametric knowledge leakage, where models rely on internal memory rather than genuine tool-augmented execution. To address these challenges, we propose Video-DR, featuring a decoupled perception-exploration pipeline with stage-wise tool unlocking that compels exhaustive cross-frame visual grounding prior to web retrieval. Our framework adopts a two-stage training recipe: supervised fine-tuning followed by Group Relative Policy Optimization (GRPO), enabling autonomous exploration that breaks the imitation-learning ceiling. Furthermore, we curate Video-DR-Bench, a human-AI collaborative benchmark comprising 200 complex, multi-hop VQA instances. Empirical results demonstrate that our Video-DeepResearch-35B-A3B establishes a new state-of-the-art of 64.0% average accuracy, surpassing proprietary Claude-4.5-Sonnet (59.0%) by 5.0 points and significantly outperforming GPT-5 (52.5%) and Gemini 2.5 Pro (57.5%). The 30B-A3B variant achieves 59.3%, competitive with Claude-4.5-Sonnet and demonstrating the effectiveness of our training paradigm even at compact scale. Code: https://github.com/Osilly/Vision-DeepResearch.
Alex: Oh—so they're training the AI to value visual evidence through trial and error, rather than just telling it what to do.
Sam: Exactly. And the results are notable. Their model outperformed much larger, proprietary systems on a new benchmark the researchers created for this study. The finding is that by changing how an agent is trained to interact with data, you can get better performance than simply throwing more raw computing power at the problem.
Alex: That said, the benchmark itself still required a lot of human effort to build, right?
Sam: Correct. The researchers relied on careful human annotation to make sure the benchmark was accurate, which makes it difficult to scale up the testing process quickly. They note that future work will need to find ways to automate that evaluation and reduce the overall resource requirements.
Alex: So the framework proves the principle works, but making it practical for everyday use is still the next challenge.
Sam: That's a fair assessment. It's foundational work. The real takeaway is that the AI's failure wasn't a lack of raw intelligence—it was a lack of proper structure in how it approached the task. When you force an agent to ground its reasoning in verified visual evidence before drawing conclusions, the quality of its research improves in a meaningful way. It's a shift from guessing to investigating.
Alex: It changes how we think about what "smart" means for an AI. It's not just about what the model knows—it's about how it goes about finding what it doesn't know.
Sam: Precisely. And that distinction may matter quite a bit as these systems take on more complex, real-world tasks.
Alex: That's it for this look at Video-DeepResearch. Thanks for listening to ResearchPod.