Passive models for long video understanding typically rely on a "watch-it-all" paradigm, processing frames uniformly regardless of query difficulty, causing computational cost to grow with video duration. Although interactive frameworks have emerged, they often rely on global pre-scanning, and their context cost still scales with video length. We propose OmniAgent, the first native omni-modal agent that formulates video understanding as a POMDP-based iterative Observation-Thought-Action cycle. OmniAgent executes on-demand actions to selectively distill audio-visual cues into a persistent textual memory, effectively decoupling reasoning complexity from raw video duration. To operationalize this, we introduce (1) Agentic Supervised Fine-Tuning to bootstrap native active perception via best-of-N trajectory synthesis with dual-stage quality control, and (2) Agentic Reinforcement Learning with TAURA (Turn-aware Adaptive Uncertainty Rescaled Advantage), which leverages turn-level entropy to steer credit assignment toward pivotal discovery turns. Crucially, OmniAgent exhibits positive test-time scaling, where performance improves as the number of reasoning turns increases, validating the efficacy of active perception. Empirical results across ten benchmarks (e.g., VideoMME, LVBench) demonstrate that OmniAgent achieves state-of-the-art performance among open-source models. Notably, on LVBench, our 7B agent outperforms the 10$\times$ larger Qwen2.5-VL-72B (50.5% vs. 47.3%).
Alex: Welcome to another episode of ResearchPod. Today, we're looking at a new approach to helping AI understand long videos — a system the researchers call OmniAgent.
Sam: So what's the core problem here? Why is long video such a challenge for AI?
Alex: Most AI video models work by processing every single frame — every image — in sequence. That works fine for a short clip, but as videos get longer, the computing cost grows enormously. A two-hour documentary isn't just twice as hard as a one-hour film; the processing demands can scale up far faster than that.
Sam: That's like trying to memorize an entire textbook word-for-word, when you could just take notes on the important parts.
Alex: That's a good way to put it. The researchers call the old approach the "watch-it-all" paradigm, and they argue it's simply not sustainable. OmniAgent takes a different path — it behaves less like a machine grinding through footage, and more like a detective working a case.
Sam: How does a detective approach actually work for a video?
Alex: Think about how a real investigator handles security footage. They don't sit and watch twelve hours of tape from start to finish. They ask questions: "What time did the alarm go off? Let me jump to that moment. Okay, someone entered through the side door — let me check thirty minutes earlier." They move purposefully, guided by logic.
Alex: OmniAgent does the same thing. It runs on a cycle the researchers call "Observation, Thought, Action." It looks at a small piece of the video, thinks about what it found, decides what to do next, and then acts — jumping to a new timestamp, extracting audio, or reading on-screen text. Then it repeats.
Sam: So it's not passively watching. It's actively choosing where to look, based on what it's already learned.
Alex: Precisely. And here's what makes it efficient: after it examines a clip, it converts what it saw into a short written summary and discards the raw video data. It keeps the notes, not the footage. So its working memory stays the same size whether the video is five minutes or five hours long.
Sam: That's the key, isn't it. The memory doesn't grow with the video length.
Alex: Exactly. The reasoning is completely separated from the length of the content. That's what allows a relatively small model to handle material that would overwhelm a much larger one.
Sam: How does it learn to make good decisions about where to look? That seems like the hard part.
Alex: It is. The researchers use a training approach they call Agentic Supervised Fine-Tuning. They show the model thousands of worked examples — full sequences of a search, including the wrong turns. The model sees a case where it jumped to the wrong timestamp, recognised the mistake, and corrected course. It learns that failure isn't a dead end; it's information.
Sam: So it's trained on the whole messy process, not just the clean final answer.
Alex: Right. And on top of that, they developed a method called TAURA. The name is dense, but the idea is straightforward. Not all moments in a search are equally important. Finding a key piece of evidence is more significant than confirming something you already knew. TAURA teaches the model to weight those "discovery" moments more heavily during training, so it gets better at recognising when it's actually found something that matters.
Sam: Like learning to pay attention at the right moments, not just any moment.
Alex: That's a fair way to put it. There's also a step called a Rationality Audit, where a separate model checks whether the reasoning chain actually holds together logically. This prevents the agent from stumbling onto correct answers through lucky guesses rather than sound thinking — which would make it unreliable in practice.
Sam: Does all of this actually translate into better performance?
Alex: The study suggests it does. On a standard long-video benchmark called LVBench, a version of OmniAgent with around seven billion parameters — that's a rough measure of model size and complexity — outperformed a passive model roughly ten times larger. The smaller, more deliberate system beat the bigger, brute-force one.
Sam: That's a notable result. A much smaller model winning because of how it thinks, not how big it is.
Alex: Though the researchers are clear about the trade-off. This approach depends entirely on the quality of the model's reasoning at each step. If it draws a wrong conclusion early on — misreads a clue, so to speak — it may go looking in entirely the wrong place and never recover. The system is only as reliable as its own logic.
Sam: So it's more efficient, but also more fragile in a specific way.
Alex: That's a fair characterisation. It trades raw computational brute force for something more like directed intelligence — which is more powerful when it works, but requires the reasoning to hold up throughout. The paper frames this as a meaningful step toward AI systems that can handle long-form content by thinking carefully about where to look, rather than simply processing everything in sight. Thanks for listening to ResearchPod.