Kaican Li, Weiyan Xie, Lewei Yao, Jiannan Wu, Lanqing Hong, Yongxiang Huang, Nevin L. Zhang
6 min
Long-document understanding with Multimodal Large Language Models (MLLMs) faces a fundamental trade-off: high-resolution inputs are necessary for reading fine-grained visual details, but they lead to excessive computational costs and context rot. The authors investigate whether an agentic approach—where the model treats visual resolution as an adaptive resource—can overcome these limitations without relying on external retrieval systems.
InSight-doc introduces a coarse-to-fine reasoning framework. Instead of processing all pages at high resolution, the model begins with a low-resolution overview of the document. During the reasoning process, the model can emit tool calls to zoom into specific regions of interest. These crops are then appended to the visual context, allowing the model to perform multi-hop, interleaved reasoning. To train this capability, the authors constructed a dataset of 17.9K supervised fine-tuning (SFT) trajectories and 19.2K reinforcement learning (RL) examples, teaching the model how to decide when and where to zoom.
InSight-doc-8B demonstrates substantial improvements over baseline models across various document VQA benchmarks. By dynamically acquiring visual evidence, the model achieves:
This work provides a scalable, retriever-free solution for processing visually rich, multi-page documents. By moving away from static, high-resolution input pipelines, InSight-doc offers a more human-like reading strategy that balances computational efficiency with the need for high-fidelity visual information, making it highly applicable to real-world document analysis tasks.
Long-document understanding often requires reasoning over many visually rich pages, making inference costly and prone to context rot. In this work, we propose InSight-doc, an agentic visual perception framework that treats visual resolution as an adaptive reasoning-time resource. InSight-doc starts from low resolution and selectively zooms into high-resolution regions for finer evidence, without relying on any external retriever. To train such an agent, we construct an active-perception corpus of 17.9K high-quality SFT examples with region-level zoom-in trajectories, accompanied by 19.2K hard RL examples. Through SFT+RL, InSight-doc-8B improves the baseline by 4.3--16.4 accuracy points over document VQA benchmarks. On long documents, it reduces hallucination by more than 40% and inference latency by 41%--68% while maintaining an accuracy lead. Our code, datasets, and model are released at https://github.com/m-Just/InSight-doc .
Alex: That's where the training comes in. The researchers built a specialized dataset to teach the model a "zoom-in trajectory"—think of it as a worked example of good reading strategy. The trajectory shows the model the full path: here's the blurry overview, here's the reasoning step, here's the zoom, here's what you find, here's the answer.
Sam: So they're showing it how an expert would navigate the document, step by step.
Alex: Exactly. And they do this in two stages. The first stage is called supervised fine-tuning—the model watches expert examples and learns to imitate them. That's like a student shadowing a librarian, observing how they search efficiently. The second stage is reinforcement learning, where the model practices on its own and gets feedback based on whether its final answer is correct. That's the student going off and doing research independently, learning from their mistakes.
Sam: Show them the process, then let them practice until they're reliable.
Alex: Right. And the result of combining those two stages is that the model learns to weave thinking and action together. It doesn't just retrieve and report—it reasons, zooms, reads, updates its reasoning, and then answers. The paper calls this a "multimodal chain-of-thought," because the chain of reasoning includes both text and visual evidence.
Sam: Does that actually make the answers more accurate? Or is it mainly a speed improvement?
Alex: The paper suggests both. By focusing only on relevant patches rather than processing everything at once, the model avoids what they call "diluted attention"—where the signal gets lost in the noise. That focus leads to a meaningful drop in hallucination rates. And on the speed side, for long documents, they report cutting the time to generate an answer by up to sixty-eight percent compared to the high-resolution-everything approach.
Sam: That's a significant gap. Though I'm wondering—what's the catch? Can the model zoom into the wrong place and get stuck?
Alex: That's a fair concern, and the paper acknowledges it. Each zoom-in step adds a small amount of overhead—you're making multiple requests rather than one big batch. But the researchers argue that the savings from not processing every page at full resolution far outweigh those extra steps. The net result is still substantially faster and more accurate.
Sam: And I'd imagine if the model zooms into the wrong spot, it might just... not find what it needs and have to try again?
Alex: That's the idea. And this is actually where the design differs from an older approach. Many document AI systems use what's called an external retriever—a separate piece of software that tries to pick the relevant pages before the main model even sees them. The risk there is that the retriever makes a bad guess, and the model never gets to see the right information at all. InSight-doc removes that separation. The model itself decides where to look, which means its reasoning and its navigation are always in sync.
Sam: So it's not outsourcing the navigation to a separate system that might steer it wrong. The model is responsible for its own search.
Alex: Exactly. It stays grounded because it has to verify details through a visual crop before it commits to an answer. It can't just confabulate—it has to go look.
Sam: That's a meaningful shift. Instead of dumping everything on the model and hoping it copes, you're giving it a way to be selective and deliberate.
Alex: And that's the core contribution of the paper. It's not a faster processor or a bigger model—it's a smarter reading strategy. One that mirrors how a careful human reader actually works. Thanks for listening to ResearchPod.