Kaican Li, Weiyan Xie, Lewei Yao, Jiannan Wu, Lanqing Hong, Yongxiang Huang, Nevin L. Zhang
6 min
Abstract
Long-document understanding often requires reasoning over many visually rich pages, making inference costly and prone to context rot. In this work, we propose InSight-doc, an agentic visual perception framework that treats visual resolution as an adaptive reasoning-time resource. InSight-doc starts from low resolution and selectively zooms into high-resolution regions for finer evidence, without relying on any external retriever. To train such an agent, we construct an active-perception corpus of 17.9K high-quality SFT examples with region-level zoom-in trajectories, accompanied by 19.2K hard RL examples. Through SFT+RL, InSight-doc-8B improves the baseline by 4.3--16.4 accuracy points over document VQA benchmarks. On long documents, it reduces hallucination by more than 40% and inference latency by 41%--68% while maintaining an accuracy lead. Our code, datasets, and model are released at https://github.com/m-Just/InSight-doc .
Alex: That's where the training comes in. The researchers built a specialized dataset to teach the model a "zoom-in trajectory"—think of it as a worked example of good reading strategy. The trajectory shows the model the full path: here's the blurry overview, here's the reasoning step, here's the zoom, here's what you find, here's the answer.
Sam: So they're showing it how an expert would navigate the document, step by step.
Alex: Exactly. And they do this in two stages. The first stage is called supervised fine-tuning—the model watches expert examples and learns to imitate them. That's like a student shadowing a librarian, observing how they search efficiently. The second stage is reinforcement learning, where the model practices on its own and gets feedback based on whether its final answer is correct. That's the student going off and doing research independently, learning from their mistakes.
Sam: Show them the process, then let them practice until they're reliable.
Alex: Right. And the result of combining those two stages is that the model learns to weave thinking and action together. It doesn't just retrieve and report—it reasons, zooms, reads, updates its reasoning, and then answers. The paper calls this a "multimodal chain-of-thought," because the chain of reasoning includes both text and visual evidence.
Sam: Does that actually make the answers more accurate? Or is it mainly a speed improvement?
Alex: The paper suggests both. By focusing only on relevant patches rather than processing everything at once, the model avoids what they call "diluted attention"—where the signal gets lost in the noise. That focus leads to a meaningful drop in hallucination rates. And on the speed side, for long documents, they report cutting the time to generate an answer by up to sixty-eight percent compared to the high-resolution-everything approach.
Sam: That's a significant gap. Though I'm wondering—what's the catch? Can the model zoom into the wrong place and get stuck?
Alex: That's a fair concern, and the paper acknowledges it. Each zoom-in step adds a small amount of overhead—you're making multiple requests rather than one big batch. But the researchers argue that the savings from not processing every page at full resolution far outweigh those extra steps. The net result is still substantially faster and more accurate.
Sam: And I'd imagine if the model zooms into the wrong spot, it might just... not find what it needs and have to try again?
Alex: That's the idea. And this is actually where the design differs from an older approach. Many document AI systems use what's called an external retriever—a separate piece of software that tries to pick the relevant pages before the main model even sees them. The risk there is that the retriever makes a bad guess, and the model never gets to see the right information at all. InSight-doc removes that separation. The model itself decides where to look, which means its reasoning and its navigation are always in sync.
Sam: So it's not outsourcing the navigation to a separate system that might steer it wrong. The model is responsible for its own search.
Alex: Exactly. It stays grounded because it has to verify details through a visual crop before it commits to an answer. It can't just confabulate—it has to go look.
Sam: That's a meaningful shift. Instead of dumping everything on the model and hoping it copes, you're giving it a way to be selective and deliberate.
Alex: And that's the core contribution of the paper. It's not a faster processor or a bigger model—it's a smarter reading strategy. One that mirrors how a careful human reader actually works. Thanks for listening to ResearchPod.