Xuehang Guo, Pingyue Zhang, Ruiyi Zhang, Zhenhailong Wang, Hanrui Lyu, Heng Ji, Tong Sun, Qingyun Wang, Manling Li
5 min
Abstract
Chart question answering (CQA) requires multimodal large language models (MLLMs) to integrate visual comprehension with logical reasoning, yet current models struggle with accurate visual grounding and coherent reasoning chains. While extrinsic chain-of-thought prompting and visual cues significantly improve performance, current MLLMs lack intrinsic visual grounded reasoning capabilities, leading to inaccurate perception and reasoning disconnected from visual evidence. To address these limitations, we propose CURV, a curriculum learning framework that develops intrinsic visual reasoning capabilities by reformulating CQA as multi-step visual grounded reasoning, where each step coordinates logical reasoning with dynamic visual grounding through spatial attention concentration. To assist model learning, we further introduce CCQA, a three-level curriculum dataset with scalable synthetic generation across diverse chart types and reasoning patterns. Our curriculum systematically progresses from basic single-operation reasoning to complex multi-chart compositional tasks. Experiments demonstrate that CURV achieves up to $\uparrow20.50\%$ improvements over baselines and is generalizable to real-world benchmarks (up to $\uparrow12.30\%$) and out-of-domain multimodal reasoning tasks (up to $\uparrow10.20\%$), validating the effectiveness of internalizing visual reasoning with dynamic grounding for enhanced chart understanding capabilities. Code is available at: https://xhguo7.github.io/CURV/.
Alex: Exactly. And because the training is structured that way, the model eventually internalises the skill. It stops needing external prompts to look carefully — it just does it automatically.
Sam: How is this different from something like standard "Chain-of-Thought" prompting? That also breaks problems into steps.
Alex: Good question. Chain-of-Thought breaks a problem into a sequence of text sentences, which is useful. But it doesn't force the model to actually look at the image at each step. The model can write out a perfectly logical-sounding chain of reasoning while barely engaging with the visual data at all.
Sam: So CURV is different because it interleaves those text steps with actual coordinate points on the image?
Alex: Precisely. The researchers call the overall structure a "Meta-Learning Paradigm." Think of it as a four-part sequence for every question: first, the query; then the reasoning step; then the visual location that justifies it; and finally, the answer. The image is treated as a map to be navigated, not just a picture to glance at.
Sam: That actually explains why the gains hold up on charts the model has never seen before. It's not memorising specific charts — it's learning the process of how to look.
Alex: That is the key insight. By training on the mechanism of reasoning rather than just the final answers, the model becomes genuinely more reliable. The paper reports accuracy improvements of over twenty percent in some cases.
Sam: What are the limits of this approach? It sounds like it works well, but presumably it isn't perfect.
Alex: The researchers are candid about this. Because the training data is largely synthetic — meaning it was generated to follow standard chart formats — the model can struggle with unusual or irregular charts it hasn't encountered before. A hand-drawn sketch, or a highly unconventional visualisation, might trip it up.
Sam: So it's still bound by the vocabulary of chart types it was trained on?
Alex: Exactly. The current dataset covers seven common chart types, and it's focused on English-language charts, which limits how broadly it can be applied. These are real constraints the paper acknowledges.
Sam: What does the research point toward next?
Alex: The paper suggests the natural direction is toward what they call "agentic" systems — models that can autonomously use external tools, or collaborate with other AI agents, to tackle problems that go beyond reading a single chart. The goal is a model with a kind of flexible, self-directed gaze it can aim at any visual task.
Sam: So rather than a model that's been coached on specific chart types, you'd have one that can genuinely navigate visual information the way a skilled analyst would.
Alex: That's the long-term vision. And what CURV demonstrates is that the path there runs through the process of reasoning — making sure every logical step is tied to real visual evidence — rather than simply training on more data and hoping for better scores.
Sam: It's a meaningful shift in how we think about what these models actually need to learn.
Alex: It really is. Thanks for listening to ResearchPod.