Haotian Lin, Silin Chen, Xiaodong Gu, Yuling Shi, Chengxi Pan, Jiaqi Ge, Mengfan Li, Jianghong Huang, MENGCHIEH CHUANG, Beijun Shen, Haibing Guan
6 min
Abstract
LLM-based coding agents have significantly advanced automated software issue resolution, yet they remain highly prone to factual errors caused by insufficient repository understanding. Recent methods attempt to mitigate this limitation through pre-repair repository exploration; however, their fix-driven strategies explore repositories without identifying the agent's knowledge gaps, often yielding imprecise context that fails to bridge the underlying understanding deficit. In this paper, we propose ACQUIRE, a QA-driven framework for software issue resolution. Mirroring how experienced developers first comprehend unfamiliar code before attempting a fix, ACQUIRE explicitly acquires repository knowledge prior to repair. The framework decouples knowledge acquisition from patch generation through two stages: in the first stage, a Questioner and an Answerer collaborate to acquire structured repository knowledge, where the Questioner poses targeted questions and the Answerer produces evidence-grounded answers through autonomous exploration; in the second stage, the Resolver leverages the resulting QA knowledge to generate informed patches. By transforming implicit knowledge gaps into explicit, factually reliable understanding, ACQUIRE accelerates knowledge-intensive repair stages and enables more accurate resolution. Experiments on SWE-bench Verified demonstrate that ACQUIRE consistently outperforms representative pre-repair methods, raising Pass@1 by up to 4.4 percentage points with modest additional cost and time.
Sam: The study suggests it is. The time spent asking questions is offset by far fewer failed repair attempts. The agent stops wandering through the codebase trying ineffective patches and instead starts from a position of factual understanding.
Alex: So the bottleneck wasn't the AI's intelligence. It was the absence of a diagnostic phase.
Sam: That's the core insight. And there's another design choice worth understanding. When the Answerer goes looking for facts, it does so in parallel—the system spins up independent AI instances, each focused on answering one specific question. And crucially, they don't share notes with each other.
Alex: Why does that matter? Wouldn't sharing information be helpful?
Sam: You'd think so, but if one instance goes down a wrong path early on, sharing notes spreads that error to the others. Keeping them independent means each answer is based purely on evidence, not on what another instance already assumed. It's a bit like having multiple doctors give independent opinions before comparing results—you get a cleaner read on what's actually true.
Alex: That makes sense. Were there other design decisions that affected performance?
Sam: One significant one: how many question-and-answer pairs to generate. The researchers tested different amounts and found that performance peaked at two pairs. Adding a third actually caused a slight dip in success rates.
Alex: Why would more information make the agent worse?
Sam: It's a focus problem. When you give the agent too much context, its attention gets spread thin. Imagine trying to solve a puzzle while someone keeps handing you extra pieces that don't belong—it becomes harder to see the pattern you're actually looking for. Two targeted questions provide enough grounding to make a precise repair. More than that starts to clutter the reasoning.
Alex: So there's an optimal amount of knowledge, and more isn't always better.
Sam: Right. It's about signal versus noise. To make this concrete—imagine a bug where a documentation tool misaligns text because it splits a line at the wrong character. A standard agent might try editing several different files, fail repeatedly, and never find the root cause. The Questioner in ACQUIRE recognises that what it actually needs to understand is the parsing logic—the rules the system uses to read and break up text. It asks a specific question about that. The Answerer then goes directly to the relevant file and finds the exact line of code responsible for the incorrect split. Because the agent now has a factual, evidence-based answer, it can write a precise fix that addresses the root cause in a single attempt.
Alex: One targeted question versus wandering through the whole codebase. How do the researchers know the questions being generated are actually good ones? You can imagine an agent asking questions that sound structured but aren't really useful.
Sam: That's a fair concern, and they addressed it directly. They used what's called an "LLM-as-a-judge" protocol—a separate, high-performance AI model that evaluates the questions on dimensions like diagnostic usefulness, depth of reasoning, and how well they cover the problem space. Essentially, one AI grades the quality of another AI's questions.
Alex: And the category-guided approach scored better than just letting the agent ask whatever it wanted?
Sam: Consistently. Forcing the agent to categorise its ignorance—to look specifically for gaps in logic, design rules, location, and coding standards—produced much better coverage than open-ended questioning. It prevents the agent from fixating on one angle and missing the broader picture.
Alex: Are there limits to what the framework can do?
Sam: The main one the paper flags is that the knowledge is currently gathered at the start and then fixed in place. A more capable version would let the agent refresh its understanding dynamically as it works—updating its picture of the codebase in real time as new information emerges. That would be more powerful, but also more computationally expensive.
Alex: So it's a meaningful step toward reliability, even if there's room to grow.
Sam: That's the right framing. It's a disciplined, diagnostic approach that makes the agent's behaviour more predictable. The underlying principle—understand the system before you try to fix it—is straightforward, but enforcing it systematically turns out to make a real difference.
Alex: Thanks for walking through this, Sam. And thanks to everyone listening to ResearchPod.