Ruoyu Wang, Jierun Chen, Shaowei Wang, Chaofan Tao, Sidi Yang, Yuxin Jiang, Kim-Hui Yap, Lifeng Shang, Xiaohui Li, Haoli Bai
3 min
Abstract
Coding agents increasingly generate pull requests (PRs) for real-world software issues, yet one-shot PR generation remains open-loop: the PR is proposed without systematic review, diagnosis, or revision. We introduce \textbf{SWE-Review}, a framework for closing this loop with agentic code review. Given an issue and an AI-generated PR, a reviewer agent explores the repository, decides whether the PR should be accepted, and provides structured feedback for revision. We evaluate this setting with our proposed \textbf{SWE-Review-Bench} to measure both review correctness and downstream revision usefulness. We further curate \textbf{SWE-Review-Traj} dataset to study broader applications of agentic review and fill the data-scarcity gap for open reviewer training. Experiments show that agentic review continuously improves PRs through a generate-review-revise loop, outperforms single-turn fixed-context review in both decision accuracy and resolve rate after revision, transfers beyond review to improve issue-resolution models, and enables effective and efficient test-time scaling. These results position agentic code review as a practical mechanism for moving AI coding agents from one-shot PR generation toward closed-loop issue resolution.
Sam: That's a more meaningful measure than just asking whether the final code passes. It's testing the quality of the whole process.
Alex: Precisely. And that connects to another concept in the paper — test-time scaling. Normally, when people want an AI system to perform better, they train a bigger model, which is expensive and slow. Test-time scaling is a different approach: instead of making the model larger, you give it more time and computing resources to think through the problem while it's actually working. The review-and-revise loop is a practical example of that idea.
Sam: So the improvement comes from the process, not just the raw power of the model. Spend more effort checking your work, get better results.
Alex: That's the paper's argument, yes. And it's a meaningful distinction, because it suggests you don't necessarily need a more powerful AI to get more reliable code — you need a more careful workflow.
Sam: It's a more grounded approach than I expected. Less about building a smarter AI and more about building a more disciplined one.
Alex: That's a fair way to put it. The move from one-shot generation to a closed-loop investigation — generate, review, revise — is what the researchers see as the path toward AI coding agents you can actually depend on. Thanks for listening to ResearchPod.