(Im)Paired Programming: Coding Agents Improve Productivity but Harm Understanding

This study investigates the tension between developer productivity and code comprehension when using AI-powered coding agents. The authors conducted a controlled, between-subjects experiment with 54 computer science students tasked with building a web-based game. Participants were divided into two groups: one using an AI agent that directly edits code (e.g., Aider), and another using a chatbot that provides only high-level syntax guidance, forcing users to write the code themselves. The researchers measured performance through initial task accuracy, comprehension tests (recall and reasoning), and a follow-up extension task where participants had to modify their code without AI assistance.

The results demonstrate a clear trade-off: while coding agents enable users to complete initial tasks faster and with higher accuracy, they significantly impair the user's understanding of the underlying code. Participants in the agent group performed worse on comprehension questions and failed to show improved performance in the extension task compared to the chatbot group. Notably, the study found that low-effort interaction patterns—such as copy-pasting prompts and auto-accepting AI-generated edits—were strongly correlated with lower comprehension. Despite these deficits, users still expressed a preference for agents due to their ease of use, suggesting that productivity metrics currently used to evaluate AI tools may be masking a decline in developer skill and oversight capabilities.

As coding agents become standard in development environments, the shift from writing code to merely prompting and reviewing risks creating a generation of developers who can produce functional software without understanding how it works. This lack of comprehension poses significant risks for long-term maintenance, security, and the ability to troubleshoot errors when AI tools are unavailable. The authors argue that the field must move beyond simple task-completion benchmarks and prioritize the development of agents that promote active engagement, readability, and deeper cognitive involvement from the programmer.