Evaluating the Impact of Explainable AI on Trust in AI-Assisted Code Review

Large language models (LLMs) are increasingly used to automate code reviews, yet their "black box" nature makes it difficult for developers to evaluate the validity of AI-generated feedback. This study investigates how Explainable AI (XAI) influences developer trust and decision-making during code review. The researchers conducted a within-subjects user study with 34 participants, who evaluated code change requests under three conditions: Condition A (full explanations plus feedback), Condition B (feedback only), and Condition C (no explanations).

The study found that the level of explanation significantly alters how developers perceive and interact with AI tools. Interestingly, there is a disconnect between perceived trust and actual agreement. Participants reported the highest levels of trust when provided with full explanations (Condition A). However, they were most likely to agree with the AI's recommendations when provided with only moderate explanations (Condition B). The researchers hypothesize that providing too much information (full explanations) may prompt developers to scrutinize the AI's reasoning more critically, leading them to disagree more often than they would with less detailed feedback.

Despite the differences in trust and agreement, the level of explanation did not significantly impact the time developers took to complete their reviews. Across all conditions, participants consistently cited code readability and correctness as the primary factors influencing their decisions to accept or reject AI-generated suggestions. The findings suggest that while XAI is effective at boosting user confidence, designers of AI-assisted development tools must carefully balance the depth of explanations to ensure they support, rather than hinder, efficient decision-making.