The Correct Answer Trap: Pedagogically-Grounded Detection and Feedback for Hidden Misconceptions

Automated tutoring systems often rely on answer correctness to provide feedback. However, this approach fails when students arrive at the correct answer using flawed logic—a phenomenon the authors term the correct answer trap (CAT). Because these students provide the correct final output, standard systems reinforce their misconceptions rather than correcting them. This paper investigates how to reliably detect these hidden errors using student explanations.

Using 20,964 responses from the Eedi mathematics platform, the authors evaluated various models. Fine-tuned classifiers (like BERT) performed poorly, detecting only 57% of these misconceptions because they rely on answer-based shortcuts. While reasoning-capable LLMs (such as Gemma 4) improved detection to 84%, they also generated a high volume of false alarms—roughly 8 false positives for every genuine misconception detected. This makes standalone automated flagging impractical for direct teacher intervention.

To address this, the authors propose a detect-verify-escalate pipeline based on a graduated assessment rubric. Instead of a binary correct/incorrect classification, the system categorizes responses into four states: clear reasoning, needs clarification, misconception detected, or wrong answer. This structure allows the system to distinguish between genuine errors and ambiguous explanations. Uncertain cases are routed to diagnostic follow-up questions rather than immediately alerting a teacher, which helps filter the review queue and provides formative feedback to the student.

This research highlights that diagnostic accuracy is a prerequisite for effective automated feedback. By separating method validity from answer correctness, the proposed pipeline reduces the burden on human instructors while ensuring that students who "get the right answer for the wrong reason" receive the pedagogical support they need. It shifts the focus of AI tutoring from merely grading outputs to interpreting the underlying reasoning process.