Sfiso Cebolenkosi Mahlaba
8 min
This study explores how 30 third-year preservice teachers (PSTs) in South Africa evaluate the accuracy of ChatGPT-generated solutions to curriculum-based mathematical word problems. Given the rise of generative AI in education, the research investigates whether future teachers can effectively navigate AI-provided feedback or if they succumb to uncritical trust. Grounded in metacognition theory, the paper examines how PSTs use metacognitive knowledge (MK)—the awareness of their own thinking processes—and metacognitive experiences (ME)—real-time feelings of confidence, doubt, or cognitive load—to validate AI outputs.
The researcher employed an exploratory, naturalistic case study design. Students were tasked with solving two contextualized mathematical word problems using ChatGPT-3.5 and subsequently writing an assignment that justified their acceptance or rejection of the AI's results. A key feature of the study was the lack of standardization: participants were encouraged to use their own self-directed prompts. While this limits direct comparisons across student cases, it offers high ecological validity by simulating how individuals naturally interact with AI outside of highly controlled laboratory settings.
The study highlights that evaluative success depends on the effective application of metacognitive knowledge. While some PSTs effectively utilized declarative and conditional knowledge—such as citing formal mathematical theorems or recognizing contextual flaws like 'double-counting' poles in a fence problem—others demonstrated significant gaps in metacognitive regulation. In many cases, students failed to perform deep verification and instead exhibited 'blind trust' or relied on confirmation bias, especially when the task’s cognitive load was high or the AI-generated procedure appeared superficially familiar. The results suggest that procedural fluency (following steps) is not a substitute for conceptual oversight; students who struggled to solve the problems independently were far more likely to accept incorrect AI solutions without scrutiny.
As AI becomes more integrated into classrooms, teachers must act as critical mediators rather than passive consumers of AI feedback. This paper emphasizes that fostering 'epistemic vigilance'—the capability to interrogate the validity of AI-generated reasoning—is a vital professional competency. The findings suggest that teacher education programs must prioritize explicit metacognitive training to ensure that future teachers can identify AI limitations, avoid over-reliance, and teach their own students to engage with AI in a reflective, mathematically rigorous manner.
Alex: Those sound fairly easy. Is a study built on two simple problems telling us much?
Sam: The author concedes they're simple and not fully real-world. The argument is that they're common in South African classrooms, they're tricky because of language, and they're the kind of thing AI tends to stumble on. And the goal wasn't whether students could solve them. It was how they evaluated someone else's solution.
Alex: So how do you analyse "how they evaluated"? That's slippery.
Sam: The lens is metacognition, thinking about your own thinking, from the psychologist John Flavell. It splits into two parts. Metacognitive knowledge is what you know about tasks and strategies. Metacognitive experiences are the feelings in the moment, like confidence, doubt, or a sense of how hard something is.
Alex: And the knowledge part has sub-types?
Sam: Three. Declarative is knowing facts and rules. Procedural is knowing the steps. Conditional is knowing when and why a method applies. The author coded every justifying statement in the thirty assignments into these categories. And, importantly for the design, each one was marked as effective or ineffective. So a student who claims to check the steps but still gets it wrong is coded differently from one who checks properly.
Alex: So they're separating "said the right kind of thing" from "actually did it well." What came out?
Sam: On the fence problem, most students rejected forty-eight. Twenty of those gave a corrected solution, usually with a diagram showing the shared corners. On the farmer problem, it flipped. Most students accepted ChatGPT's square as correct, even though it was wrong. Only a handful got it right. One cited the isoperimetric theorem, which says that for a fixed perimeter a circle encloses the most area. The others compared the areas of different shapes.
Alex: Why the flip? Same students, same tool.
Sam: The paper's reading is that the second problem leaned on procedure. Some students saw the derivative step, recognised it from calculus, and accepted the answer on familiarity. One student followed the optimisation correctly but never questioned the assumption that the field had to be square. The maths was coherent. The premise wasn't checked.
Alex: And the opening line, catching one error and missing the next?
Sam: That's one student whose first ChatGPT answer to the fence problem was twelve poles. They rejected it, re-prompted, got forty-eight, and accepted that. The author's point is that spotting one error doesn't mean the scrutiny is sustained. There were other failures too. Two students rejected the square answer, then worked it out themselves and landed on exactly the same answer.
Alex: That sounds like where trust comes in. You said they trusted the AI more when they couldn't solve it.
Sam: Yes, though on thin data. Only three students wrote anything about mental effort. One said outright they trusted ChatGPT because the problem was too demanding for them to solve. Another accepted an answer because it looked simple. The author links that to the fluency effect, where ease of processing gets mistaken for correctness.
Alex: The headline was that students relied on knowledge more than in-the-moment experiences. Couldn't that just be the assignment? Written justifications after the fact will naturally look like knowledge.
Sam: The author says almost exactly that. Conditional knowledge was by far the most frequently coded category. But the paper attributes that dominance partly to task design, not to some stable trait. Written assignments capture justification, not real-time monitoring. That's why the recommendations include think-aloud protocols and interviews.
Alex: So how much should I trust the whole thing?
Sam: As an exploratory snapshot, reasonably. As a measurement, no. The biggest limitation is that prompts were neither standardised nor collected. So students may have been judging different ChatGPT outputs. The author is explicit that you can't compare one student's evaluation to another's. On top of that, it's one cohort at one urban university, and the author was also the module lecturer. The paper reports reflexive notes and peer debriefing to manage that, but it's still an insider view.
Alex: Leaving prompts uncontrolled seems like a choice, not an accident.
Sam: It was deliberate. The author argues it buys ecological validity, meaning you see how students actually use the tool when left alone. The cost is comparability. The paper itself says the abstract and conclusions should be read as patterns of engagement, not settled evaluative behaviour.
Alex: What does it suggest teacher educators actually do?
Sam: Build tasks where students evaluate AI solutions to contextual word problems and must justify their verdicts. Train not just knowledge, but the regulation of it, meaning checking whether your own reasoning holds. And provide scaffolds for independent verification, so heavy problems don't default to trusting the machine. The author also calls for studies comparing trained and untrained prompting.
Alex: So who should sit down with the full paper, and where should they start?
Sam: Mathematics teacher educators, and anyone designing assignments that use AI. Start with the findings on conditional knowledge, especially the ineffective cases. They're concrete and very usable for teaching. Then read the limitations section, which is unusually candid about what the design can't support.
Alex: And for everyone else, the line to carry around?
Sam: Catching the AI's first mistake proves nothing about the second. And the moment you can't check the answer is exactly when you start believing it.
Alex: A good reason to keep pencil and paper close.