Agentic Knowledge Tracing: A Multi-Agent LLM Architecture for Stealth Assessment of Financial Literacy in Serious Games

This paper introduces the Agentic BKT pipeline, a multi-agent architecture designed to perform stealth assessment of financial literacy within serious games. Traditional assessment methods often rely on intrusive post-game quizzes, which can disrupt the learning experience. To solve this, the authors developed a 2D platformer game aligned with the OECD/INFE financial literacy framework. The pipeline processes player actions through four distinct phases: event logging, LLM-based event classification, domain-specific agent reasoning (covering risk, investing, spending, and credit), and an expert judge agent that synthesizes these inputs into a final mastery score.

A critical innovation of this work is the use of domain-specific agents. Instead of treating all player actions as a monolithic sequence, the system routes events to specialized agents that analyze behavioral trajectories. For example, the Risk Mitigation Agent evaluates how a player reacts to financial losses over time, rather than just labeling individual gambling actions as 'good' or 'bad.' This allows the system to capture temporal patterns of learning and decision-making that simpler models miss.

The authors evaluated the pipeline with 193 K-12 students across 264 game sessions. The results demonstrate that the Agentic BKT mastery estimates are significantly correlated with learning gains (r = 0.276) and post-test scores (r = 0.333), while showing no correlation with pre-test scores, confirming both convergent and discriminant validity. Notably, the multi-agent approach tripled the predictive validity compared to a single-LLM baseline (r = 0.095), highlighting that domain decomposition and session-level reasoning are essential for accurately capturing the multidimensional nature of financial literacy.

This research provides a scalable, non-intrusive way to assess complex competencies in real-time. By moving away from binary 'right/wrong' event tracking toward agentic, context-aware reasoning, the framework offers a blueprint for embedding sophisticated assessment into any serious game. It demonstrates that LLMs can effectively act as both classifiers and reasoning engines to bridge the gap between open-ended gameplay and formal educational standards.