Assessment in Team Problem-Solving Exercises in Computing Education

Tabletop exercises (TTXs) are increasingly used in cybersecurity education to simulate crisis responses in a safe, collaborative environment. While these exercises are effective for building practical skills, they present a significant assessment challenge: the open-ended nature of the tasks makes manual evaluation time-consuming and prone to delays. Instructors often struggle to provide timely, meaningful feedback, which is critical for student learning. This study explores how learning analytics and artificial intelligence can automate this process, allowing for faster, more consistent feedback without requiring constant instructor intervention.

To address the gap in automated TTX assessment, the researchers analyzed data from 81 participants across 24 teams in two distinct cybersecurity exercises. The study utilized two primary automated approaches:

• Clustering: An unsupervised machine learning technique used to group teams based on similarities in their task-solving approaches, such as milestone completion and tool usage. This allows instructors to identify patterns in student behavior and provide targeted feedback to specific groups rather than grading each team from scratch.
• Large Language Models (LLMs): The researchers employed LLMs to evaluate the quality of team communication based on standardized rubrics. They compared the performance of different models, specifically noting the evolution in accuracy between GPT-4o and GPT-5.2 when compared against human expert benchmarks.

The study was conducted using the open-source INJECT platform, which logs all team activities, tool interactions, and communications, providing a rich dataset for analysis.

The results indicate that both methods are viable for real-world classroom application. Clustering proved to be a reliable, low-computation method for identifying team performance profiles, which can significantly reduce the time instructors spend on initial assessment. Regarding LLMs, the study highlights that while earlier models often struggled to match instructor judgment, newer iterations like GPT-5.2 show promising accuracy in applying standardized rubrics to complex, open-ended student responses. These tools have been integrated into the INJECT platform to support educators in scaling their teaching practices.