Study-Strategy Clusters from EdNet Logs Track Engagement, Not Mastery

Learning analytics platforms frequently rely on unsupervised clusters of intelligent tutoring system interaction logs to identify distinct learner types, assuming that these study styles can serve as proxies for knowledge acquisition and mastery. This paper investigates whether unsupervised study-strategy profiles derived from EdNet-KT3 logs actually forecast later learning outcomes, or if they simply track engagement patterns such as persistence and completion. The researchers analyze a seeded sample of active learners, segmenting their interaction timelines to ensure that cluster features rely exclusively on early practice behaviors while learning outcomes are evaluated strictly on later sessions.

To prevent data leakage between behavioral profiling and outcome evaluation, the authors split each learner's timeline in half by response count. Using locked features covering action frequencies, video dwell share, and strategy rates, they apply principal component analysis followed by k-means clustering. The initial parent cut yields five clusters, consisting of four contrast poles alongside a large, near-mean residual majority. Reclustering this residual group reveals a bootstrap-stable hierarchy of eight distinct study strategies, tested via adjusted Rand index resampling and feature-definition ablations.

When clusters are fit using only early practice data, they reliably predict later engagement metrics, including continuing practice and completing late sessions. However, these early behavioral profiles show no significant predictive relationship with later unassisted accuracy, which measures first-attempt correctness without system help. Furthermore, a supervised self-attentive knowledge tracing model demonstrates that mastery signals remain nearly independent of behavioral style labels. Consequently, log-based clusters effectively describe how students study and persist, but they should not be treated as standalone proxies for knowledge gains.