Light-weight Pronunciation Assessment via Discrete Speech Token Surprisal

How can we perform automated pronunciation assessment in low-resource settings where labeled learner data, phoneme inventories, and forced alignment tools are unavailable or too costly to collect?

The authors introduce a framework that relies exclusively on native speech resources. First, they discretize speech using a frozen self-supervised learning (SSL) encoder and a K-means codebook trained on native data. They then train a token-level language model (TLM) to measure the "surprisal" of learner speech; higher surprisal indicates a deviation from native phonotactic patterns. When reference text is available, the system uses a Text-to-Discrete-Unit (Text2DUnit) module to generate a canonical native token sequence, which is then compared to the learner's acoustic tokens using Dynamic Time Warping (DTW) in the shared discrete space. These surprisal and alignment features are combined via a simple regression model to produce a final pronunciation score.

The proposed method achieves competitive performance on the SpeechOcean762 dataset, with transcript-guided features significantly improving correlation scores compared to audio-only approaches. The system demonstrates strong generalization capabilities, maintaining consistent performance when transferred to the L2-ARCTIC dataset without retraining. Notably, the framework remains stable even when the native training data is reduced by an order of magnitude (from 960 hours to 100 hours), making it highly suitable for low-resource environments.

By eliminating the need for expensive, labeled non-native corpora and complex forced-alignment pipelines, this approach lowers the barrier to entry for developing computer-assisted language learning (CALL) tools. It provides a scalable, efficient solution for assessing pronunciation in under-resourced languages or specialized speaking styles where traditional supervised models fail due to data scarcity.