A Comparative Study of Pretrained Transformer Models for Quranic ASR: Speech Representations, Label Formats, and Dataset Composition

This paper investigates the application of self-supervised, pretrained Transformer models to the domain of Quranic Automatic Speech Recognition (ASR). While Quranic recitation is central to Islamic education, existing ASR systems often struggle with the unique phonetic requirements of Classical Arabic, the diversity of reciter proficiency, and the lack of comprehensive, high-quality training data. The authors conduct a systematic empirical study to determine how different speech feature extractors, output label formats, and training strategies influence transcription accuracy.

The researchers utilized a large-scale dataset exceeding 870 hours of audio, combining professional recitations from EveryAyah.com with user-generated recordings from the Tarteel.io platform. They evaluated three prominent self-supervised architectures—Wav2Vec2.0, HuBERT, and XLS-R—against a Citrinet baseline. The methodology involved freezing the CNN feature encoders of these models and fine-tuning the Transformer decoders using Connectionist Temporal Classification (CTC) loss. A key component of the study was an ablation analysis that tested four different output label formats: standard Arabic text, Arabic with diacritics (Tashkeel), English transliteration, and Buckwalter transliteration.

The study found that the Wav2Vec2-XLSR-53 model provided the most robust speech representations for Quranic recitation. Notably, the researchers discovered that removing diacritics from the training text improved model performance, likely due to the reduction of label complexity and ambiguity. The proposed configuration achieved a WER of 0.08 on professional data and 0.11 on mixed data, outperforming the Citrinet baseline (WER 0.163) while simultaneously accelerating training efficiency from 140 hours down to 40 hours. These results suggest that domain-specific fine-tuning of large, pretrained speech models is a highly effective strategy for specialized, low-resource, or linguistically complex speech tasks.