Riazi-8B: An Urdu Large Language Model for Mathematical Reasoning

Research Context and Approach

Large language models (LLMs) have demonstrated significant progress in mathematical reasoning, yet these advancements are heavily concentrated in English-centric resources. Low-resource languages like Urdu suffer from a lack of reasoning-oriented datasets and models capable of multi-step inference. Existing Urdu models often prioritize linguistic fluency over logical consistency, frequently failing to maintain correct reasoning traces during complex problem-solving.

To address this, the authors introduce Riazi-8B, an Urdu-focused LLM built upon the Qwen3-8B architecture. The development process follows a two-step adaptation strategy:

1. Continued Pre-Training (CPT): The base model is further trained on Urdu Wikipedia to enhance its linguistic grounding.
2. Supervised Fine-Tuning (SFT): The model is trained on an Urdu-translated version of the GSM8K dataset, which provides step-by-step mathematical reasoning traces.

Both stages utilize LoRA (Low-Rank Adaptation) to ensure parameter-efficient training, allowing the model to acquire reasoning capabilities without requiring a full-scale retraining of the base model.

Evaluation and Findings

The authors evaluate Riazi-8B on the MGSM-Urdu benchmark, comparing it against existing Urdu-adapted models (Alif-8B, Qalb-8B) and a strong multilingual baseline (Llama-8B). The evaluation framework employs three primary metrics: Exact Match (EM) for answer accuracy, Urdu Output Purity (UOP) for language fidelity, and Step Completeness Score (SCS) for reasoning structure.

Riazi-8B consistently outperforms all baselines across these metrics. Beyond quantitative accuracy, the authors employed an 'LLM-as-a-Judge' framework using a rubric-based assessment to score responses on correctness, reasoning, fluency, clarity, and completeness. Riazi-8B achieved the highest scores across all qualitative dimensions, demonstrating that combining language-specific adaptation with reasoning-focused supervision is an effective strategy for extending LLM capabilities to low-resource languages.