Ariful Islam Farhad, MD. Nayem Islam
9 min
As Large Language Models (LLMs) continue to advance, their application remains heavily skewed toward high-resource languages like English. This research addresses the critical gap in AI-driven mathematics education for the Bangla-speaking community, where students and educators lack access to domain-specific, language-aligned tools for solving mathematical problems.
The authors compiled a comprehensive, high-quality dataset of 57,168 Bangla mathematical problems, including 17,568 novel problems extracted from Bangladeshi textbooks, board exams, and Math Olympiad archives. The dataset covers multiple formats, including multiple-choice, short-answer, and complex Olympiad-style problems. The researchers fine-tuned two base models, Qwen-3.0-4B and LLaMA-3.2-3B, using a multi-stage pipeline that combined Parameter-Efficient Fine-Tuning (PEFT) with Goal-Reinforced Preference Optimization (GRPO). This approach allowed the models to learn structured, step-by-step reasoning in Bangla while maintaining computational efficiency.
The fine-tuned models, specifically the Qwen-based variant, demonstrated substantial improvements in both mathematical accuracy and linguistic quality. In Olympiad-level tasks, the fine-tuned model achieved an 85% accuracy rate, significantly outperforming the base model. Qualitative analysis confirmed that the fine-tuned models successfully transitioned from generating incoherent or English-only responses to providing fluent, pedagogically aligned, and logically structured solutions in Bangla. The research validates that targeted dataset curation and domain-specific alignment can effectively bridge the reasoning gap for low-resource languages.
This work provides a foundational resource for inclusive AI in education. By enabling AI to communicate and reason in Bangla, the authors have created a tool that can be integrated into digital learning platforms and tutoring systems to support millions of students in Bangladesh and beyond. It sets a new standard for developing domain-specific, language-focused AI technologies for underrepresented languages.
Sam: It’s both, despite stronger wording about authenticity early in the thesis. The collection contains about fifty-seven thousand problems overall. About seventeen and a half thousand are described as novel problems drawn directly from Bangla books, guides, and related educational sources.
Alex: That distinction changes what a researcher would use it for. Where does the rest come from?
Sam: Most of the collection is synthetic Bangla content adapted from existing English mathematics datasets. Generative models helped produce those adaptations. The original portion includes national curriculum textbooks, exam-oriented material, and Bangladesh Mathematical Olympiad problems, spanning ordinary classroom work and advanced reasoning.
Alex: The useful unit isn’t just a question, though. How did they establish answers and explanations?
Sam: They extracted text from scanned documents, corrected recognition errors, and standardized mathematical notation. Some answers came from source books or existing dataset answers. Other explanations were generated by models and then reviewed by mathematics educators, with further cleaning and manual checks.
Alex: Human review is valuable, but it doesn’t automatically establish an error rate. How strong is their quality claim?
Sam: The authors estimate almost perfect question-answer accuracy, based on their verification process. The excerpt does not provide an independent audit or a sampling calculation supporting that estimate. I would treat the review workflow as evidence of care, not the near-perfect estimate as a measured guarantee.
Alex: Give us a concrete problem that shows what the training material teaches beyond a final answer.
Sam: One Olympiad example asks about a two-digit number added to the number formed by reversing its digits. Their sum must be a hundred and fifty-four. The supplied solution separates the tens and units digits, derives a constraint on their sum, and checks which digit pairs are allowed.
Alex: So the explanation has to respect the digit ranges, not just manipulate an expression and stop.
Sam: It ends by identifying five possible original numbers. That example illustrates the intended training target: interpreting the wording, setting up the relationship, checking constraints, and explaining the conclusion in Bangla. Some Olympiad entries also include Program-of-Thought, meaning executable Python code representing the solution.
Alex: How did they turn those different kinds of examples into a model? I’d expect textbook questions and Olympiad proofs to demand different things.
Sam: Their pipeline separates question types into training stages rather than treating everything as one undifferentiated collection. It moves through multiple-choice questions, short answers, and Olympiad problems, including reasoning and code-based examples. Synthetic material adds further variety; a dedicated creative-question stage was deferred because of resource limits.
Alex: And there’s a second distinction here: what gets trained, versus what gets rewarded.
Sam: They use supervised fine-tuning: training existing models on questions paired with desired responses. Then they describe a reward-guided optimization stage that favors logical correctness, completeness, and clarity. The base models are Qwen and LLaMA, selected for their combination of reasoning, Bangla generation, and computational practicality.
Alex: Would reproducing this require training every parameter on the entire collection?
Sam: No. They use low-rank adaptation, or LoRA, which trains small additions while keeping most model parameters frozen. Typical runs sampled roughly five to six thousand examples, rather than using the whole collection. Different variants used different dataset combinations, so this isn’t a single uniform training run.
Alex: Let’s get to the main comparison. What changed relative to the same models before adaptation?
Sam: On Olympiad-style problems, the Qwen-based model’s reported accuracy rose from seventy percent to eighty-five percent. The adapted LLaMA model reached sixty percent, also improving substantially over its own starting model. Both adapted models improved on multiple-choice and short-answer tasks too.
Alex: That supports task-specific adaptation, but not necessarily every ingredient in the pipeline. Can we tell whether rewards, code examples, or the original Bangla material drove the gains?
Sam: Not from the comparisons supplied in this excerpt. They compare base models with adapted versions, without isolating those contributions. The thesis also reports better overlap with reference explanations, using language-output metrics, but overlap alone does not establish mathematical validity.
Alex: My bigger question is generalization. How many test problems were there, and were they clearly separated from training?
Sam: The excerpt doesn’t give evaluation-set sizes or enough detail about train-test separation. It describes evaluating unseen problem types, but the reported accuracy comparisons don’t let us assess that claim closely. Without those details, we cannot judge how stable the scores are or rule out overlap with training material.
Alex: And classroom usefulness is another step beyond benchmark accuracy. Did they measure learning outcomes?
Sam: Not in the provided excerpt. Tutoring platforms, mobile applications, and rural classrooms are proposed uses, rather than evaluated deployments. Likewise, open release of datasets, weights, and tools is a stated commitment, but the excerpt doesn’t establish a completed release.
Alex: Given the promise and those gaps, who should read the thesis in full, and where should they start?
Sam: Researchers in low-resource language technology, mathematical reasoning, or educational AI should read it. Start with Data Collection and Annotation to understand provenance and verification. Then read Training Configuration and Accuracy Evaluation together, checking dataset mixtures and what the base-versus-adapted comparison actually establishes.
Alex: And for someone who won’t read further, what’s the one conclusion to carry away?
Sam: Multilingual support is not the same as native-language mathematical teaching. This thesis suggests that targeted educational data can narrow that gap, while leaving generalization and classroom benefit to be established.
Alex: Keep that distinction in mind when evaluating the next language-specific tutor.