Does a Modern-Handwriting Warm-Up Help Historical Arabic OCR? A Reproducible, Compute-Matched Evaluation on Muharaf and KHATT

This study investigates whether an intermediate training stage using modern Arabic handwriting (KHATT) improves the performance of Handwritten Text Recognition (HTR) models when fine-tuned on historical Arabic manuscripts (Muharaf). The authors argue that existing claims in the field are often based on single, under-controlled experiments. To test the stability of this transfer-learning strategy, the researchers conducted four separate, non-replicated implementations of the same nominal ablation, varying common training parameters like learning rates, epoch budgets, and base checkpoints. They then performed a compute-matched follow-up experiment to isolate the effect of the warm-up domain while holding all other training variables constant.

The initial four pilot runs yielded highly unstable results, with Character Error Rate (CER) effects ranging from -17.64 to +14.52 points and even reversing sign. The authors demonstrate that this variance is largely driven by specific engineering choices—such as asymmetric learning rate reductions—rather than the inherent value of the warm-up data. In the controlled, compute-matched experiment, the modern handwriting warm-up resulted in a small negative effect (+2.42 CER points worse than the control). When compared against a disjoint slice of real target data, the domain-specific penalty was only about 0.6 points, suggesting that the warm-up provides little to no benefit for historical HTR in this configuration.

This paper highlights a critical methodological issue in machine learning research: the tendency to rely on confidence intervals from a single implementation to make broad claims. The authors show that implementation-level variance can far exceed line-level uncertainty, rendering single-run results potentially misleading. By releasing the SaudiHeritage-OCR package—including a unified normalizer, scorer, and verified KHATT decoder—the authors provide the infrastructure necessary for the community to conduct more robust, reproducible evaluations of HTR strategies on historical Arabic material.