A Leakage-Aware Comparative Benchmark of Machine Learning, Deep Learning, and Transformer Models for Reliable Leukemia Detection

Automated classification of acute lymphoblastic leukemia (ALL) from blood smear images is a critical task for clinical decision support. While many studies report near-perfect performance on the C-NMC 2019 dataset, these results are often achieved using random image-level splitting. This approach allows images from the same patient to appear in both training and testing sets, leading to data leakage where models learn patient-specific artifacts rather than generalized disease features. This study aims to provide a rigorous, leakage-aware benchmark to clarify the true performance of various machine learning and deep learning architectures.

The author establishes a strict subject-disjoint protocol, ensuring that no image from a specific patient appears in more than one partition. The study compares five models across three families: classical machine learning (LightGBM and RBF-SVM) using frozen features from EfficientNet-B0, fine-tuned convolutional neural networks (EfficientNet-B0 and B1), and a Vision Transformer (ViT-Tiny). The models were developed using three subject-disjoint folds from 73 subjects and evaluated on a genuinely external preliminary-phase test set of 1,867 images from 28 unseen subjects. Beyond standard discrimination metrics, the study evaluates model calibration using expected calibration error (ECE) and Brier scores, and quantifies leakage-induced inflation by comparing random versus subject-disjoint splitting.

Under the honest, subject-disjoint evaluation, EfficientNet-B1 emerged as the top performer with an AUROC of 0.913, a sensitivity of 0.87, and a specificity of 0.80. The study confirms that random splitting inflates AUROC by approximately 0.04, even in conservative frozen-feature models. Furthermore, the analysis reveals that ViT-Tiny and frozen-feature classifiers tend to over-predict the malignant class, resulting in high sensitivity but poor specificity. The results highlight the necessity of reporting calibration metrics alongside discrimination scores to ensure that models are trustworthy for clinical use.