Empirical Evaluation of Data Poisoning Attacks in Supervised Learning

This study provides an empirical benchmark comparing two primary forms of training-time data poisoning: label flipping and backdoor poisoning. Using MNIST and Fashion-MNIST as testbeds, the authors evaluate how these attacks impact three common classifiers—Logistic Regression, Linear SVM, and Random Forest—across varying poisoning rates. The research aims to clarify how different attack strategies manifest in model performance and to highlight the limitations of relying solely on standard accuracy metrics for security assessment.

The researchers employed a controlled experimental protocol to isolate the effects of each attack. In the label flipping scenario, a percentage of training labels were randomly corrupted to measure the resulting degradation in global model performance. In the backdoor scenario, a specific visual trigger (a 3x3 white square) was added to a subset of training images, which were then relabeled to a target class. The models were evaluated using standard metrics like accuracy and macro-F1, supplemented by an Attack Success Rate (ASR) to measure the effectiveness of the backdoor trigger.

The study reveals a critical distinction between indiscriminate and targeted poisoning. Label flipping acts as an indiscriminate attack, causing visible drops in overall accuracy and macro-F1, particularly in linear models. Conversely, backdoor poisoning is highly covert; it achieves near-perfect attack success rates (0.9667 to 1.0000) while maintaining clean-test performance nearly identical to the baseline. Notably, the study finds that high-performing models, such as the Random Forest, are not inherently more secure, as they proved just as susceptible to embedding malicious trigger-based behavior as the weaker linear models.

These results demonstrate that standard classification metrics are insufficient for detecting security compromises in machine learning. Because backdoor attacks can hide in plain sight while maintaining high clean-test accuracy, practitioners must adopt security-oriented evaluation metrics like ASR to identify potential vulnerabilities. This paper provides a reproducible baseline that emphasizes the need for a more nuanced understanding of model trustworthiness beyond simple predictive performance.