ADABORD: a novel AdaBoost approach for ordinal classification

Ordinal Classification (OC) involves predicting target variables with a natural order, where the cost of misclassification typically increases with the distance between the predicted and actual classes. Traditional classification methods often treat these labels as nominal, ignoring the ordinal structure and failing to optimize for ordinal-aware performance. This paper introduces ADABORD, an ensemble learning framework designed to explicitly incorporate ordinal information into the AdaBoost algorithm.

The authors modify two fundamental components of the standard AdaBoost (SAMME) algorithm to make it ordinal-aware:

1. Base Estimator: Instead of standard decision trees, ADABORD uses decision trees with an Ordinal Gini (OGini) splitting criterion. This criterion accounts for the cumulative distribution of classes at each split, ensuring the tree structure respects the ordinal nature of the data.
2. Error Function: The authors replace the standard misclassification-based error with the absolute Ranked Probability Score (aRPS). This metric measures the discrepancy between predicted and observed cumulative distributions, allowing the model to penalize errors based on ordinal distance without requiring arbitrary numerical assignments to categories.

By combining these components, ADABORD remains a score-free method, meaning it does not impose unjustified metric assumptions on the ordinal labels.

ADABORD was evaluated against seven state-of-the-art methods using the TOC-UCO repository, which contains 46 diverse ordinal datasets. The results demonstrate that ADABORD significantly outperforms competing methods, particularly in datasets with five or more classes. The authors attribute this success to the synergy between the OGini-based trees and the aRPS error function, which together better capture the ordinal structure and handle the inherent class imbalance often found at the extremes of ordinal scales.