FairBED: A Bayesian Experimental Design Approach to Gathering Fairer Data

Traditional machine learning fairness interventions often focus on post-hoc model adjustments or reweighting existing, biased datasets. This paper asks whether we can instead intervene at the source by designing the data acquisition process itself to be inherently fairer, without sacrificing the utility of the data for predictive tasks.

The authors introduce FairBED, a framework that treats data collection as a Bayesian experimental design (BED) problem. The core innovation is to define fairness as the lack of information leakage about sensitive attributes (e.g., race or gender). The authors propose two primary objectives:

• Unconditional FairBED (Ru): Maximizes information gain about the target variable while minimizing marginal information gain about the sensitive attribute.
• Conditional FairBED (Rc): A stricter objective that minimizes information gain about the sensitive attribute given the target variable.

By optimizing these objectives, the data acquisition process is guided to select samples that are informative for the task at hand but uninformative about protected characteristics. The authors formally prove that, under certain conditions, this approach leads to models that satisfy demographic parity.

Empirical results demonstrate that FairBED consistently achieves a superior trade-off between predictive accuracy and fairness compared to random sampling or standard BED. Because the fairness is embedded in the dataset itself, the benefits generalize across different downstream models. Furthermore, the authors show that FairBED can be effectively combined with existing fairness-aware training techniques, enhancing their performance compared to when those techniques are applied to naively collected data.

This work shifts the focus of algorithmic fairness from "fixing" biased models to "preventing" biased data. By providing a principled, information-theoretic way to guide data collection, FairBED offers a proactive strategy for practitioners in high-stakes domains like healthcare and criminal justice to mitigate societal biases before they are encoded into machine learning systems.