Unraveling Machine Behavior by Multi-Level Bias Analysis and Detection: Methodology and Application to Computer Vision

This study moves beyond traditional input-output bias evaluation, which treats neural networks as black boxes. Instead, the authors introduce a multi-level taxonomy to identify where and how bias is encoded within the network's internal architecture. By analyzing the model at three distinct levels—latent space, layer activations, and model parameters—the researchers aim to uncover the structural origins of demographic bias.

The authors propose three specific detection techniques:

• SpaceBias (Level 2): Analyzes the penultimate latent space using neighbor-probability distributions. It quantifies bias by applying the two-sample Kolmogorov-Smirnov test to compare the clustering consistency of different demographic groups.
• ActivationBias (Level 3): Examines the activation levels of convolutional filters. It uses the Mann-Whitney U test to detect disparities, based on the observation that underrepresented groups often trigger lower activation levels in final convolutional layers.
• WeightBias (Level 4): Employs a secondary neural network to identify biased patterns directly within the convolutional filter parameters, independent of any input data.

Through experiments on the DiveFace dataset and a colored-MNIST benchmark, the authors evaluated over 127,000 models. The results demonstrate that internal disparity and detection performance correlate strongly with the balance of the training distribution. As the training data becomes more balanced, the internal bias metrics decrease smoothly. This work provides a robust framework for auditing AI models, offering deeper diagnostic capabilities for developers and regulators to understand how societal biases are internalized during the learning process.