WIDER-FAIR: An Annotated Version of the WIDER-FACE Dataset for Fairness Evaluation

As machine learning models are increasingly deployed in high-stakes environments, ensuring fairness across demographic groups has become a critical requirement. While face detection is a foundational task for many applications, existing benchmarks often lack the demographic annotations necessary to evaluate and mitigate bias. This paper addresses this gap by introducing WIDER-FAIR, a dataset designed to facilitate fairness evaluation in face detection.

The authors curated WIDER-FAIR by manually annotating a subset of 16,256 images from the widely used WIDER-FACE benchmark. Each face was annotated for perceived ethnicity (Asian, Black, Indian, or White) and sex (male or female). To ensure data quality, the authors implemented a rigorous pipeline that excluded images with extreme occlusion, blur, or excessively small bounding boxes. They validated the consistency of these annotations using face embeddings, K-Nearest Neighbors classification, and t-SNE visualization. Finally, they trained a YOLOv5 model to perform ablation studies, systematically removing specific demographic groups from the training set to observe the impact on detection performance and fairness metrics.

The experiments reveal that face detection models exhibit clear performance disparities across demographic groups. Specifically, the detection recall for Black individuals is notably lower than for other groups. The ablation studies demonstrate that the presence of faces of Black individuals in the training set is particularly crucial for model generalization; excluding this group from training leads to a more significant increase in fairness disparity compared to excluding any other ethnic group. These results underscore the necessity of diverse, demographically annotated datasets for identifying and addressing systemic biases in computer vision models.