Webly Supervised Multi-Label Recognition: Evaluation Benchmark and Dual-Branch Multi-Label Contrastive Learning

Multi-label image recognition (MLR) is essential for understanding real-world scenes, but training high-performance models typically requires expensive, manually annotated datasets. While webly supervised learning (using freely available web images and search keywords) has been successful for single-label tasks, it remains underexplored for multi-label recognition. This paper addresses the lack of unified benchmarks and the challenge of label noise in web-collected multi-label data.

The authors first construct a new WS-MLR benchmark, consisting of two large-scale datasets, Web-COCO and Web-Pascal, which contain approximately 300,000 images retrieved using category-word combinations. They also re-implement several representative baselines under a unified evaluation protocol. To tackle the inherent label noise—which includes both false-positive and false-negative labels—they propose the Dual-Branch Multi-Label Contrastive Learning (DBMLCL) framework. This framework uses two independent network branches to learn category-specific instance-level and category-level feature representations. By computing similarities between these representations, the model can estimate the presence of categories and iteratively correct noisy labels during training.

The study demonstrates that the DBMLCL framework consistently outperforms representative baselines on the newly constructed Web-COCO and Web-Pascal datasets. By explicitly modeling category-specific features and using contrastive learning to refine label predictions, the model effectively mitigates the negative impact of noisy web labels. The authors provide the datasets, code, and trained models to facilitate future research in this area.

This work provides a critical foundation for moving away from labor-intensive, manually annotated datasets in multi-label recognition. By establishing a standard benchmark and a robust framework for handling noisy web data, it enables more scalable and cost-effective development of computer vision models capable of recognizing multiple objects in complex, real-world images.