Analyzing Image Encoder Choices and Graph Homophily in GCN Frameworks for Breast Ultrasound Classification

This study investigates how the choice of image encoder—ranging from lightweight convolutional neural networks (CNNs) to advanced self-supervised vision transformers—impacts the performance of Graph Convolutional Networks (GCNs) in classifying breast ultrasound images. Specifically, the authors examine whether better image representations lead to higher graph homophily, which in turn facilitates more effective classification.

The researchers evaluated five distinct image encoders: a custom lightweight CNN, ResNet-18, and three transformer-based models (MAE ViT, DINO ViT-S/8, and I-JEPA ViT). Using a combined dataset of 6,619 benign and malignant breast ultrasound scans, they constructed cosine-similarity k-nearest-neighbor graphs where each node represents a patient scan. These graphs were then processed by a single-layer GCN with a linear classification head. The study systematically compared the encoders based on classification metrics (accuracy, AUC, sensitivity, specificity, and F1-score) and measured the degree of graph homophily to determine if it serves as a reliable indicator of model performance.

The study demonstrates a clear hierarchy in performance: higher-capacity encoders consistently outperformed simpler architectures across all metrics. DINO ViT-S/8 emerged as the top-performing encoder, followed by I-JEPA ViT. Crucially, the authors identified a strong linear correlation (R² ≈ 0.853) between test-set graph homophily and classification accuracy. This suggests that the primary mechanism by which stronger encoders improve GCN performance is by producing more class-consistent graph neighborhoods, which allows the GCN to aggregate more relevant diagnostic information during message passing.