Semi-Supervised Learning for Molecular Graphs via Ensemble Consensus

In molecular science, labeled data is expensive to acquire, while unlabeled data is abundant. Traditional semi-supervised learning (SSL) methods often rely on data augmentation to enforce consistency, but such augmentations are difficult to design for molecules, where minor structural changes can drastically alter chemical properties. This paper investigates whether an ensemble-based consensus approach can effectively leverage unlabeled molecular data without requiring explicit, label-preserving augmentations.

The authors propose a semi-supervised training framework where an ensemble of graph neural networks is trained simultaneously. Each member learns from a small set of labeled data using a standard supervised loss, while also minimizing a consensus loss on a large set of unlabeled data. The consensus loss penalizes discrepancies between an individual model's prediction and the average prediction of the entire ensemble. This creates a self-reinforcing loop where the collective knowledge of the ensemble guides the improvement of individual members. The authors provide a theoretical justification based on the decomposition of ensemble error, showing that the ensemble consensus is a superior supervisory signal compared to individual predictions.

The study demonstrates that the ensemble consensus objective consistently boosts predictive accuracy across diverse molecular datasets, task types, and graph neural network architectures. A key finding is that individual models trained with this consensus objective often outperform full ensembles trained via traditional supervised learning. Furthermore, the method acts as a regularizer, leading to more robust models and reduced calibration error. The authors observe that this approach effectively eliminates the performance gap between individual members and the full ensemble, suggesting that the consensus training forces models toward more stable and accurate solutions in the loss landscape.