Transfer Learning Architectures for Scalable Multi-Fidelity Bayesian Optimization

In self-driving laboratories, multi-fidelity Bayesian optimization (MFBO) is essential for balancing cheap, approximate data with expensive, high-fidelity experiments. While Gaussian processes (GPs) are the standard surrogate model, they often struggle with the high-dimensional, non-smooth, and heterogeneous search spaces typical of molecular and materials science. This study investigates whether transfer-learning (TL) architectures—which learn representations from abundant low-fidelity data and adapt them to sparse high-fidelity observations—can serve as more effective engines for closed-loop optimization than traditional GP-based methods.

The authors conducted a comprehensive benchmark comparing eleven transfer-learning surrogates against four established GP-based methods (including standard MFGP, NARGP, deep-kernel GP, and sparse variational MFGP). To ensure a fair comparison, all fifteen surrogates were tested under an identical acquisition policy, fidelity budget, and model size across nine diverse tasks, ranging from synthetic functions to real-world chemistry and materials problems. The study specifically isolated the surrogate's role by keeping the acquisition policy fixed, allowing the researchers to determine whether performance gains were driven by the surrogate's predictive mean, uncertainty calibration, or representation-learning capabilities.

The results reveal a clear performance divide. GPs excel on smooth, low-dimensional synthetic benchmarks (like the Branin function) where their kernel-based priors are well-suited to the objective. However, on molecular and materials discovery tasks—such as covalent organic frameworks (COFs) and solvation free energy—transfer-learning surrogates significantly outperform GPs. This advantage is not merely due to better uncertainty estimation; rather, it stems from the surrogate's ability to learn a more accurate predictive mean and maintain a better representation of the near-optimal region in complex descriptor spaces. Furthermore, the study finds that greedy exploitation of the transfer-learned mean is a more robust strategy than relying on uncertainty-driven exploration, which often fails to provide reliable benefits in these high-dimensional regimes.