Zhongwei Yu, Yan Song, Xue Yan, Anjie Liu, Xingyu Lu, Yihang Chen, Huichi Zhou, Siyuan Guo, Luoyang Sun, Sihan Chen, Xiangning Yu, Jun Wang
5 min
Abstract
Scientific discovery often involves optimising expensive-to-evaluate objectives over vast, structured, and open-ended hypothesis spaces, such as molecules, protein sequences, and computer programs. Generative models such as large language models (LLMs) provide expressive priors over such spaces, but their likelihoods and self-assessments are unreliable proxies for the objectives and calibrated epistemic uncertainty, especially for novel candidates outside the observed data distribution. We introduce the Large Discovery Model (LDM), an empirically grounded recurrent architecture that couples a generative model with a Bayesian non-parametric reward surrogate model. The generative model proposes and refines candidate designs, while the surrogate predicts their performance and quantifies uncertainty, yielding an uncertainty-aware value that guides candidate generation, refinement, and selection. The discovery memory and the surrogate model are continually updated as each new experimental observation arrives. We evaluate LDM on three scenarios spanning different design modalities and objectives, including neural-network training, antibody design, and molecular optimisation. Compared to LLM-only reflection or traditional statistical search across these domains, LDM achieves a $2.4\times$ greater reduction in validation BPB, an $18.2\%$ relative decrease in binding energy, and more than $60\%$ relative gains in molecular multi-objective performance. These results suggests that LDM could serve as a general-purpose discovery engine for effective search over open-ended hypothesis spaces.
Alex: Right, because that would be wasteful. You'd keep testing the same thing. Instead, the sampling rule spreads attention across genuinely different high-value options.
Sam: And does the system explain its choices, or just output a number?
Alex: Both, actually. It can attach a short written rationale to each selected experiment — a plain-language explanation of why this idea is worth running. That's useful because it makes the system's reasoning transparent, not just a black box score.
Sam: That's a meaningful detail. So what are the limits of this approach?
Alex: The main bottleneck is the statistical model itself. As you run more experiments and collect more data, the computation required to update the model grows very steeply — roughly as the cube of the number of past observations. So if you've run a thousand experiments, the math gets significantly heavier than if you'd run a hundred.
Sam: Meaning it slows down the more successful it is?
Alex: In a sense, yes. That scaling issue puts a practical ceiling on how many experiments the system can handle before performance degrades. The paper is candid about this.
Sam: Are there other constraints?
Alex: The system is also only as good as the language model's existing knowledge. If the model starts with weak domain understanding — say, it hasn't been trained on much chemistry — then its proposals won't be very useful, and the statistical layer can't fully compensate for that. Garbage in, garbage out, essentially.
Sam: So what does the paper suggest for fixing these problems?
Alex: Two main directions. First, adding memory retrieval — letting the system pull from external databases and published literature on the fly, rather than relying only on what it learned during training. Second, using mathematical approximations that are less computationally expensive, so the system can scale to larger datasets without grinding to a halt.
Sam: It sounds like the framework is solid in principle, but the engineering work is still ongoing.
Alex: That's a fair summary. What the paper establishes is a clear way of thinking about scientific discovery as a decision problem — not just a prediction task. The question isn't only "what's the best idea?" but "what's the best idea to test next, given everything we know and don't know?" That reframing is what makes this approach distinct.
Sam: And pairing generative flexibility with rigorous uncertainty tracking is what makes that possible.
Alex: Precisely. It's a meaningful step toward systems that don't just generate plausible science, but can actually navigate the process of finding it. Thanks for listening to ResearchPod.