Scientific discovery often involves optimising expensive-to-evaluate objectives over vast, structured, and open-ended hypothesis spaces, such as molecules, protein sequences, and computer programs. Generative models such as large language models (LLMs) provide expressive priors over such spaces, but their likelihoods and self-assessments are unreliable proxies for the objectives and calibrated epistemic uncertainty, especially for novel candidates outside the observed data distribution. We introduce the Large Discovery Model (LDM), an empirically grounded recurrent architecture that couples a generative model with a Bayesian non-parametric reward surrogate model. The generative model proposes and refines candidate designs, while the surrogate predicts their performance and quantifies uncertainty, yielding an uncertainty-aware value that guides candidate generation, refinement, and selection. The discovery memory and the surrogate model are continually updated as each new experimental observation arrives. We evaluate LDM on three scenarios spanning different design modalities and objectives, including neural-network training, antibody design, and molecular optimisation. Compared to LLM-only reflection or traditional statistical search across these domains, LDM achieves a $2.4\times$ greater reduction in validation BPB, an $18.2\%$ relative decrease in binding energy, and more than $60\%$ relative gains in molecular multi-objective performance. These results suggests that LDM could serve as a general-purpose discovery engine for effective search over open-ended hypothesis spaces.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at a study on scientific discovery — specifically, how AI systems that are very good at writing and generating ideas often struggle to actually do science.
Sam: So the paper is asking: why can't these text-generating models just... invent new things on their own?
Alex: Exactly. The core problem is that real experiments are expensive. You can't just run thousands of them to find out what works. So the model needs to be smart about which experiments are worth trying.
Sam: And right now, these models aren't good at that?
Alex: Not reliably. They can generate plausible-sounding ideas all day, but they don't have a built-in way to judge which ideas are actually valuable to test. They're missing a sense of scientific priority.
Sam: So how do the researchers fix that?
Alex: They pair the language model with a second system — a mathematical model that tracks uncertainty. Think of it like an experienced gold prospector who keeps a map of every hole they've already dug. Before digging a new one, they look at the map and ask: where do I still have the least information? Where's the most promising ground I haven't touched yet?
Sam: So instead of guessing randomly, the system learns what it doesn't know.
Alex: Right. And that distinction matters. There's a difference between randomness — like a coin flip — and genuine ignorance, where you simply haven't collected data yet. The paper calls the second kind "epistemic uncertainty," which just means: the gaps in what the model actually knows. When the system spots a high-uncertainty zone, it flags that as a priority for testing.
Sam: So it's not just picking ideas it thinks are good — it's also picking ideas that would teach it the most?
Alex: Exactly. That balance — between exploiting what already looks promising and exploring unknown territory — is the heart of the system. The researchers call their full approach the Large Discovery Model. It combines a language model that proposes ideas with a statistical model that scores each proposal based on predicted value and uncertainty.
Sam: How does the scoring actually work in practice?
Alex: The two signals — predicted reward and uncertainty — get combined into a single score for each candidate idea. The system then uses that score to decide which proposals are worth sending to a real experiment. Crucially, it also avoids picking the same idea twice by sampling candidates proportionally to their scores, without repeating.
Sam: So it's not just picking the top-ranked idea over and over?
Alex: Right, because that would be wasteful. You'd keep testing the same thing. Instead, the sampling rule spreads attention across genuinely different high-value options.
Sam: And does the system explain its choices, or just output a number?
Alex: Both, actually. It can attach a short written rationale to each selected experiment — a plain-language explanation of why this idea is worth running. That's useful because it makes the system's reasoning transparent, not just a black box score.
Sam: That's a meaningful detail. So what are the limits of this approach?
Alex: The main bottleneck is the statistical model itself. As you run more experiments and collect more data, the computation required to update the model grows very steeply — roughly as the cube of the number of past observations. So if you've run a thousand experiments, the math gets significantly heavier than if you'd run a hundred.
Sam: Meaning it slows down the more successful it is?
Alex: In a sense, yes. That scaling issue puts a practical ceiling on how many experiments the system can handle before performance degrades. The paper is candid about this.
Sam: Are there other constraints?
Alex: The system is also only as good as the language model's existing knowledge. If the model starts with weak domain understanding — say, it hasn't been trained on much chemistry — then its proposals won't be very useful, and the statistical layer can't fully compensate for that. Garbage in, garbage out, essentially.
Sam: So what does the paper suggest for fixing these problems?
Alex: Two main directions. First, adding memory retrieval — letting the system pull from external databases and published literature on the fly, rather than relying only on what it learned during training. Second, using mathematical approximations that are less computationally expensive, so the system can scale to larger datasets without grinding to a halt.
Sam: It sounds like the framework is solid in principle, but the engineering work is still ongoing.
Alex: That's a fair summary. What the paper establishes is a clear way of thinking about scientific discovery as a decision problem — not just a prediction task. The question isn't only "what's the best idea?" but "what's the best idea to test next, given everything we know and don't know?" That reframing is what makes this approach distinct.
Sam: And pairing generative flexibility with rigorous uncertainty tracking is what makes that possible.
Alex: Precisely. It's a meaningful step toward systems that don't just generate plausible science, but can actually navigate the process of finding it. Thanks for listening to ResearchPod.