Tao Li, Kaiyuan Hou, Tuan Vinh, Monika Raj, Zhichun Guo, Carl Yang
8 min
Abstract
Lead optimization in drug discovery requires improving therapeutic properties while ensuring that proposed molecular modifications correspond to feasible synthetic routes. Existing approaches either prioritize property scores without enforcing synthesizability, or rely on expensive enumeration over large reaction networks, while direct application of Large Language Models (LLMs) frequently produces chemically invalid structures. We introduce MolReAct, a framework that formulates lead optimization as a Markov Decision Process over a synthesis-constrained action space defined by validated reaction templates. A tool-augmented LLM agent serves as a dynamic reaction environment that invokes specialized chemical analysis tools to identify reactive sites and propose chemically grounded transformations from matched templates. A policy model trained via Group Relative Policy Optimization (GRPO) selects among these constrained actions to maximize long-term oracle reward across multi-step reaction trajectories. A SMILES-based caching mechanism further reduces end-to-end optimization time by approximately 43%. Across 13 property optimization tasks from the Therapeutic Data Commons and one structure-based docking task, MolReAct achieves an average Top-10 score of 0.563, outperforming the strongest synthesizable baseline by 10.4% in relative improvement, and attains the best sample efficiency on 10 of 14 tasks. Ablations confirm that both tool-augmented reaction proposals and trajectory-level policy optimization contribute complementary gains. By grounding every step in validated reaction templates, MolReAct produces molecules that are property-improved and each accompanied by an explicit synthetic pathway.
Alex: Right, and since every choice is locked to those proven recipes, the entire path from start to finish is a blueprint chemists can actually follow in the lab?
Sam: Yes—every generated path traces valid reactions, linking computer suggestions directly to lab work. The paper tests this on 13 tasks optimizing drug properties from a standard benchmark called the Therapeutic Data Commons, plus one for docking molecules to targets. It tops the average score for the best ten suggestions, beating the next synthesizable method by 10.4%, and uses fewer tries on most tasks.
Alex: That's a clear step forward in making the ideas practical. It closes that synthesis gap without the huge compute drain of checking everything possible. So with paths always buildable, how do they make the whole training process efficient enough for multiple steps? Doesn't calling that language model every single time slow things down?
Sam: They add a simple lookup system to avoid repeating work. When the system encounters the same molecule again—identified by its standard text code—they store and reuse the list of safe reactions and the exact next molecules those produce. This cuts down on language model calls during training, speeding things up without changing the logic. The learning part, which chooses among those safe options at each step, uses a specialized language model fine-tuned for decisions. It reads the current molecule's text code along with plain descriptions of each possible reaction—like the recipe name, ingredients, and resulting structure—then scores them to pick the best one.
Alex: Right—like saving game levels you've already explored so you don't replay them. And since each choice leads to exactly one next molecule—no chance involved—does that simplify connecting the final score back through the steps?
Sam: Yes, exactly. Applying a specific reaction recipe and ingredient to a molecule always yields the same next structure, like following a fixed baking instruction. This predictable chain lets the end-of-path rating guide improvements reliably across the whole sequence.
Alex: Huh. So no randomness means cleaner learning from those group comparisons.
Sam: The paper confirms this setup delivers the top average performance on those 14 drug optimization tasks among methods that guarantee buildable results—about a 10% edge over the next best. It also needs fewer attempts to reach strong outcomes.
Alex: That's a solid tie between the mechanism and the results. Grounding everything in real chemistry pays off without excess computation. You've mentioned the results hold up across those drug tasks. But in the lab, chemists need actual ingredients—do the suggested changes use stuff that's easy to buy or make?
Sam: The paper checks this directly on four protein-target tasks. They scan the proposed ingredients against a huge catalog from a supplier called Enamine, which lists over two million ready-to-buy chemicals. About two-thirds to three-quarters of the unique ingredients match exactly, meaning chemists can order them off the shelf. For the rest, they calculate a simple score for how easy they are to build from basics—averaging around 2.5, which points to straightforward assembly.
Alex: So most tweaks rely on common stock, not exotic rarities. And filtering to only buyable ones barely hurts performance?
Sam: Yes—on three of four tasks, the top scores stay the same or improve slightly when limiting to catalog matches, as the filter drops weaker ideas. One task dips more because it favors specialized parts with less coverage, but overall, the system naturally picks practical options.
Alex: Huh, that grounds it even more. What about teasing apart why the pieces work together—like, how much comes from the smart reaction suggestions versus the learning that picks sequences?
Sam: They run targeted tests on those four protein tasks, isolating parts step by step. First, they check single-step suggestions alone: for a batch of test molecules, the tool-guided picks produce higher-scoring products on average than random recipe choices. Then, swapping to a fixed random selector for sequences drops scores notably, showing the trained decision-maker adds clear value over the proposals alone.
Alex: Okay, so the tools make good starting options, but the learning sharpens the path. And the pathways illustrate real binding improvements—like for targets such as sEH, adding an aromatic amine via C-N coupling extends the molecule into a deeper pocket.
Sam: Similar gains hold across DRD2, GSK3β, and JNK3—suggesting chemically sensible tweaks that fit protein shapes. The paper flags bounds—like sticking to 115 templates, so no new reactions, and skipping conditions or protecting groups. It yields valid sequences but not full lab recipes.
Alex: Fair limits for now. Still, routinely spitting out patentable candidates with traced routes could trim years from discovery—bridging that valley without lab dead ends.
Sam: The paper suggests exactly that potential. By constraining to synthesizable steps, it advances reliable early optimization—a meaningful link between computation and bench work.
Alex: That's a grounded step forward. Thanks for breaking it down, Sam. Thanks for listening to ResearchPod.