Zilu Zhang, Jing Bao, Lifeng Qiao, Sheng Xu, Qiantai Feng, Siqi Sun, Yang Yang
8 min
Abstract
Protein-complex structure prediction with AlphaFold 3 depends on MSA content, depth, and cross-chain pairing. Current pipelines apply uniform recipes, typically maximizing depth and pairing by species, yet even whether cross-chain pairing helps AF3 remains contested. We show that the value of both depth and pairing is strongly target-dependent, so no single recipe is optimal across interfaces. We introduce an autonomous agent that treats a frozen AF3 as a black-box experimental instrument. For each target, the agent formulates biological hypotheses, analyzes and reorganizes the MSA, and iteratively refines sequence selection and pairing using predictor label-free signals alone. On 1{,}015 protein--protein interfaces from three benchmarks released after AF3's training cutoff, the agent improves overall interface DockQ over AF3's default pipeline and fixed MSA recipes on every benchmark, while using on average 60\% fewer MSA sequences than the default. Gains are largest for inter-species heteromers, where species-based pairing is unreliable: mean DockQ increases from 0.465 to 0.612 on FoldBench and from 0.482 to 0.562 on HD439. The improvements arise from both sequence construction and pairing policy, including deliberate non-pairing. These results recast MSA design as a target-specific problem and show that an autonomous agent can adapt it at test time through biological reasoning and self-directed, confidence-guided experimentation.
Alex: Walk me through a run. What stops that freedom from becoming an uncontrolled search?
Sam: Every target starts with the default AlphaFold input as a mandatory control. The agent examines the sequences, proposes a different alignment, and submits it to the unchanged predictor. It then receives confidence feedback and can inspect the predicted structure.
Alex: So it can ask whether a change helped, but not check against the answer key.
Sam: The reference structure stays hidden from the agent. Network access is disabled, and the runtime keeps true accuracy out of its visible files. After each round, the agent records what it learned, then decides what to test next.
Alex: Give me an example where that biological diagnosis changes the input in a way a uniform recipe misses.
Sam: One complex contains parts of the related natural proteins p63 and p73, plus a DARPin—a designed protein built from repeating units. The designed protein has no natural orthologs: corresponding proteins inherited across species. Yet sequence searches retrieve many members of the broader natural repeat family.
Alex: So a large alignment could mostly represent that broad family, rather than useful evidence about this designed binding partner?
Sam: In the default input, the overwhelming majority of rows contained only DARPin-related sequences. The agent separated candidates for the related natural proteins, paired selected candidates by species, and restricted the designed protein’s sequences to a small supplement. The resulting prediction’s interface accuracy rose sharply.
Alex: That example combines filtering and pairing. Did the agent establish which changes mattered, or just find a successful combination?
Sam: It ran its own follow-up comparisons. Expanding the DARPin supplement or removing selected pairings substantially degraded performance. The agent made those decisions using predictor confidence, not the hidden accuracy score. It’s a concrete example of a hypothesis becoming a testable input procedure.
Alex: A striking case can still be exceptional. What happens across the benchmarks?
Sam: The evaluation covers 1,015 protein–protein interfaces across three benchmarks released after AlphaFold’s training cutoff. Overall average accuracy improved on every benchmark, compared with both the default pipeline and the fixed recipes tested.
Alex: What does accuracy mean here, and how large was a typical benchmark-level change?
Sam: They use DockQ, a score of agreement between a predicted protein interface and its reference structure. On the heterodimer benchmark, the mean increased from about 0.58 with default inputs to about 0.62 with agent-selected inputs. Higher means closer agreement.
Alex: Does that comparison isolate the agent, or could it simply have started with better sequence searches?
Sam: Against the default pipeline, it starts with identical sequence resources. That is the cleanest comparison. The other fixed recipes are also input-matched on the heterodimer benchmark, but use separate sequence searches on the FoldBench datasets. Those latter comparisons mix input-source differences with construction differences.
Alex: And repeated searching gives it extra chances. Are the final scores benefiting from more sampled structures?
Sam: Each evaluated proposal produces the same twenty-five structures. The additional runs search for better inputs; they are not pooled into a larger final sample. The authors also reran selected inputs with fresh random seeds, and the gains persisted.
Alex: There is still a cost to finding those inputs. Does a smaller final alignment mean a cheaper workflow overall?
Sam: Not established here. The selected alignments used about sixty percent fewer sequences than the default. But finding them required roughly six predictor runs per target on average. Shallower final inputs should not be confused with demonstrated end-to-end computational savings.
Alex: What do the results say about pairing itself? Is the practical advice simply to stop doing it?
Sam: No. The largest gains were for complexes joining proteins from different species, where species matching is unreliable. But for distinct proteins within one species, the agent’s pairing policy added value beyond sequence selection.
Alex: How did they separate those contributions?
Sam: They shuffled row order independently for each chain, preserving its sequences but disrupting the agent’s organization across chains. That hurt accuracy for within-species complexes. The test measures the whole policy, including leaving sequences deliberately unpaired—not just the value of selected pairs.
Alex: The weakest link still sounds like confidence. If the predictor is confidently wrong, the agent could choose the wrong candidate.
Sam: That is the central limitation, and the authors measure it. On the first FoldBench set, final selection lost forty-one percent of the improvement available in the best candidates tested. The agent sometimes found a better input but failed to nominate it.
Alex: So searching well and selecting well are separate problems. Does better feedback help both?
Sam: In an oracle experiment, the agent could see true structural accuracy. It then found better candidates as well as selecting better ones. That is not a deployable setting for unknown structures, but it identifies feedback quality as a bottleneck.
Alex: Who should read the full document, and where should they start?
Sam: Researchers predicting protein complexes should start with Main Results and the controlled comparisons in the experimental setup. Then read Exploration Gain and Nomination Loss: it explains why confidence-guided improvement still leaves performance behind. For researchers building scientific agents, that separation between available feedback and hidden accuracy is the essential design issue.
Alex: And for someone outside those areas, what is the line to carry away?
Sam: For these protein predictions, choosing the right evidence beat simply supplying more evidence. The agent’s contribution was adapting that choice to the target—not replacing the predictor or solving its uncertainty.
Alex: Thanks for listening.