BixBench3: Benchmarking AI agents on research-study-scale computational biology tasks

As AI agents are increasingly deployed to automate scientific research, there is a critical need to evaluate their ability to perform end-to-end computational studies rather than isolated tasks. The authors introduce BixBench3, a benchmark designed to test whether AI agents can process raw biological data into structured scientific results. The benchmark consists of 20 tasks derived from published computational biology papers, covering 17 assay types and 9 scientific domains. For each task, an agent is provided with a research objective, methodological guidance, and raw data. The agent must then execute the necessary analysis pipeline to generate specific data artifacts, which are programmatically graded against the original study's results.

Across 13 frontier large language models, the highest-scoring agent (GPT 5.6 Sol) achieved a score of 0.48, meaning it successfully reproduced less than half of the requested artifacts to a standard that preserved the original biological interpretation. The study identifies three primary factors that limit agent performance: the volume of raw data, the length of the analysis chain, and the complexity of the scientific domain. Agents performed significantly worse on tasks involving more than 100 GB of data compared to those with less than 50 GB. Furthermore, performance dropped as the depth of the analysis increased, with artifacts requiring three or more sequential steps being significantly harder to reproduce than those requiring only one or two. Notably, the study found that higher costs and greater token usage did not correlate with better performance, as several models achieved Pareto-optimal results with intermediate computational expenditure.

BixBench3 represents the most rigorous, long-horizon evaluation of AI agents in computational biology to date. By moving beyond simple question-answering or isolated bioinformatics tasks, this benchmark highlights the specific bottlenecks—such as data management and long-term coherence—that currently prevent AI from autonomously conducting full-scale research studies. These findings provide a roadmap for developing more robust agentic architectures capable of handling the extended, multi-step workflows required in modern scientific discovery.