AUTOSYNTHESIS: An agentic system for automated meta-analysis

AutoSynthesis is a multi-agent framework designed to address the time-intensive nature of systematic reviews and meta-analyses. By utilizing a series of specialized LLM agents, the system automates the entire research pipeline: formulating search strategies, retrieving literature, screening papers for eligibility, extracting quantitative data, calculating standardized effect sizes (e.g., Hedges' g), and performing random-effects meta-analysis. The system also includes modules for heterogeneity analysis, risk-of-bias assessment, and the generation of PRISMA-compliant reports, providing a transparent audit trail of every decision made during the process.

The authors evaluated AutoSynthesis by comparing its output against established human-conducted meta-analyses, specifically focusing on the persuasive power of large language models. The system demonstrated strong performance, retrieving and synthesizing relevant evidence with high accuracy. In benchmarking tests, the pooled effect estimates produced by AutoSynthesis were within a ±0.12 Hedges' g margin of expert-conducted reviews. Furthermore, the system successfully replicated key diagnostic findings, such as substantial between-study heterogeneity and evidence of small-study effects, indicating that the automated workflow is capable of producing reliable, high-quality scientific summaries.

Manual meta-analysis is a bottleneck in scientific progress, often requiring hundreds of hours and significant expertise to complete. AutoSynthesis offers a scalable, reproducible, and near real-time alternative that lowers the barrier to evidence-based decision-making. By enabling "living" meta-analyses that can continuously integrate new evidence, this framework has the potential to significantly accelerate the synthesis of knowledge in fields ranging from clinical medicine and education to public policy, ensuring that practitioners have access to the most current and rigorous evidence base.