AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both

Systematic reviews have become essential for clinical and policy decisions, but they vary significantly in quality and are increasingly incorporating non-randomised studies alongside traditional randomised controlled trials (RCTs). The original AMSTAR instrument, while widely used, was primarily designed for RCTs and needed updating to handle the complexities of observational data and modern systematic review methodologies. AMSTAR 2 addresses these gaps, providing a framework to assess the conduct and reliability of reviews across diverse evidence types.

The updated tool consists of 16 items that guide users through a structured evaluation of a review's conduct. Key revisions include:

• Better handling of non-randomised studies: Explicit criteria for assessing risk of bias in non-randomised designs, drawing on the ROBINS-I tool.
• Alignment with PICO: Improved research question alignment using the Population, Intervention, Control, and Outcome framework.
• Revised Response Format: The removal of 'not applicable' and 'cannot answer' options to force evaluators to confront methodological gaps, with the addition of a 'partial Yes' option.
• Critical Domains: The identification of seven 'critical' domains (such as protocol registration, literature search, and risk of bias assessment) that, if flawed, should severely diminish an appraiser's confidence in the review's results.

AMSTAR 2 is not meant to be a scoring system. Instead, it provides a structured approach to qualitative assessment. Reviewers are encouraged to use the results to categorize their confidence in a review as high, moderate, low, or critically low based on the presence of critical flaws. While the tool requires more time to complete than the original—reflecting the increased rigor needed to evaluate modern systematic reviews—it serves as a robust mechanism for distinguishing high-quality evidence from potentially biased summaries.