ResearchPod Summary
Systematic reviews often rely on nonrandomized observational studies for environmental and occupational health assessments. A common criticism of current GRADE (Grading of Recommendations, Assessment, Development, and Evaluations) application is that it may overly penalize this evidence for inherent limitations like potential selection bias or residual confounding. This paper argues that merely identifying a risk of bias at the individual study level is insufficient; reviewers must critically assess the magnitude and direction of that bias across the entire body of evidence to determine its actual impact.
The authors propose that rather than simply aggregating individual study scores, reviewers should employ sensitivity analyses to test whether specific biases could truly explain observed associations. When multiple studies with different designs, populations, and potential bias patterns consistently show similar results, it provides evidence that those identified biases are not the primary drivers of the observed effects. This approach helps shift the focus from a rigid, design-based penalty system to a more holistic understanding of internal validity.
The paper introduces several techniques for probing bias across a set of studies:
By incorporating these analyses throughout the systematic review process—from protocol development to the final certainty rating—reviewers can justify more nuanced, evidence-based conclusions. The authors demonstrate this through case studies on the associations between trichloroethylene and kidney cancer, as well as PFAS and liver enzymes, showing how these techniques can support a one-level downgrade rather than an automatic two-level reduction.
AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.