Improving statistical reporting in psychology

Transparent and comprehensive statistical reporting is essential for the credibility and cumulative progress of psychological science. This paper addresses the gap between theoretical best practices and the practical challenges researchers face when documenting their work. By providing a structured approach to reporting, the authors aim to help researchers move beyond ambiguous or insufficient descriptions, ensuring that their methodology and results are fully reproducible and interpretable.

The authors emphasize two critical stages where transparency is paramount: the planning stage and the data analysis stage.

• Planning Stage: Researchers are encouraged to move beyond vague hypotheses toward formalization, clearly defining constructs and operationalizations. The paper advocates for preregistration of hypotheses and analysis plans to reduce researcher degrees of freedom and mitigate hindsight bias.
• Data Analysis Stage: The paper provides detailed guidance on handling outliers, missing data, and statistical model specification. It emphasizes the importance of disclosing software versions and providing analysis code, which allows readers to verify the robustness of the reported findings.

Because psychology utilizes diverse statistical paradigms, the authors provide tailored guidance for:

• Frequentist Statistics: Emphasizing the need for exact p-values, effect sizes with confidence intervals, and the avoidance of common errors like post hoc power analysis.
• Bayesian Statistics: Highlighting the necessity of reporting prior distributions, Bayes factors, and convergence diagnostics to ensure the reliability of posterior estimates.
• Sequential Designs: Providing instructions for documenting stopping rules, decision boundaries, and potential biases in effect size estimates when using group sequential or sequential probability ratio tests.