How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-Analyses, and Meta-Syntheses

Annual Review of Psychology

Volumn 70, Issue , 2019, Pages 747-770

  Siddaway, Andy P ^a   Wood, Alex M ^b   Hedges, Larry V ^c  

^a UNIVERSITY OF STIRLING   (United Kingdom)

^b LONDON SCHOOL OF ECONOMICS   (United Kingdom)

^c NORTHWESTERN UNIVERSITY   (United States)

Author keywords

evidence; guide; meta analysis; meta synthesis; narrative; systematic review; theory

Indexed keywords

HUMAN; META ANALYSIS (TOPIC); PRACTICE GUIDELINE; PUBLISHING;

GUIDELINES AS TOPIC; HUMANS; META-ANALYSIS AS TOPIC; PUBLICATION BIAS; SYSTEMATIC REVIEWS AS TOPIC;

EID: 85059501064     PISSN: 00664308     EISSN: 15452085     Source Type: Book Series    
DOI: 10.1146/annurev-psych-010418-102803     Document Type: Review

References (54)

1. APA Publ. Commun. Board Work. Group J. Artic. Rep. Stand. 2008. Reporting standards for research in psychology:Why do we need them?What might they be? Am. Psychol. 63:848-49
2. Baumeister RF. 2013. Writing a literature review. In The Portable Mentor: Expert Guide to a Successful Career in Psychology, ed. MJ Prinstein, MD Patterson, pp. 119-32. New York: Springer. 2nd ed.
3. Baumeister RF, Leary MR. 1995. The need to belong: desire for interpersonal attachments as a fundamental human motivation. Psychol. Bull. 117:497-529
4. Baumeister RF, Leary MR. 1997. Writing narrative literature reviews. Rev. Gen. Psychol. 3:311-20
5. Bem DJ. 1995. Writing a review article for Psychological Bulletin. Psychol. Bull. 118:172-77
6. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. 2009. Introduction to Meta-Analysis. New York: Wiley
7. Borenstein M, Higgins JPT, Hedges LV, Rothstein HR. 2017. Basics of meta-analysis: I2 is not an absolute measure of heterogeneity. Res. Synth. Methods 8:5-18
8. Braver SL, Thoemmes FJ, Rosenthal R. 2014. Continuously cumulating meta-analysis and replicability. Perspect. Psychol. Sci. 9:333-42
9. Bushman BJ. 1994. Vote-counting procedures. In The Handbook of Research Synthesis, ed. H Cooper, LV Hedges, pp. 193-214. New York: Russell Sage Found.
10. Cesario J. 2014. Priming, replication, and the hardest science. Perspect. Psychol. Sci. 9:40-48
11. Chalmers I. 2007. The lethal consequences of failing to make use of all relevant evidence about the effects of medical treatments: the importance of systematic reviews. In Treating Individuals: From Randomised Trials to Personalised Medicine, ed. PM Rothwell, pp. 37-58. London: Lancet
12. Cochrane Collab. 2003. Glossary. Rep., Cochrane Collab., London. http://community.cochrane.org/glossary
13. Cohn LD, Becker BJ. 2003. How meta-analysis increases statistical power. Psychol. Methods 8:243-53
14. Cooper HM. 2003. Editorial. Psychol. Bull. 129:3-9
15. Cooper HM. 2016. Research Synthesis and Meta-Analysis: A Step-by-Step Approach. Thousand Oaks, CA: Sage. 5th ed.
16. CooperHM, Hedges LV, Valentine JC. 2009. TheHandbook of Research Synthesis andMeta-Analysis.New York: Russell Sage Found. 2nd ed.
17. Cumming G. 2014. The new statistics: why and how. Psychol. Sci. 25:7-29
18. Earp BD, Trafimow D. 2015. Replication, falsification, and the crisis of confidence in social psychology. Front. Psychol. 6:621
19. Etz A, Vandekerckhove J. 2016. A Bayesian perspective on the reproducibility project: psychology. PLOS ONE 11:e0149794
20. Ferguson CJ, Brannick MT. 2012. Publication bias in psychological science: prevalence, methods for identifying and controlling, and implications for the use of meta-analyses. Psychol. Methods 17:120-28
21. Fleiss JL, Berlin JA. 2009. Effect sizes for dichotomous data. In The Handbook of Research Synthesis and Meta-Analysis, ed. H Cooper, LV Hedges, JC Valentine, pp. 237-53. New York: Russell Sage Found. 2nd ed.
22. Garside R. 2014. Should we appraise the quality of qualitative research reports for systematic reviews, and if so, how? Innovation 27:67-79
23. Hedges LV, Olkin I. 1980. Vote count methods in research synthesis. Psychol. Bull. 88:359-69
24. Hedges LV, Pigott TD. 2001. The power of statistical tests in meta-analysis. Psychol. Methods 6:203-17
25. Higgins JPT, Green S, eds. 2011. Cochrane Handbook for Systematic Reviews of Interventions, Version 5.1.0. London: Cochrane Collab.
26. John LK, Loewenstein G, Prelec D. 2012. Measuring the prevalence of questionable research practices with incentives for truth telling. Psychol. Sci. 23:524-32
27. Juni P, Witschi A, Bloch R, Egger M. 1999. The hazards of scoring the quality of clinical trials formeta-analysis. JAMA 282:1054-60
28. Klein O, Doyen S, Leys C, Magalhães de Saldanha da Gama PA, Miller S, et al. 2012. Low hopes, high expectations: expectancy effects and the replicability of behavioral experiments. Perspect. Psychol. Sci. 7(6):572-84
29. Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, ChalmersTC. 1992. Cumulativemeta-analysis of therapeutic trials for myocardial infarction. N. Engl. J. Med. 327:248-54
30. Light RJ, Smith PV. 1971. Accumulating evidence: procedures for resolving contradictions among different research studies. Harvard Educ. Rev. 41:429-71
31. Lipsey MW, Wilson D. 2001. Practical Meta-Analysis. London: Sage
32. Matt GE, Cook TD. 1994. Threats to the validity of research synthesis. In The Handbook of Research Synthesis, ed. H Cooper, LV Hedges, pp. 503-20. New York: Russell Sage Found.
33. Maxwell SE, LauMY, Howard GS. 2015. Is psychology suffering from a replication crisis? What does "failure to replicate" really mean? Am. Psychol. 70:487-98
34. Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, et al. 2010. CONSORT explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 340:c869
35. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. 2009. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 339:332-36
36. Morrison A, Polisena J, Husereau D, Moulton K, Clark M, et al. 2012. The effect of English-language restriction on systematic review-based meta-analyses: a systematic review of empirical studies. Int. J. Technol. Assess. Health Care 28:138-44
37. Nelson LD, Simmons J, Simonsohn U. 2018. Psychology's renaissance. Annu. Rev. Psychol. 69:511-34
38. Noblit GW, Hare RD. 1988. Meta-Ethnography: Synthesizing Qualitative Studies. Newbury Park, CA: Sage
39. Olivo SA, Macedo LG, Gadotti IC, Fuentes J, Stanton T, Magee DJ. 2008. Scales to assess the quality of randomized controlled trials: a systematic review. Phys. Ther. 88:156-75
40. Open Sci. Collab. 2015. Estimating the reproducibility of psychological science. Science 349:943
41. Paterson BL, Thorne SE, Canam C, Jillings C. 2001. Meta-Study of Qualitative Health Research: A Practical Guide to Meta-Analysis and Meta-Synthesis. Thousand Oaks, CA: Sage
42. Patil P, Peng RD, Leek JT. 2016. What should researchers expect when they replicate studies? A statistical view of replicability in psychological science. Perspect. Psychol. Sci. 11:539-44
43. Rosenthal R. 1979. The "file drawer problem" and tolerance for null results. Psychol. Bull. 86:638-41
44. Rosnow RL, Rosenthal R. 1989. Statistical procedures and the justification of knowledge in psychological science. Am. Psychol. 44:1276-84
45. Sanderson S, Tatt ID, Higgins JP. 2007. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int. J. Epidemiol. 36:666-76
46. Schreiber R, Crooks D, Stern PN. 1997. Qualitative meta-analysis. In Completing a Qualitative Project: Details and Dialogue, ed. JM Morse, pp. 311-26. Thousand Oaks, CA: Sage
47. Shrout PE, Rodgers JL. 2018. Psychology, science, and knowledge construction: broadening perspectives from the replication crisis. Annu. Rev. Psychol. 69:487-510
48. StroebeW, Strack F. 2014. The alleged crisis and the illusion of exact replication. Perspect. Psychol. Sci. 9:59-71
49. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, et al. 2000. Meta-analysis of observational studies in epidemiology (MOOSE): a proposal for reporting. JAMA 283:2008-12
50. Thorne S, Jensen L, Kearney MH, Noblit G, Sandelowski M. 2004. Qualitative meta-synthesis: reflections on methodological orientation and ideological agenda. Qual. Health Res. 14:1342-65
51. Tong A, Flemming K, McInnes E, Oliver S, Craig J. 2012. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med. Res. Methodol. 12:181-88
52. Trickey D, Siddaway AP, Meiser-Stedman R, Serpell L, Field AP. 2012. A meta-analysis of risk factors for post-traumatic stress disorder in children and adolescents. Clin. Psychol. Rev. 32:122-38
53. Valentine JC, Biglan A, Boruch RF, Castro FG, Collins LM, et al. 2011. Replication in prevention science. Prev. Sci. 12:103-17
54. Valentine JC, Cooper H. 2005. Can we measure the quality of causal research in education? In Experimental Methods for Educational Interventions: Prospects, Pitfalls and Perspectives, ed. GD Phye, DH Robinson, J Levin, pp. 85-112. San Diego, CA: Elsevier