Flaws in the application and interpretation of statistical analyses in systematic reviews of therapeutic interventions were common: a cross-sectional analysis

Journal of Clinical Epidemiology

Volumn 95, Issue , 2018, Pages 7-18

  Page, Matthew J ^a   Altman, Douglas G ^b   McKenzie, Joanne E ^a   Shamseer, Larissa ^c,d   Ahmadzai, Nadera ^c   Wolfe, Dianna ^c   Yazdi, Fatemeh ^c   Catalá López, Ferrán ^c,e   Tricco, Andrea C ^f,g   Moher, David ^c,d  

^a MONASH UNIVERSITY   (Australia)

^b UNIVERSITY OF OXFORD   (United Kingdom)

^c OTTAWA HOSPITAL RESEARCH INSTITUTE   (Canada)

^d UNIVERSITY OF OTTAWA   (Canada)

^e UNIVERSITY OF VALENCIA   (Spain)

^f ST MICHAEL'S HOSPITAL   (Canada)

^g UNIVERSITY OF TORONTO   (Canada)

Author keywords

Meta analysis; Meta research; Methodology; Quality; Reporting; Systematic reviews

Indexed keywords

ARTICLE; COCHRANE LIBRARY; CROSS-SECTIONAL STUDY; HUMAN; MEDLINE; PRIORITY JOURNAL; PUBLICATION; RANDOM SAMPLE; SENSITIVITY ANALYSIS; STATISTICAL ANALYSIS; STATISTICAL SIGNIFICANCE; SYSTEMATIC REVIEW (TOPIC); CLINICAL TRIAL (TOPIC); EPIDEMIOLOGY; META ANALYSIS (TOPIC); METHODOLOGY;

CLINICAL TRIALS AS TOPIC; CROSS-SECTIONAL STUDIES; DATA INTERPRETATION, STATISTICAL; EPIDEMIOLOGIC METHODS; HUMANS; META-ANALYSIS AS TOPIC; RESEARCH DESIGN; SYSTEMATIC REVIEWS AS TOPIC;

EID: 85042514658     PISSN: 08954356     EISSN: 18785921     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2017.11.022     Document Type: Article

References (42)

1. Deeks, J.J., Higgins, J.P.T., Altman, D.G., Analysing data and undertaking meta-analyses. Higgins, J.P.T., Green, S., (eds.) Cochrane handbook for systematic reviews of interventions version 5.1.0, 2011, The Cochrane Collaboration Available at: www.cochrane-handbook.org. Accessed July 1, 2017.
2. Ioannidis, J.P., Patsopoulos, N.A., Rothstein, H.R., Reasons or excuses for avoiding meta-analysis in forest plots. BMJ 336 (2008), 1413–1415.
3. Murad, M.H., Montori, V.M., Ioannidis, J.P., Jaeschke, R., Devereaux, P.J., Prasad, K., et al. How to read a systematic review and meta-analysis and apply the results to patient care: users' guides to the medical literature. JAMA 312 (2014), 171–179.
4. McKenzie, J.E., Beller, E.M., Forbes, A.B., Introduction to systematic reviews and meta-analysis. Respirology 21:4 (2016), 626–637.
5. Fisher, D.J., Carpenter, J.R., Morris, T.P., Freeman, S.C., Tierney, J.F., Meta-analytical methods to identify who benefits most from treatments: daft, deluded, or deft approach?. BMJ, 356, 2017, j573.
6. Riley, R.D., Gates, S., Neilson, J., Alfirevic, Z., Statistical methods can be improved within Cochrane pregnancy and childbirth reviews. J Clin Epidemiol 64 (2011), 608–618.
7. Donegan, S., Williams, L., Dias, S., Tudur-Smith, C., Welton, N., Exploring treatment by covariate interactions using subgroup analysis and meta-regression in cochrane reviews: a review of recent practice. PLoS One, 10, 2015, e0128804.
8. Koletsi, D., Valla, K., Fleming, P.S., Chaimani, A., Pandis, N., Assessment of publication bias required improvement in oral health systematic reviews. J Clin Epidemiol 76 (2016), 118–124.
9. Atakpo, P., Vassar, M., Publication bias in dermatology systematic reviews and meta-analyses. J Dermatol Sci 82:2 (2016), 69–74.
10. Hedin, R.J., Umberham, B.A., Detweiler, B.N., Kollmorgen, L., Vassar, M., Publication bias and nonreporting found in majority of systematic reviews and meta-analyses in anesthesiology journals. Anesth Analg 123 (2016), 1018–1025.
11. Herrmann, D., Sinnett, P., Holmes, J., Khan, S., Koller, C., Vassar, M., Statistical controversies in clinical research: publication bias evaluations are not routinely conducted in clinical oncology systematic reviews. Ann Oncol 28 (2017), 931–937.
12. Richardson, M., Garner, P., Donegan, S., Cluster randomised trials in cochrane reviews: evaluation of methodological and reporting practice. PLoS One, 11, 2016, e0151818.
13. Nolan, S.J., Hambleton, I., Dwan, K., The use and reporting of the cross-over study design in clinical trials and systematic reviews: a systematic assessment. PLoS One, 11, 2016, e0159014.
14. Page, M.J., Shamseer, L., Altman, D.G., Tetzlaff, J., Sampson, M., Tricco, A.C., et al. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med, 13(5), 2016, e1002028.
15. Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev, 4(1), 2015, 1.
16. Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ, 349, 2015, g7647.
17. Moher, D., Tetzlaff, J., Tricco, A.C., Sampson, M., Altman, D.G., Epidemiology and reporting characteristics of systematic reviews. Plos Med, 4, 2007, e78.
18. Shemilt, I., Khan, N., Park, S., Thomas, J., Use of cost-effectiveness analysis to compare the efficiency of study identification methods in systematic reviews. Syst Rev, 5(1), 2016, 140.
19. Higgins, J.P.T., Lasserson, T., Chandler, J., Tovey, D., Churchill, R., Methodological expectations of Cochrane intervention reviews. 2016, Cochrane, London.
20. Schriger, D.L., Altman, D.G., Vetter, J.A., Heafner, T., Moher, D., Forest plots in reports of systematic reviews: a cross-sectional study reviewing current practice. Int J Epidemiol 39 (2010), 421–429.
21. StataCorp. Stata statistical software: release 14. 2015, StataCorp LP, College Station, TX.
22. The Nordic Cochrane Centre (The Cochrane Collaboration). Review Manager (RevMan). 5.1. 2011, The Nordic Cochrane Centre, The Cochrane Collaboration, Copenhagen.
23. R Development Core Team. R: a language and environment for statistical computing. 2012, R Foundation for Statistical Computing, Vienna, Austria.
24. Riley, R.D., Higgins, J.P., Deeks, J.J., Interpretation of random effects meta-analyses. BMJ, 342, 2011, d549.
25. Higgins, J.P., Thompson, S.G., Deeks, J.J., Altman, D.G., Measuring inconsistency in meta-analyses. BMJ 327 (2003), 557–560.
26. AHRQ Publication No. 10(14)-EHC063-EF Methods guide for effectiveness and comparative effectiveness reviews, 2014, Agency for Healthcare Research and Quality, Rockville, MD Chapters available at: www.effectivehealthcare.ahrq.gov. Accessed July 1, 2017.
27. IOM (Institute of Medicine). Finding what works in health care: standards for systematic reviews. 2011, The National Academies Press, Washington, DC.
28. DerSimonian, R., Laird, N., Meta-analysis in clinical trials. Control Clin Trials 7 (1986), 177–188.
29. Sidik, K., Jonkman, J.N., Simple heterogeneity variance estimation for meta-analysis. J R Stat Soc Ser C Appl Stat 54 (2005), 367–384.
30. Peters, J.L., Sutton, A.J., Jones, D.R., Abrams, K.R., Rushton, L., Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. J Clin Epidemiol 61 (2008), 991–996.
31. Egger, M., Davey Smith, G., Schneider, M., Minder, C., Bias in meta-analysis detected by a simple, graphical test. BMJ 315 (1997), 629–634.
32. Sterne, J.A., Sutton, A.J., Ioannidis, J.P., Terrin, N., Jones, D.R., Lau, J., et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ, 343, 2011, d4002.
33. Borenstein, M., Hedges, L.V., Higgins, J.P.T., Rothstein, H.R., Introduction to meta-analysis. 2009, John Wiley & Sons, Ltd, West Sussex, UK.
34. Altman, D.G., Bland, J.M., Interaction revisited: the difference between two estimates. BMJ, 326, 2003, 219.
35. Ioannidis, J.P., The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q 94:3 (2016), 485–514.
36. Petropoulou, M., Nikolakopoulou, A., Veroniki, A.A., Rios, P., Vafaei, A., Zarin, W., et al. Bibliographic study showed improving statistical methodology of network meta-analyses published between 1999 and 2015. J Clin Epidemiol 82 (2017), 20–28.
37. Hutton, B., Wolfe, D., Moher, D., Shamseer, L., Reporting guidance considerations from a statistical perspective: overview of tools to enhance the rigour of reporting of randomised trials and systematic reviews. Evid Based Ment Health 20:2 (2017), 46–52.
38. Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Plos Med, 6, 2009, e1000097.
39. Page, M.J., Moher, D., Mass production of systematic reviews and meta-analyses: an exercise in mega-silliness?. Milbank Q 94:3 (2016), 515–519.
40. Ioannidis, J.P.A., Greenland, S., Hlatky, M.A., Khoury, M.J., Macleod, M.R., Moher, D., et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet 383 (2014), 166–175.
41. Moher, D., Glasziou, P., Chalmers, I., Nasser, M., Bossuyt, P.M., Korevaar, D.A., et al. Increasing value and reducing waste in biomedical research: who's listening?. Lancet 387 (2016), 1573–1586.
42. IntHout, J., Ioannidis, J.P., Rovers, M.M., Goeman, J.J., Plea for routinely presenting prediction intervals in meta-analysis. BMJ Open, 6(7), 2016, e010247.