Reproducible research practices are underused in systematic reviews of biomedical interventions

Journal of Clinical Epidemiology

Volumn 94, Issue , 2018, Pages 8-18

  Page, Matthew J ^a   Altman, Douglas G ^b   Shamseer, Larissa ^c,d   McKenzie, Joanne E ^a   Ahmadzai, Nadera ^c   Wolfe, Dianna ^c   Yazdi, Fatemeh ^c   Catalá López, Ferrán ^c,e   Tricco, Andrea C ^f,g   Moher, David ^c,d  

^a MONASH UNIVERSITY   (Australia)

^b UNIVERSITY OF OXFORD   (United Kingdom)

^c OTTAWA HOSPITAL RESEARCH INSTITUTE   (Canada)

^d UNIVERSITY OF OTTAWA   (Canada)

^e UNIVERSITY OF VALENCIA   (Spain)

^f ST MICHAEL'S HOSPITAL   (Canada)

^g UNIVERSITY OF TORONTO   (Canada)

Author keywords

Data sharing; Methodology; Quality; Reporting; Reproducibility; Systematic reviews

Indexed keywords

CLINICAL EXAMINATION; HUMAN; MEDICAL LITERATURE; MEDICAL RESEARCH; META ANALYSIS (TOPIC); METHODOLOGY; PATIENT-REPORTED OUTCOME; PRIORITY JOURNAL; REPRODUCIBILITY; REVIEW; SYSTEMATIC REVIEW (TOPIC); PROCEDURES;

BIOMEDICAL RESEARCH; HUMANS; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN; SYSTEMATIC REVIEWS AS TOPIC;

EID: 85040199860     PISSN: 08954356     EISSN: 18785921     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2017.10.017     Document Type: Review

References (39)

1. Iqbal, S.A., Wallach, J.D., Khoury, M.J., Schully, S.D., Ioannidis, J.P., Reproducible research practices and transparency across the biomedical literature. PLoS Biol, 14(1), 2016, e1002333.
2. Nosek, B.A., Alter, G., Banks, G.C., Borsboom, D., Bowman, S.D., Breckler, S.J., et al. Scientific standards. Promoting an open research culture. Science 348 (2015), 1422–1425.
3. Errington, T.M., Iorns, E., Gunn, W., Tan, F.E., Lomax, J., Nosek, B.A., An open investigation of the reproducibility of cancer biology research. Elife, 3, 2014, e04333.
4. Goodman, S.N., Fanelli, D., Ioannidis, J.P., What does research reproducibility mean?. Sci Transl Med, 8(341), 2016, 341ps12.
5. Lakens, D., Hilgard, J., Staaks, J., On the reproducibility of meta-analyses: six practical recommendations. BMC Psychol, 4(1), 2016, 24.
6. Elliott, J.H., Turner, T., Clavisi, O., Thomas, J., Higgins, J.P., Mavergames, C., et al. Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. PLoS Med, 11(2), 2014, e1001603.
7. Crequit, P., Trinquart, L., Ravaud, P., Live cumulative network meta-analysis: protocol for second-line treatments in advanced non-small-cell lung cancer with wild-type or unknown status for epidermal growth factor receptor. BMJ Open, 6(8), 2016, e011841.
8. Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., The PRISMA group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med, 6, 2009, e1000097.
9. Stevens, A., Shamseer, L., Weinstein, E., Yazdi, F., Turner, L., Thielman, J., et al. Relation of completeness of reporting of health research to journals' endorsement of reporting guidelines: systematic review. BMJ, 348, 2014, g3804.
10. Tam, W.W., Lo, K.K., Khalechelvam, P., Endorsement of PRISMA statement and quality of systematic reviews and meta-analyses published in nursing journals: a cross-sectional study. BMJ Open, 7(2), 2017, e013905.
11. Peters, J.P., Hooft, L., Grolman, W., Stegeman, I., Reporting quality of systematic reviews and meta-analyses of otorhinolaryngologic articles based on the PRISMA statement. PLoS One, 10, 2015, e0136540.
12. Liu, Y., Zhang, R., Huang, J., Zhao, X., Liu, D., Sun, W., et al. Reporting quality of systematic reviews/meta-analyses of acupuncture. PLoS One, 9, 2014, e113172.
13. Tunis, A.S., McInnes, M.D., Hanna, R., Esmail, K., Association of study quality with completeness of reporting: have completeness of reporting and quality of systematic reviews and meta-analyses in major radiology journals changed since publication of the PRISMA statement?. Radiology 269 (2013), 413–426.
14. Gagnier, J.J., Kellam, P.J., Reporting and methodological quality of systematic reviews in the orthopaedic literature. J Bone Joint Surg Am 95 (2013), e771–e777.
15. Wolfenden, L., Grimshaw, J., Williams, C.M., Yoong, S.L., Time to consider sharing data extracted from trials included in systematic reviews. Syst Rev, 5(1), 2016, 185.
16. Loder, E., Groves, T., The BMJ requires data sharing on request for all trials. BMJ, 350, 2015, h2373.
17. Page, M.J., Shamseer, L., Altman, D.G., Tetzlaff, J., Sampson, M., Tricco, A.C., et al. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med, 13(5), 2016, e1002028.
18. Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev, 4(1), 2015, 1.
19. Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ, 349, 2015, g7647.
20. Moher, D., Tetzlaff, J., Tricco, A.C., Sampson, M., Altman, D.G., Epidemiology and reporting characteristics of systematic reviews. PLoS Med, 4, 2007, e78.
21. Liberati, A., Altman, D.G., Tetzlaff, J., Mulrow, C., Gotzsche, P.C., Ioannidis, J.P., et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ, 339, 2009, b2700.
22. Higgins, J.P.T., Lasserson, T., Chandler, J., Tovey, D., Churchill, R., Methodological expectations of Cochrane intervention reviews. 2016, Cochrane, London.
23. StataCorp. Stata Statistical Software: Release 14. 2015, StataCorp LP, College Station, TX.
24. DerSimonian, R., Laird, N., Meta-analysis in clinical trials. Control Clin Trials 7 (1986), 177–188.
25. Sidik, K., Jonkman, J.N., A comparison of heterogeneity variance estimators in combining results of studies. Stat Med 26 (2007), 1964–1981.
26. The Nordic Cochrane Centre (The Cochrane Collaboration). Review Manager (RevMan). 5.1. 2011, The Nordic Cochrane Centre, The Cochrane Collaboration, Copenhagen.
27. Ioannidis, J.P., The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q 94:3 (2016), 485–514.
28. Kidwell, M.C., Lazarevic, L.B., Baranski, E., Hardwicke, T.E., Piechowski, S., Falkenberg, L.S., et al. Badges to acknowledge open practices: a simple, low-cost, effective method for increasing transparency. PLoS Biol, 14(5), 2016, e1002456.
29. McKiernan, E.C., Bourne, P.E., Brown, C.T., Buck, S., Kenall, A., Lin, J., et al. How open science helps researchers succeed. Elife, 5, 2016, e16800.
30. Nosek, B.A., Errington, T.M., Making sense of replications. Elife, 6, 2017, e23383.
31. Open Science Collaboration. PSYCHOLOGY. Estimating the reproducibility of psychological science. Science, 349, 2015, aac4716.
32. Hill, J., Hoyt, J., van Eijk, A.M., D'Mello-Guyett, L., Ter Kuile, F.O., Steketee, R., et al. Factors affecting the delivery, access, and use of interventions to prevent malaria in pregnancy in sub-Saharan Africa: a systematic review and meta-analysis. PLoS Med, 10(7), 2013, e1001488.
33. Altman, D.G., Cates, C., The need for individual trial results in reports of systematic reviews. BMJ, 323, 2001, 776.
34. Gotzsche, P.C., Hrobjartsson, A., Maric, K., Tendal, B., Data extraction errors in meta-analyses that use standardized mean differences. JAMA 298 (2007), 430–437.
35. Barnes, C., Boutron, I., Giraudeau, B., Porcher, R., Altman, D.G., Ravaud, P., Impact of an online writing aid tool for writing a randomized trial report: the COBWEB (Consort-based WEB tool) randomized controlled trial. BMC Med, 13, 2015, 221.
36. Shamseer, L., Roberts, J., Disclosure of data and statistical commands should accompany completely reported studies. J Clin Epidemiol 70 (2016), 272–274.
37. Ip, S., Hadar, N., Keefe, S., Parkin, C., Iovin, R., Balk, E.M., et al. A Web-based archive of systematic review data. Syst Rev, 1, 2012, 15.
38. Li, T., Vedula, S.S., Hadar, N., Parkin, C., Lau, J., Dickersin, K., Innovations in data collection, management, and archiving for systematic reviews. Ann Intern Med 162 (2015), 287–294.
39. Moher, D., Goodman, S.N., Ioannidis, J.P., Academic criteria for appointment, promotion and rewards in medical research: where's the evidence?. Eur J Clin Invest 46 (2016), 383–385.