Does health informatics have a replication crisis?

Journal of the American Medical Informatics Association

Volumn 25, Issue 8, 2018, Pages 963-968

  Coiera, Enrico ^a   Ammenwerth, Elske ^b   Georgiou, Andrew ^a   Magrabi, Farah ^a  

^a MACQUARIE UNIVERSITY   (Australia)

^b UNIVERSITY FOR HEALTH SCIENCES   (Austria)

Author keywords

Evaluation; Evidence based informatics; Replication; Research bias

Indexed keywords

ARTICLE; CLINICAL RESEARCH; EVIDENCE BASED PRACTICE; MEDICAL INFORMATICS; PUBLICATION; REPLICATION STUDY; SCIENTIFIC LITERATURE; SYSTEMATIC REVIEW (TOPIC); METHODOLOGY; REPRODUCIBILITY;

EVIDENCE-BASED PRACTICE; MEDICAL INFORMATICS; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN;

EID: 85055022836     PISSN: 10675027     EISSN: 1527974X     Source Type: Journal    
DOI: 10.1093/jamia/ocy028     Document Type: Article

References (40)

1. Popper K. Conjectures and Refutations. London: Routledge and Kegan Paul; 1963.
2. Ioannidis JP. Why most published research findings are false. PLoS Med 2005;2(8):e124.
3. Freedman LP, Cockburn IM, Simcoe TS. The economics of reproducibility in preclinical research. PLOS Biol 2015;13(6):e1002165.
4. Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P. Comparison of registered and published primary outcomes in randomized controlled trials. JAMA 2009;302(9):977-984.
5. Simonsohn U, Simmons JP, Nelson LD. Better P-curves: Making P-curve analysis more robust to errors, fraud, and ambitious P-hacking, a Reply to Ulrich and Miller. J Exp Psychol Gen 2015;144(6):1146-1152.
6. Chalmers l. Underreporting research is scientific misconduct. JAMA 1990;263(10):1405-1408.
7. Macleod MR, McLean AL, Kyriakopoulou A, et al. Risk of bias in reports of in vivo research: A focus for improvement. PLoS Biol 2015;13(10): E1002273.
8. Curtis MJ, Abernethy DR. Replication - why we need to publish our findings. Pharmacol Res Perspect 2015;3(4):e00164.
9. Anderson CJ, Bahnik S, Barnett-Cowan M, et al. Response to Comment on "Estimating the reproducibility of psychological science". Science 2016;351:1037-1039.
10. Open Science Collaboration. Estimating the reproducibility of psychological science. Science 2015;349(6251):943.
11. Peng RD. Reproducible research and Biostatistics. Biostatistics 2009; 10(3):405-408.
12. Drummond C. Replicability is not reproducibility: nor is it good science. In: Proceedings of the EvaluationMethods for Machine LearningWorkshop 26th International Conference forMachine Learning; 2009:4; Montreal.
13. Liu JLY, Wyatt JC. The case for randomized controlled trials to assess the impact of clinical information systems. J Am Med Inform Assoc 2011; 18(2):173-180.
14. Pashler H, Harris CR. Is the replicability crisis overblown? Three arguments examined. Perspect Psychol Sci 2012;7(6):531-536.
15. Nakagawa S, Parker TH. Replicating research in ecology and evolution: Feasibility, incentives, and the cost-benefit conundrum. BMC Biol 2015;13(1):88.
16. Ammenwerth E, Rigby M. Evidence-Based Health Informatics. Amsterdam: IOS Press; 2016.
17. Kim MO, Coiera E, Magrabi F. Problems with health information technology and their effects on care delivery and patient outcomes: A systematic review. J Am Med Inform Assoc 2017;24(2):246-250.
18. Black AD, Car J, Pagliari C, et al. The impact of eHealth on the quality and safety of health care: A systematic overview. PLoSMed 2011;8(1):e1000387.
19. Shekelle P, Morton SC, Keeler EB. Costs and Benefits of Health Information Technology. Evidence Report/Technology Assessment. Rockville, MD: AHRQ; 2006.
20. Madigan D, Ryan PB, Schuemie M, et al. Evaluating the impact of database heterogeneity on observational study results. Am J Epidemiol 2013;178(4):645-651.
21. Han YY, Carcillo JA, Venkataraman ST, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics 2005;116(6):1506-1512.
22. Brunette DD, Tersteeg J, Brown N, et al. Implementation of computerized physician order entry for critical patients in an Academic Emergency Department is not associated with a change in mortality rate. Western J Emerg Med 2013;14(2):114-120.
23. Al-Dorzi HM, Tamim HM, Cherfan A, Hassan MA, Taher S, Arabi YM. Impact of computerized physician order entry (CPOE) system on the outcome of critically ill adult patients: A before-after study. BMC Med Inform Decis Making 2011;11:71.
24. Del Beccaro MA, Jeffries HE, Eisenberg MA, Harry ED. Computerized provider order entry implementation: no association with increased mortality rates in an intensive care unit. Pediatrics 2006;118(1):290-295.
25. Longhurst CA, Parast L, Sandborg CI, et al. Decrease in hospital-wide mortality rate after implementation of a commercially sold computerized physician order entry system. Pediatrics 2010;126(1):14-21.
26. Ammenwerth E, Talmon J, Ash JS, et al. Impact of CPOE on mortality rates - contradictory findings, important messages. Methods Archive 2006;45(6):586-593.
27. Helfrich CD, Damschroder LJ, Hagedorn HJ, et al. A critical synthesis of literature on the promoting action on research implementation in health services (PARIHS) framework. Implement Sci 2010;5:82.
28. Johnson AEW, Pollard TJ, Shen L, et al. MIMIC-III, a freely accessible critical care database. Scientific Data 2016;3:160035.
29. Hripcsak G, Duke JD, Shah NH, et al. Observational Health Data Sciences and Informatics (OHDSI): Opportunities for Observational Researchers. Stud Health Technol Inform 2015;216:574-578.
30. Kelly CD. Replicating empirical research in behavioral ecology: Howandwhy it should be done but rarely ever is. Quart Rev Biol 2006;81(3):221-236.
31. Palmer AR. Quasi-replication and the contract of error: Lessons from sex ratios, heritabilities and fluctuating asymmetry. Ann Rev Ecol Syst 2000;31(1):441-480.
32. Brender J, Talmon J, de Keizer N, Nyk€anen P, Rigby M, Ammenwerth E. STARE-HI - Statement on Reporting of Evaluation Studies in Health Informatics: Explanation and Elaboration. Appl Clin Inform 2013;4(3):331-358.
33. De Angelis C, Drazen JM, Frizelle FA, et al. Clinical trial registration: A statement from the International Committee of Medical Journal Editors. New Engl J Med 2004;351:1250-1251.
34. Taichman DB, Sahni P, Pinborg A, et al. Data Sharing Statements for Clinical Trials-A Requirement of the International Committee of Medical Journal Editors. N Engl J Med 2017;376:2277-2279.
35. Dunn AG, Coiera E, Bourgeois FT, Mandl KD. Conflict of interest disclosure in biomedical research: A review of current practices, biases, and the role of public registries in improving transparency. Res Integrity Peer Rev 2016;1(1):1.
36. Almugbel R, Hung L-H, Hu J, et al. Reproducible Bioconductor workflows using browser-based interactive notebooks and containers. J Am Med Inform Assoc 2018;25(1):4-12.
37. Coiera E, Choong MK, Tsafnat G, Hibbert P, Runciman WB. Linking quality indicators to clinical trials: An automated approach. Int J Qual Healthcare 2017;29(4):571-578.
38. Cotterill S, Knowles S, Martindale A-M, et al. Getting messier with TIDieR: Embracing context and complexity in intervention reporting. BMC Med Res Methodol 2018;18(1):12.
39. Walker R, Rocha da Silva P. Emerging trends in peer review - A survey. Front Neurosci 2015;9(169). doi: 10.3389/fnins.2015.00169.
40. Moher D, Altman DG. Four proposals to help improve the medical research literature. PLOS Med 2015;12(9):e1001864.