Improving reproducibility by using high-throughput observational studies with empirical calibration

Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences

Volumn 376, Issue 2128, 2018, Pages

  Schuemie, Martijn J ^a,b   Ryan, Patrick B ^a,b,c   Hripcsak, George ^a,c,d   Madigan, David ^a,e   Suchard, Marc A ^a,f,g  

^a Observational Health Data Sciences and Informatics   (United States)

^b JANSSEN RESEARCH AND DEVELOPMENT   (United States)

^c Department of Medicine   (United States)

^d NEW YORK PRESBYTERIAN HOSPITAL   (United States)

^e Columbia University ^*  (United States)

^f Department of Biomathematics ^*  (United States)

^g UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Medicine; Observational research; Publication bias; Reproducibility

Indexed keywords

CALIBRATION; MEDICINE;

EMPIRICAL CALIBRATION; GENERATION PROCESS; HIGH THROUGHPUT; OBSERVATIONAL RESEARCH; OBSERVATIONAL STUDY; PUBLICATION BIAS; REPRODUCIBILITIES; STANDARDIZED METHODS;

THROUGHPUT;

ARTICLE; CALIBRATION; EFFECT SIZE; HUMAN; OBSERVATIONAL STUDY; PUBLICATION BIAS; RELIABILITY; REPRODUCIBILITY;

EID: 85059988612     PISSN: 1364503X     EISSN: None     Source Type: Journal    
DOI: 10.1098/rsta.2017.0356     Document Type: Article

References (41)

1. Baker M. 2016 1,500 scientists lift the lid on reproducibility. Nature 533, 452-454. (doi:10.1038/533452a)
2. Ioannidis JP. 2005 Why most published research findings are false. PLoS Med. 2, e124. (doi:10.1371/journal.pmed.0020124)
3. Munafò MR et al. 2017 A manifesto for reproducible science. Nat. Hum. Behav. 1, 0021. (doi:10.1038/s41562-016-0021)
4. Dal-Ré R, Ioannidis JP, Bracken MB, Buffler PA, Chan A-W, Franco EL, La Vecchia C, Weiderpass E. 2014 Making prospective registration of observational research a reality. Sci. Transl. Med. 6, 224cm1. (doi:10.1126/scitranslmed.3007513)
5. Hripcsak G et al. 2016 Characterizing treatment pathways at scale using the OHDSI network. Proc. Natl Acad. Sci. USA 113, 7329-7336. (doi:10.1073/pnas.1510502113)
6. Schuemie MJ, Hripcsak G, Ryan PB, Madigan D, Suchard MA. 2018 Empirical confidence interval calibration for population-level effect estimation studies in observational healthcare data. Proc. Natl Acad. Sci. USA 9, 201708282. (doi:10.1073/pnas.1708282114)
7. WHO. 2017 WHO fact sheet: depression.
8. Hernan MA, Robins JM. 2016 Using big data to emulate a target trial when a randomized trial is not available. Am. J. Epidemiol. 183, 758-764. (doi:10.1093/aje/kwv254)
9. Rosenbaum PR, Rubin DB. 1983 The central role of the propensity score in observational studies for causal effects. Biometrika 70, 41-55. (doi:10.1093/biomet/70.1.41)
10. Suchard MA, Simpson SE, Zorych I, Ryan P, Madigan D. 2013 Massive parallelization of serial inference algorithms for a complex generalized linear model. ACM Trans. Model. Comput. Simul. 23, 1-17. (doi:10.1145/2414416.2414791)
11. Voss EA, Boyce RD, Ryan PB, van der Lei J, Rijnbeek PR, Schuemie MJ. 2016 Accuracy of an automated knowledge base for identifying drug adverse reactions. J. Biomed. Inform. 66, 72-81. (doi:10.1016/j.jbi.2016.12.005)
12. Avillach P et al. 2013 Design and validation of an automated method to detect known adverse drug reactions in MEDLINE: a contribution from the EU-ADR project. J. Am. Med. Inform. Assoc. 20, 446-452. (doi:10.1136/amiajnl-2012-001083)
13. Kilicoglu H, Rosemblat G, Fiszman M, Rindflesch TC. 2011 Constructing a semantic predication gold standard from the biomedical literature. BMC Bioinformatics 12, 486. (doi:10.1186/1471-2105-12-486)
14. US Food & Drug Administration. 2016 FDA adverse event reporting system (FAERS). cited 2016; See http://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Surveillance/AdverseDrugEffects/
15. Duke J, Friedlin J, Li X. 2013 Consistency in the safety labeling of bioequivalent medications. Pharmacoepidemiol. Drug Saf. 22, 294-301. (doi:10.1002/pds.3351)
16. Wisniewski AF et al. 2016 Good signal detection practices: evidence from IMI PROTECT. Drug Saf. 39, 469-490. (doi:10.1007/s40264-016-0405-1)
17. Noren GN, Caster O, Juhlin K, Lindquist M. 2014 Zoo or savannah? Choice of training ground for evidence-based pharmacovigilance. Drug Saf. 37, 655-659. (doi:10.1007/ s40264-014-0198-z)
18. Lee YC, Lin C-H, Lin M-S, Lu Y, Chang C-H, Lin J-W. 2016 Comparison of the effects of serotonin-norepinephrine reuptake inhibitors versus selective serotonin reuptake inhibitors on cerebrovascular events. J. Clin. Psychiatry 77, e1-e7. (doi:10.4088/JCP.14m09394)
19. Duke JD et al. 2017 Risk of angioedema associated with levetiracetam compared with phenytoin: findings of the observational health data sciences and informatics research network. Epilepsia 58, e101-e106. (doi:10.1111/epi.13828)
20. Ramcharran D, Qiu H, Schuemie MJ, Ryan PB. 2017 Atypical antipsychotics and the risk of falls and fractures among older adults: an emulation analysis and an evaluation of additional confounding control strategies. J. Clin. Psychopharmacol. 37, 162-168. (doi:10.1097/JCP.0000000000000647)
21. Wang Y, Desai M, Ryan PB, Defalco FJ, Schuemie MJ, Stang PE, Berlin JA, Yuan Z. 2017 Incidence of diabetic ketoacidosis among patients with type 2 diabetes mellitus treated with SGLT2 inhibitors and other antihyperglycemic agents. Diabetes Res. Clin. Pract. 128, 83-90. (doi:10.1016/j.diabres.2017.04.004)
22. Madigan D, Ryan PB, Schuemie M, Stang PE, Overhage JM, Hartzema AG, Suchard MA, Dumouchel W, Berlin JA. 2013 Evaluating the impact of database heterogeneity on observational study results. Am. J. Epidemiol. 178, 645-651. (doi:10.1093/aje/kwt010)
23. Higgins JPT, Thompson SG, Deeks JJ, Altman DG. 2003 Measuring inconsistency in meta-analyses. BMJ 327, 557-560. (doi:10.1136/bmj.327.7414.557)
24. Gartlehner G et al. 2008 Drug class review: second generation antidepressants. Final report update 4. Portland, OR: Oregon Health & Science University.
25. Cipriani A, La Ferla T, Furukawa TA, Signoretti A, Nakagawa A, Churchill R, McGuire H, Barbui C. 2010 Sertraline versus other antidepressive agents for depression. Cochrane Database Syst. Rev. 4, CD006117. (doi:10.1002/14651858.CD006117)
26. Gelenberg AJ et al. 2010 Practice guideline for the treatment of patients with major depressive disorder third edition. Am. J. Psychiatry 167, 1. (doi:10.1176/appi.ajp.2009.09111601)
27. Walker PW, Cole JO, Gardner EA, Hughes AR, Johnston JA, Batey SR, Lineberry CG. 1993 Improvement in fluoxetine-associated sexual dysfunction in patients switched to bupropion. J. Clin. Psychiatry 54, 459-465.
28. Schuemie MJ, Ryan PB, Dumouchel W, Suchard MA, Madigan D. 2014 Interpreting observational studies: why empirical calibration is needed to correct p-values. Stat. Med. 33, 209-218. (doi:10.1002/sim.5925)
29. Cardwell CR, Abnet CC, Cantwell MM, Murray LJ. 2010 Exposure to oral bisphosphonates and risk of esophageal cancer. JAMA 304, 657-663. (doi:10.1001/jama.2010.1098)
30. de Vries F, De Vries C, Cooper C, Leufkens B, Van Staa T-P. 2006 Reanalysis of two studies with contrasting results on the association between statin use and fracture risk: the general practice research database. Int. J. Epidemiol. 35, 1301-1308. (doi:10.1093/ije/dyl147)
31. Green J, Czanner G, Reeves G, Watson J, Wise L, Beral V. 2010 Oral bisphosphonates and risk of cancer of oesophagus, stomach, and colorectum: case-control analysis within a UK primary care cohort. BMJ 341, c4444. (doi:10.1136/bmj.c4444)
32. Grodstein F, Stampfer MJ, Manson JAE, Colditz GA, Willett WC, Rosner B, Speizer FE, Hennekens CH. 1996 Postmenopausal estrogen and progestin use and the risk of cardiovascular disease. N Engl. J. Med. 335, 453-461. (doi:10.1056/NEJM1996081533 50701)
33. Manson JE et al. 2003 Estrogen plus progestin and the risk of coronary heart disease. N Engl. J. Med. 349, 523-534. (doi:10.1056/NEJMoa030808)
34. Mayes LC, Horwitz RI, Feinstein AR. 1988 A collection of 56 topics with contradictory results in case-control research. Int. J. Epidemiol. 17, 680-685. (doi:10.1093/ije/17.3.680)
35. Meier CR, Schlienger RG, Kraenzlin ME, Schlegel B, Jick H. 2000 HMG-CoA reductase inhibitors and the risk of fractures. JAMA 283, 3205-3210. (doi:10.1001/jama.283.24.3205)
36. van Staa TP, Wegman S, de Vries F, Leufkens B, Cooper C. 2001 Use of statins and risk of fractures. JAMA 285, 1850-1855. (doi:10.1001/jama.285.14.1850)
37. Varas-Lorenzo C, García-Rodríguez LA, Perez-Gutthann S, Duque-Oliart A. 2000 Hormone replacement therapy and incidence of acute myocardial infarction. A population-based nested case-control study. Circulation 101, 2572-2578. (doi:10.1161/01.CIR.101.22.2572)
38. Wilson PW, Garrison RJ, Castelli WP. 1986 Postmenopausal estrogen use, cigarette smoking, and cardiovascular morbidity in women over 50. The Framingham study. N Engl. J. Med. 313, 1038-1043. (doi:10.1056/NEJM198510243131702)
39. Arnold BF, Ercumen A, Benjamin-Chung J, Colford JM. 2016 Brief report: negative controls to detect selection bias and measurement bias in epidemiologic studies. Epidemiology 27, 637-641. (doi:10.1097/EDE.0000000000000504)
40. Desai JR, Hyde CL, Kabadi S, St Louis M, Bonato V, Katrina Loomis A, Galaznik A, Berger ML. 2017 Utilization of positive and negative controls to examine comorbid associations in observational database studies. Med. Care 55, 244-251. (doi:10.1097/MLR.0000000000000640)
41. Farrington CP. 1995 Relative incidence estimation from case series for vaccine safety evaluation. Biometrics 51, 228-235. (doi:10.2307/2533328)