When and How Can Real World Data Analyses Substitute for Randomized Controlled Trials?

Clinical Pharmacology and Therapeutics

Volumn 102, Issue 6, 2017, Pages 924-933

  Franklin, Jessica M ^a   Schneeweiss, Sebastian ^a  

^a BRIGHAM AND WOMEN S HOSPITAL   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; DATA ANALYSIS; DATA BASE; DECISION MAKING; DRUG COST; DRUG EFFECT; DRUG EFFICACY; DRUG SAFETY; EVIDENCE BASED PRACTICE; GOLD STANDARD; HEALTH CARE; HUMAN; PRIORITY JOURNAL; RANDOMIZED CONTROLLED TRIAL; REGISTER; SENSITIVITY ANALYSIS; STUDY DESIGN; TREATMENT DURATION; VALIDITY; DRUG APPROVAL; FACTUAL DATABASE; INFORMATION PROCESSING; PROCEDURES; RANDOMIZED CONTROLLED TRIAL (TOPIC); STATISTICS;

DATA COLLECTION; DATABASES, FACTUAL; DECISION MAKING; DRUG APPROVAL; HUMANS; RANDOMIZED CONTROLLED TRIALS AS TOPIC; REGISTRIES; STATISTICS AS TOPIC;

EID: 85029886626     PISSN: 00099236     EISSN: 15326535     Source Type: Journal    
DOI: 10.1002/cpt.857     Document Type: Article

References (79)

1. Bonamici S. H.R.34 - 114th Congress (2015–2016): 21st Century Cures Act. . Published December 13, 2016. Accessed 15 June 2017.
2. U.S. Food and Drug Administration. Prescription Drug User Fee Act (PDUFA) – PDUFA VI: Fiscal Years 2018–2022. . Accessed 15 June 2017.
3. Schneeweiss, S. & Avorn, J. A review of uses of health care utilization databases for epidemiologic research on therapeutics. J. Clin. Epidemiol. 58, 323–337 (2005).
4. Schneeweiss, S., Gagne, J.J., Glynn, R.J., Ruhl, M. & Rassen, J.A. Assessing the comparative effectiveness of newly marketed medications: methodological challenges and implications for drug development. Clin. Pharmacol. Ther. 90, 777–790 (2011).
5. Schneeweiss, S. Improving therapeutic effectiveness and safety through big healthcare data. Clin. Pharmacol. Ther. 99, 262–265 (2016).
6. Eichler, H.G. et al. Adaptive licensing: taking the next step in the evolution of drug approval. Clin. Pharmacol. Ther. 91, 426–437 (2012).
7. Eichler, H.G. et al. From adaptive licensing to adaptive pathways: delivering a flexible life-span approach to bring new drugs to patients. Clin. Pharmacol. Ther. 97, 234–246 (2015).
8. Sherman, R.E. et al. Real-world evidence—what is it and what can it tell us. N. Engl. J. Med. 375, 2293–2297 (2016).
9. Anglemyer, A., Horvath, H.T. & Bero, L. Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials. The Cochrane Collaboration. Cochrane Database of Systematic Reviews. (John Wiley & Sons, Chichester, UK, 2014). . Accessed 12 May 2016.
10. Concato, J., Shah, N. & Horwitz, R.I. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N. Engl. J. Med. 342, 1887–1892 (2000).
11. Ioannidis, J.P. et al. Comparison of evidence of treatment effects in randomized and nonrandomized studies. JAMA 286, 821–830 (2001).
12. Bhandari, M. et al. Hierarchy of evidence: differences in results between non-randomized studies and randomized trials in patients with femoral neck fractures. Arch. Orthop. Trauma Surg. 124, 10–16 (2004).
13. Dahabreh, I.J. et al. Do observational studies using propensity score methods agree with randomized trials? A systematic comparison of studies on acute coronary syndromes. Eur. Heart J. 33, 1893–1901 (2012).
14. Lonjon, G. et al. Comparison of treatment effect estimates from prospective nonrandomized studies with propensity score analysis and randomized controlled trials of surgical procedures. Ann. Surg. 259, 18–25 (2014).
15. Hemkens, L.G., Contopoulos-Ioannidis, D.G. & Ioannidis, J.P. Agreement of treatment effects for mortality from routinely collected data and subsequent randomized trials: meta-epidemiological survey. BMJ 352, i493 (2016).
16. Ioannidis, J.P. Why most published research findings are false. PLoS Med. 2, e124 (2005).
17. Suissa, S., Ernst, P., Vandemheen, K.L. & Aaron, S.D. Methodological issues in therapeutic trials of COPD. Eur. Respir. J. 31, 927–933 (2008).
18. Stampfer, M.J. et al. Postmenopausal estrogen therapy and cardiovascular disease: ten-year follow-up from the Nurses' Health Study. N. Engl. J. Med. 325, 756–762 (1991).
19. Rossouw, J.E. et al. Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Women's Health Initiative randomized controlled trial. JAMA 288, 321–333 (2002).
20. Hernán, M.A. et al. Observational studies analyzed like randomized experiments: an application to postmenopausal hormone therapy and coronary heart disease. Epidemiology 19, 766–779 (2008).
21. Goodman, S.N., Schneeweiss, S. & Baiocchi, M. Using design thinking to differentiate useful from misleading evidence in observational research. JAMA 317, 705–707 (2017).
22. Schneeweiss, S., Seeger, J.D., Landon, J. & Walker, A.M. Aprotinin during coronary-artery bypass grafting and risk of death. N. Engl. J. Med. 358, 771–783 (2008).
23. Fergusson, D.A. et al. A comparison of aprotinin and lysine analogues in high-risk cardiac surgery. N. Engl. J. Med. 358, 2319–2331 (2008).
24. Patorno, E. et al. Cardiovascular safety of glucagon-like peptide-1 receptor agonists versus other antidiabetic agents in routine care: a cohort study. Diabetes Obes. Metab. (2016). [Epub ahead of print]
25. Neal, B. et al. Canagliflozin and cardiovascular and renal events in type 2 diabetes. N. Engl. J. Med. June 2017. doi:10.1056/NEJMoa1611925.
26. Strom, B.L., Miettinen, O.S. & Melmon, K.L. Post-marketing studies of drug efficacy: how? Am. J. Med. 77, 703–708 (1984).
27. Strom, B.L., Melmon, K.L. & Miettinen, O.S. Post-marketing studies of drug efficacy: why? Am. J. Med. 78, 475–480 (1985).
28. Schneeweiss, S. et al. Real world data in adaptive biomedical innovation: a framework for generating evidence fit for decision-making. Clin. Pharmacol. Ther. 100, 633–646 (2016).
29. Matthews, K.A., Kuller, L.H., Wing, R.R., Meilahn, E.N. & Plantinga, P. Prior to use of estrogen replacement therapy, are users healthier than nonusers? Am. J. Epidemiol. 143, 971–978 (1996).
30. Glynn, R.J., Knight, E.L., Levin, R. & Avorn, J. Paradoxical relations of drug treatment with mortality in older persons. Epidemiology 12, 682–689 (2001).
31. Glynn, R., Schneeweiss, S., Wang, P., Levin, R. & Avorn, J. Selective prescribing led to overestimation of the benefits of lipid-lowering drugs. J. Clin. Epidemiol. 59, 819–828 (2006).
32. Setoguchi, S. et al. Influence of healthy candidate bias in assessing clinical effectiveness for implantable cardioverter-defibrillators: cohort study of older patients with heart failure. BMJ 348, g2866 (2014).
33. Patorno, E. et al. Observational studies of the association between glucose-lowering medications and cardiovascular outcomes: addressing methodological limitations. Diabetologia 57, 2237–2250 (2014).
34. Patorno, E. et al. The impact of study design and analysis on “measurable” and “un-measurable” patient characteristics in observational studies of glucose-lowering drugs in type 2 diabetes mellitus [abstract]. Pharmacoepidemiol Drug Saf. 25, 44–45 (2016).
35. Schneeweiss, S. A basic study design for expedited safety signal evaluation based on electronic healthcare data. Pharmacoepidemiol. Drug Saf. 19, 858–868 (2010).
36. Kelsey, J.L., Whittemore, A.S., Evans, A.S. & Thompson, W.D. Methods in Observational Epidemiology. 2nd ed. (Oxford University Press, New York, NY, 1996).
37. Ray, W.A., Stein, C.M., Daugherty, J.R., Hall, K., Arbogast, P.G. & Griffin, M.R. COX-2 selective non-steroidal anti-inflammatory drugs and risk of serious coronary heart disease. Lancet 360, 1071–1073 (2002).
38. Solomon, D.H. et al. Relationship between selective cyclooxygenase-2 inhibitors and acute myocardial infarction in older adults. Circulation 109, 2068–2073 (2004).
39. Kiyota, Y., Schneeweiss, S., Glynn, R.J., Cannuscio, C.C., Avorn, J. & Solomon, D.H. Accuracy of Medicare claims-based diagnosis of acute myocardial infarction: estimating positive predictive value on the basis of review of hospital records. Am. Heart J. 148, 99–104 (2004).
40. Lin, K. & Schneeweiss, S. Considerations for the analysis of longitudinal electronic health records linked to claims data to study the effectiveness and safety of drugs. Clin. Pharmacol. Ther. 100, 147–159 (2016).
41. Franklin, J.M., Schneeweiss, S. & Solomon, D.H. Assessment of confounders in comparative effectiveness studies from secondary databases. Am. J. Epidemiol. 185, 474–478 (2017).
42. Steiner, J.F. & Prochazka, A.V. The assessment of refill compliance using pharmacy records: methods, validity, and applications. J. Clin. Epidemiol. 50, 105–116 (1997).
43. Ho, P.M., Bryson, C.L. & Rumsfeld, J.S. Medication adherence. Circulation 119, 3028–3035 (2009).
44. Fischer, M.A. et al. Primary medication non-adherence: analysis of 195,930 electronic prescriptions. J. Gen. Intern. Med. 25, 284–290 (2010).
45. Tamblyn, R., Eguale, T., Huang, A., Winslade, N. & Doran, P. The incidence and determinants of primary nonadherence with prescribed medication in primary care. Ann. Intern. Med. 160, 441–450 (2014).
46. Schneeweiss, S., Rassen, J.A., Glynn, R.J., Avorn, J., Mogun, H. & Brookhart, M.A. High-dimensional propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology 20, 512–522 (2009).
47. Eng, P., Seeger, J.D., Loughlin, J., Clifford, C.R., Mentor, B.A. & Walker, A.M. Supplementary data collection with case - cohort analysis to address potential confounding in a cohort study of thromboembolism in oral contraceptive initiators matched on claims - based propensity scores. Pharmacoepidemiol. Drug Saf. 17, 297–305 (2008).
48. Gruber, S. et al. Design and analysis choices for safety surveillance evaluations need to be tuned to the specifics of the hypothesized drug-outcome association. Pharmacoepidemiol. Drug Saf. 25, 973–981 (2016).
49. Schisterman, E.F., Cole, S.R. & Platt, R.W. Overadjustment bias and unnecessary adjustment in epidemiologic studies. Epidemiology 20, 488–495 (2009).
50. Suissa, S. Immortal time bias in observational studies of drug effects. Pharmacoepidemiol. Drug Saf. 16, 241–249 (2007).
51. Suissa, S. Immortal time bias in pharmacoepidemiology. Am. J. Epidemiol. 167, 492–499 (2008).
52. Hernán, M.A., Sauer, B.C., Hernández-Díaz, S., Platt, R. & Shrier, I. Specifying a target trial prevents immortal time bias and other self-inflicted injuries in observational analyses. J. Clin. Epidemiol. 79, 70–75 (2016).
53. Huitfeldt, A., Hernan, M.A., Kalager, M. & Robins, J.M. Comparative effectiveness research using observational data: active comparators to emulate target trials with inactive comparators. EGEMS (Wash DC) 4, 1234 (2016).
54. Ray, W.A. Evaluating medication effects outside of clinical trials: new-user designs. Am. J. Epidemiol. 158, 915–920 (2003).
55. Johnson, E.S. et al. The incident user design in comparative effectiveness research. Pharmacoepidemiol. Drug Saf. 22, 1–6 (2013).
56. McMahon, A.D. Approaches to combat with confounding by indication in observational studies of intended drug effects. Pharmacoepidemiol. Drug Saf. 12, 551–558 (2003).
57. Rosenbaum, P.R. & Rubin, D.B. The central role of the propensity score in observational studies for causal effects. Biometrika 70, 41–55 (1983).
58. Glynn, R.J., Schneeweiss, S. & Stürmer, T. Indications for propensity scores and review of their use in pharmacoepidemiology. Basic Clin. Pharmacol. Toxicol. 98, 253–259 (2006).
59. Franklin, J.M., Eddings, W., Austin, P.C., Stuart, E.A. & Schneeweiss, S. Comparing the performance of propensity score methods in healthcare database studies with rare outcomes. Stat. Med. 36, 1946–1963 (2017).
60. Franklin, J.M., Rassen, J.A., Ackermann, D., Bartels, D.B. & Schneeweiss, S. Metrics for covariate balance in cohort studies of causal effects. Stat. Med. 33, 1685–1699 (2014).
61. Franklin, J.M., Rassen, J.A., Bartels, D.B. & Schneeweiss, S. Prospective cohort studies of newly marketed medications: using covariate data to inform the design of large-scale studies. Epidemiology 25, 126–133 (2014).
62. Angrist, J.D., Imbens, G.W. & Rubin, D.B. Identification of causal effects using instrumental variables. J. Am. Stat. Assoc. 91, 444–455 (1996).
63. Brookhart, M.A., Wang, P.S., Solomon, D.H. & Schneeweiss, S. Evaluating short-term drug effects using a physician-specific prescribing preference as an instrumental variable. Epidemiology 17, 268–275 (2006).
64. Brookhart, M.A. & Schneeweiss, S. Preference-based instrumental variable methods for the estimation of treatment effects: assessing validity and interpreting results. Int. J. Biostat. 3, Article 14 (2007).
65. Huybrechts, K.F., Gerhard, T., Franklin, J.M., Levin, R., Crystal, S. & Schneeweiss, S. Instrumental variable applications using nursing home prescribing preferences in comparative effectiveness research. Pharmacoepidemiol. Drug Saf. 23, 830–838 (2014).
66. Piantadosi, S. Clinical Trials: A Methodologic Perspective (John Wiley & Sons, Hoboken, NJ, 2013).
67. Schneeweiss, S. Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiol. Drug Saf. 15, 291–303 (2006).
68. Tchetgen Tchetgen, E. The control outcome calibration approach for causal inference with unobserved confounding. Am. J. Epidemiol. 179, 633–640 (2014).
69. Bateman, B.T. et al. Statins and congenital malformations: cohort study. BMJ 350, h1035 (2015).
70. von Elm, E., Altman, D., Egger, M., Pocock, S., Gøtzsche, P. & Vandenbroucke, J. Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet 370, 1453–1457 (2007).
71. Wang, S.V., Verpillat, P., Rassen, J.A., Patrick, A., Garry, E.M. & Bartels, D.B. Transparency and reproducibility of observational cohort studies using large healthcare databases. Clin. Pharmacol. Ther. 99, 325–332 (2016).
72. Wang, S.V., Schneeweiss, S. & Joint ISPE-ISPOR Special Task Force on RWE in Healthcare Decision-making. Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies v2.0.; 2017. . Accessed 22 June 2017.
73. U.S. Food and Drug Administration. Postmarket Requirements and Commitments. . Accessed 22 June 2017.
74. Gurwitz, J.H., Col, N.F. & Avorn, J. The exclusion of the elderly and women from clinical trials in acute myocardial infarction. JAMA J. Am. Med. Assoc. 268, 1417–1422 (1992).
75. Franklin, J.M., Dejene, S., Huybrechts, K.F., Wang, S.V. & Rothman, K.J. A bias in the evaluation of bias comparing randomized trials with nonexperimental studies. Epidemiol. Methods (2017). [Epub ahead of print]
76. Tannen, R.L., Weiner, M.G. & Xie, D. Use of primary care electronic medical record database in drug efficacy research on cardiovascular outcomes: comparison of database and randomised controlled trial findings. BMJ 338, b81 (2009).
77. Danaei, G., Rodriguez, L.A., Cantero, O.F., Logan, R. & Hernán, M.A. Observational data for comparative effectiveness research: an emulation of randomised trials of statins and primary prevention of coronary heart disease. Stat. Methods Med. Res. 22, 70–96 (2013).
78. Deeks, J.J. et al. Evaluating non-randomised intervention studies. Health Technol. Assess. 7, 1–173 (2003).
79. Schünemann, H., Brożek, J., Guyatt, G. & Oxman, A. GRADE Handbook: Handbook for Grading the Quality of Evidence and the Strength of Recommendations Using the GRADE Approach. (2016).