Applications of propensity score methods in observational comparative effectiveness and safety research: Where have we come and where should we go?

Journal of Comparative Effectiveness Research

Volumn 3, Issue 1, 2014, Pages 63-78

  Borah, Bijan J ^a,b   Moriarty, James P ^b   Crown, William H ^c   Doshi, Jalpa A ^d  

^a MAYO CLINIC COLLEGE OF MEDICINE   (United States)

^b MAYO CLINIC   (United States)

^c OPTUM   (United States)

^d UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

comparative effectiveness research; general and internal medicine; high impact; impact factors; observational studies; policy implications; propensity score methods

Indexed keywords

AMOXICILLIN; ATYPICAL ANTIPSYCHOTIC AGENT; AZITHROMYCIN; CIPROFLOXACIN; CLOPIDOGREL; LEVOFLOXACIN; NEUROLEPTIC AGENT; PRASUGREL; PROTON PUMP INHIBITOR;

BARE METAL STENT; CLINICAL RESEARCH; COMPARATIVE EFFECTIVENESS; COMPARATIVE STUDY; DRUG ELUTING STENT; DRUG SAFETY; HUMAN; INTERMETHOD COMPARISON; MEDICAID; MINIMALLY INVASIVE SURGERY; OBSERVATIONAL STUDY; PRACTICE GUIDELINE; PRIORITY JOURNAL; PROPENSITY SCORE; PROSTATECTOMY; PSYCHOSIS; REVIEW; SAFETY; TREATMENT RESPONSE;

CARDIOLOGY; COMPARATIVE EFFECTIVENESS RESEARCH; HUMANS; MODELS, STATISTICAL; PROPENSITY SCORE; PSYCHIATRY; RESEARCH DESIGN;

EID: 84890915927     PISSN: 20426305     EISSN: 20426313     Source Type: Journal    
DOI: 10.2217/cer.13.89     Document Type: Review

References (99)

1. Garber AM, Sox HC. The role of costs in comparative effectiveness research. Health Aff. (Milwood) 29(10), 1805-1811 (2010).
2. D'Agostino RB Jr, D'Agostino RB Sr. Estimating treatment effects using observational data. JAMA 297(3), 314-316 (2007).
3. Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J. Chronic Dis. 20(8), 637-648 (1967).
4. Tunis SR, Stryer DB, Clancy CM. Practical clinical trials -increasing the value of clinical research for decision making in clinical and health policy. JAMA 290(12), 1624-1632 (2003).
5. Vandenbroucke JP. When are observational studies as credible as randomised trials? Lancet 363(9422), 1728-1731 (2004).
6. Williams RJ, Tse T, Harlan WR, Zarin DA. Registration of observational studies: is it time? CMAJ 182(15), 1638-1642 (2010).
7. Dahabreh IJ, Sheldrick RC, Paulus JK et al. Do observational studies using propensity score methods agree with randomized trials? A systematic comparison of studies on acute coronary syndromes. Eur. Heart J. 33(15), 1893-1901 (2012).
8. Benson K, Hartz AJ. A comparison of observational studies and randomized, controlled trials. N. Engl. J. Med. 342(25), 1878-1886 (2000).
9. Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N. Engl. J. Med. 342(25), 1887-1892 (2000).
10. Kuss O, Legler T, Bogermann J. Treatments effects from randomized trials and propensity score analyses were similar in similar populations in an example from cardiac surgery. J. Clin. Epidemiol. 64(10), 1076-1084 (2011).
11. McKee M, Britton A, Black N, McPherson K, Sanderson C, Bain C. Methods in health services research. Interpreting the evidence: choosing between randomised and non-randomised studies. BMJ 319(7205), 312-315 (1999).
12. Albanes D. Beta-carotene and lung cancer: a case study. Am. J. Clin. Nutr. 69(6), 1345S-1350S (1999).
13. Grodstein F, Clarkson TB, Manson JE. Understanding the divergent data on postmenopausal hormone therapy. N. Engl. J. Med. 348(7), 645-650 (2003).
14. Austin PC. A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003. Stat. Med. 27(12), 2037-2049 (2008).
15. Austin PC, Mamdani MM. A comparison of propensity score methods: a case-study estimating the effectiveness of post-AMI statin use. Stat. Med. 25(12), 2084-2106 (2006).
16. Glynn RJ, Schneeweiss S, Sturmer T. Indications for propensity scores and review of their use in pharmacoepidemiology. Basic Clin. Pharmacol. Toxicol. 98(3), 253-259 (2006).
17. Shah BR, Laupacis A, Hux JE, Austin PC. Propensity score methods gave similar results to traditional regression modeling in observational studies: a systematic review. J. Clin. Epidemiol. 58(6), 550-559 (2005).
18. Stürmer T, Joshi M, Glynn RJ, Avorn J, Rothman KJ, Schneeweiss S. A review of the application of propensity score methods yielded increasing use, advantages in specific settings, but not substantially different estimates compared with conventional multivariable methods. J. Clin. Epidemiol. 59(5), 437-447 (2006).
19. Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika 70(1), 41-55 (1983).
20. Dehejia RH, Wahba S. Propensity score-matching methods for nonexperimental causal studies. Rev. Econom. Stat. 84(1), 151-161 (2002).
21. D'Agostino RB Jr. Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Stat. Med. 17(19), 2265-2281 (1998).
22. Rosenbaum PR, Rubin DB. Reducing bias in observational studies using subclassification on the propensity score. J. Am. Stat. Assoc. 79(387), 516-524 (1984).
23. Rubin DB, Thomas N. Combining propensity score matching with additional adjustments for prognostic covariates. J. Am. Stat. Assoc. 95(450), 573-585 (2000).
24. Hernan MA, Brumback B, Robins JM. Marginal structural models to estimate the causal effect of zidovudine on the survival of HIV-positive men. Epidemiology 11(5), 561-570 (2000).
25. Rosenbaum PR. Model-based direct adjustment. J. Am. Stat. Assoc. 82(398), 387-394 (1987).
26. Bang H, Robins JM. Doubly robust estimation in missing data and causal inference models. Biometrics 61(4), 962-972 (2005).
27. Schneeweiss S, Rassen JA, Glynn RJ, Avorn J, Mogun H, Brookhart MA. High-dimensional propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology 20(4), 512-522 (2009).
28. Austin PC. Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Stat. Med. 28(25), 3083-3107 (2009).
29. Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behav. Res. 46(3), 399-424 (2011).
30. Brookhart MA, Schneeweiss S, Rothman KJ, Glynn RJ, Avorn J, Stürmer T. Variable selection for propensity score models. Am. J. Epidemiol. 163(12), 1149-1156 (2006).
31. Stürmer T, Schneeweiss S, Brookhart MA, Rothman KJ, Avorn J, Glynn RJ. Analytic strategies to adjust confounding using exposure propensity scores and disease risk scores: nonsteroidal antiinflammatory drugs and short-term mortality in the elderly. Am. J. Epidemiol. 161(9), 891-898 (2005).
32. Weitzen S, Lapane KL, Toledano AY, Hume AL, Mor V. Principles for modeling propensity scores in medical research: a systematic literature review. Pharmacoepidemiol. Drug Saf. 13(12), 841-853 (2004).
33. Braitman LE, Rosenbaum PR. Rare outcomes, common treatments: analytic strategies using propensity scores. Ann. Intern. Med. 137(8), 693-695 (2002).
34. Cepeda MS, Boston R, Farrar JT, Strom BL. Comparison of logistic regression versus propensity score when the number of events is low and there are multiple confounders. Am. J. Epidemiol. 158(3), 280-287 (2003).
35. Drake C. Effects of misspecification of the propensity score on estimators of treatment effect. Biometrics 49(4), 1231-1236 (1993).
36. Rubin DB. Estimating the causal effects of smoking. Stat. Med. 20(9-10), 1395-1414 (2001).
37. Rubin DB. The design versus the analysis of observational studies for causal effects: parallels with the design of randomized trials. Stat. Med. 26(1), 20-36 (2007).
38. Stürmer T, Schneeweiss S, Avorn J, Glynn RJ. Adjusting effect estimates for unmeasured confounding with validation data using propensity score calibration. Am. J. Epidemiol. 162(3), 279-289 (2005).
39. Eng PM, Seeger JD, Loughlin J, Clifford CR, Mentor S, Walker AM. Supplementary data collection with case-cohort analysis to address potential confounding in a cohort study of thromboembolism in oral contraceptive initiators matched on claims-based propensity scores. Pharmacoepidemiol. Drug Saf. 17(3), 297-305 (2008).
40. Wang PS, Schneeweiss S, Avorn J et al. Risk of death in elderly users of conventional vs. atypical antipsychotic medications. N. Engl. J. Med. 353(22), 2335-2341 (2005).
41. Lagerqvist B, James SK, Stenestrand U et al. Long-term outcomes with drug-eluting stents versus bare-metal stents in Sweden. N. Engl. J. Med. 356(10), 1009-1019 (2007).
42. Eisenstein EL, Anstrom KJ, Kong DF et al. Clopidogrel use and long-term clinical outcomes after drug-eluting stent implantation. JAMA 297(2), 159-168 (2007).
43. O'Donoghue ML, Braunwald E, Antman EM et al. Pharmacodynamic effect and clinical efficacy of clopidogrel and prasugrel with or without a proton-pump inhibitor: an analysis of two randomised trials. Lancet 374(9694), 989-997 (2009).
44. Hu JC, Gu X, Lipsitz SR et al. Comparative effectiveness of minimally invasive vs open radical prostatectomy. JAMA 302(14), 1557-1564 (2009).
45. Ray WA, Murray KT, Hall K, Arbogast PG, Stein CM. Azithromycin and the risk of cardiovascular death. N. Engl. J. Med. 366(20), 1881-1890 (2012).
46. Gill SS, Bronskill SE, Normand SL et al. Antipsychotic drug use and mortality in older adults with dementia. Ann. Intern. Med. 146(11), 775-786 (2007).
47. Schneeweiss S, Setoguchi S, Brookhart A, Dormuth C, Wang PS. Risk of death associated with the use of conventional versus atypical antipsychotic drugs among elderly patients. CMAJ 176(5), 627-632 (2007).
48. Stefanini GG, Holmes DR Jr. Drug-eluting coronary-artery stents. N. Engl. J. Med. 368(3), 254-265 (2013).
49. Ong AT, Hoye A, Aoki J et al. Thirty-day incidence and six-month clinical outcome of thrombotic stent occlusion after bare-metal, sirolimus, or paclitaxel stent implantation. J. Am. Coll. Cardiol. 45(6), 947-953 (2005).
50. Pfisterer M, Brunner-La Rocca HP, Buser PT et al. Late clinical events after clopidogrel discontinuation may limit the benefit of drug-eluting stents: an observational study of drug-eluting versus bare-metal stents. J. Am. Coll. Cardiol. 48(12), 2584-2591 (2006).
51. Anstrom KJ, Tsiatis AA. Utilizing propensity scores to estimate causal treatment effects with censored time-lagged data. Biometrics 57(4), 1207-1218 (2001).
52. Bang H, Tsiatis AA. Estimating medical costs with censored data. Biometrika 87(2), 329-343 (2000).
53. Wiviott SD, Trenk D, Frelinger AL et al.; PRINCIPLE-TIMI 44 investigators. Prasugrel compared with high loading-and maintenance-dose clopidogrel in patients with planned percutaneous coronary intervention: the Prasugrel in Comparison to Clopidogrel for Inhibition of Platelet Activation and Aggregation-Thrombolysis in Myocardial Infarction 44 trial. Circulation 116(25), 2923-2932 (2007).
54. Wiviott SD, Braunwald E, McCabe CH et al. Prasugrel versus clopidogrel in patients with acute coronary syndromes. N. Engl. J. Med. 357(20), 2001-2015 (2007).
55. Robins JM, Hernan MA, Brumback B. Marginal structural models and causal inference in epidemiology. Epidemiology 11(5), 550-560 (2000).
56. Rubin DB. Estimating causal effects from large data sets using propensity scores. Ann. Intern. Med. 127(8 Pt 2), 757-763 (1997).
57. Zeger SL, Liang KY. Longitudinal data-analysis for discrete and continuous outcomes. Biometrics 42(1), 121-130 (1986).
58. Rothman KJ, Greenland S. Modern Epidemiology (2nd Edition). Lippincott-Raven, PA, USA (1998).
59. Rosenbaum PR, Rubin DB. Constructing a control-group using multivariate matched sampling methods that incorporate the propensity score. Am. Statistician 39(1), 33-38 (1985).
60. Austin PC, Grootendorst P, Anderson GM. A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study. Stat. Med. 26(4), 734-753 (2007).
61. Baser O. Too much ado about propensity score models? Comparing methods of propensity score matching. Value Health 9(6), 377-385 (2006).
62. Efron B, Tibshirani RJ. An Introduction to Bootstrap. Chapman and Hall, NY, USA (1993).
63. Austin PC. The performance of different propensity score methods for estimating marginal odds ratios. Stat. Med. 26(16), 3078-3094 (2007).
64. Austin PC. The performance of different propensity-score methods for estimating relative risks. J. Clin. Epidemiol. 61(6), 537-545 (2008).
65. Austin PC. The relative ability of different propensity score methods to balance measured covariates between treated and untreated subjects in observational studies. Med. Decis. Making 29(6), 661-677 (2009).
66. Austin PC. Some methods of propensity-score matching had superior performance to others: results of an empirical investigation and Monte Carlo simulations. Biom. J. 51(1), 171-184 (2009).
67. Austin PC. Different measures of treatment effect for different research questions. J. Clin. Epidemiol. 63(1), 9-10 (2010).
68. Austin PC. The performance of different propensity score methods for estimating marginal hazard ratios. Stat. Med. 32(16), 2837-2849 (2013).
69. Diprete TA, Gangl M. Assessing bias in the estimation of causal effects: Rosenbaum bounds on matching estimators and instrumental variables estimation with imperfect instruments. Sociol. Methodol. 34(1), 271-310 (2004).
70. Rosenbaum PR. Observational Studies (2nd Edition). Springer, NY, USA (2002).
71. Schneeweiss S. Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiol. Drug Saf. 15(5), 291-303 (2006).
72. Becker SO, Caliendo M. Sensitivity analysis for average treatment effects. Stata J. 7(1), 71-83 (2007).
73. Collins GS, Le Manach Y. Comparing treatment effects between propensity scores and randomized controlled trials: improving conduct and reporting. Eur. Heart J. 33(15), 1867-1869 (2012).
74. Sox HC. Defining comparative effectiveness research: the importance of getting it right. Med. Care 48(6 Suppl.), S7-S8 (2010).
75. Barry MJ, Mulley AG Jr, Fowler FJ, Wennberg JW. Watchful waiting vs immediate transurethral resection for symptomatic prostatism. The importance of patients' preferences. JAMA 259(20), 3010-3017 (1988).
76. Brown JS, Holmes JH, Shah K, Hall K, Lazarus R, Platt R. Distributed health data networks: a practical and preferred approach to multi-institutional evaluations of comparative effectiveness, safety, and quality of care. Med. Care 48(6 Suppl.), S45-S51 (2010).
77. Libby AM, Pace W, Bryan C et al. Comparative effectiveness research in DARTNet primary care practices: point of care data collection on hypoglycemia and over-the-counter and herbal use among patients diagnosed with diabetes. Med. Care 48(6 Suppl.), S39-S44 (2010).
78. Jacobs JP, Edwards FH, Shahian DM et al. Successful linking of the Society of Thoracic Surgeons adult cardiac surgery database to Centers for Medicare and Medicaid Services Medicare data. Ann. Thorac. Surg. 90(4), 1150-1156; discussion 1156-1157 (2010).
79. Institute of Medicine (IOM). Learning What Works: Observational Studies in a Learning Health System: Workshop Summary. The National Academies Press, Washington, DC, USA (2013).
80. Institute of Medicine (IOM). Learning What Works: Infrastructure Required for Comparative Effectiveness Research. The National Academies Press, Washington, DC, USA (2011).
81. Rassen JA, Solomon DH, Curtis JR, Herrinton L, Schneeweiss S. Privacy-maintaining propensity score-based pooling of multiple databases applied to a study of biologics. Med. Care 48(6 Suppl.), S83-S89 (2010).
82. Brookhart MA, Rassen JA, Schneeweiss S. Instrumental variable methods in comparative safety and effectiveness research. Pharmacoepidemiol. Drug Saf. 19(6), 537-554 (2010).
83. Terza JV, Basu A, Rathouz PJ. Two-stage residual inclusion estimation: addressing endogeneity in health econometric modeling. J. Health Econ. 27(3), 531-543 (2008).
84. Angrist JD, Imbens GW, Rubin DB. Identification of causal effects using instrumental variables. J. Am. Stat. Assoc. 91(434), 444-455 (1996).
85. Davies NM, Smith GD, Windmeijer F, Martin RM. Issues in the reporting and conduct of instrumental variable studies: a systematic review. Epidemiology 24(3), 363-369 (2013).
86. Stukel TA, Fisher ES, Wennberg DE, Alter DA, Gottlieb DJ, Vermeulen MJ. Analysis of observational studies in the presence of treatment selection bias: effects of invasive cardiac management on AMI survival using propensity score and instrumental variable methods. JAMA 297(3), 278-285 (2007).
87. Venkitachalam L, Lei Y, Magnuson EA et al. Survival benefit with drug-eluting stents in observational studies: fact or artifact? Circ. Cardiovasc. Qual. Outcomes 4(6), 587-594 (2011).
88. Kuehn BM. FDA warns antipsychotic drugs may be risky for elderly. JAMA 293(20), 2462 (2005).
89. US Government. The Patient Protection and Affordable Care Act, Public Law 111-148, 111th Congress (2010). www.gpo.gov/fdsys/pkg/PLAW-111publ148/ pdf/PLAW-111publ148.pdf
90. PubMed homepage. www.ncbi.nlm.nih.gov/pubmed
91. Scopus homepage. www.scopus.com
92. Thomson Reuters. Web of Science -Journal Citation Reports. http://wokinfo.com/products-tools/ analytical/jcr
93. US FDA. Information for healthcare professionals: conventional antipsychotics (2008). www.fda.gov/drugs/ afetyafetyinformationforpatientsandproviders/ ucm124830.htm
94. US FDA. FDA drug safety communication: Azithromycin (Zithromax or Zmax) and the risk of potentially fatal heart rhythms (2013). www.fda.gov/drugs/ drugsafety/ucm341822. htm
95. Keele L. An overview of rbounds: an R package for Rosenbaum bounds sensitivity analysis with matched data (2010). www.personal.psu.edu/ljk20/ rbounds%20 vignette.pdf
96. Developing a Protocol for Observational Comparative Effectiveness Research. www.effectivehealthcare.ahrq.gov/ehc/ products/440/1166/User-Guide-to- Observational-CER-1-10-13.pdf
97. International Society for Pharmacoeconomics and Outcomes Research. Outcomes research guidelines index. www.ispor.org/guidelinesindex
98. European Network of Centres for Pharmacoepidemiology and Pharmacovigilance Standards and Guidances. www.encepp.eu/standards-and- guidances/ index.shtml
99. National Cancer Institute (NCI). SEER-Medicare Linked Database. http://appliedresearch.cancer.gov/seermedicare