Applying Propensity Score Methods in Medical Research: Pitfalls and Prospects

Medical Care Research and Review

Volumn 67, Issue 5, 2010, Pages 528-554

  Bradley, Cathy J ^b   Gardiner, Joseph C ^a  

^a MICHIGAN STATE UNIVERSITY   (United States)

^b VIRGINIA COMMONWEALTH UNIVERSITY   (United States)

Author keywords

evaluation methods; observational study; propensity score matching; treatment effect

Indexed keywords

ARTICLE; EPIDEMIOLOGY; HUMAN; MEDICAL RESEARCH; METHODOLOGY; PATIENT SELECTION; PROPENSITY SCORE; RANDOMIZED CONTROLLED TRIAL (TOPIC); REPRODUCIBILITY; STANDARD; STATISTICAL ANALYSIS; STATISTICS; TREATMENT OUTCOME;

BIAS (EPIDEMIOLOGY); BIOMEDICAL RESEARCH; DATA INTERPRETATION, STATISTICAL; HUMANS; PATIENT SELECTION; PROPENSITY SCORE; RANDOMIZED CONTROLLED TRIALS AS TOPIC; REPRODUCIBILITY OF RESULTS; TREATMENT OUTCOME;

EID: 79952118952     PISSN: 10775587     EISSN: None     Source Type: Journal    
DOI: 10.1177/1077558710361486     Document Type: Article

References (81)

1. Abadie, A., Drukker, D., Leber Herr, J., & Imbens, G.W. (2004). Implementing matching estimators for average treatment effects in Stata. Stata Journal, 4, 290-311.
2. Abidov, A., Rozanski, A., Hachamovitch, R., Hayes, S.W., Aboul-Enein, F., Cohen, I., et al. (2005). Prognostic significance of dyspnea in patients referred for cardiac stress testing. New England Journal of Medicine, 353, 1889-1898.
3. Agresti, A. (1990). Categorical data analysis. New York: Wiley.
4. Angrist, J.D., & Pischke, J. (2009). Mostly harmless econometrics: An empiricist’s companion. Princeton, NJ: Princeton University Press.
5. Aronow, H.D., Topol, E.J., Roe, M.T., Houghtaling, P.L., Wolski, K.E., Lincoff, A.M., et al. (2001). Effect of lipid-lowering therapy on early mortality after acute coronary syndromes: An observational study. Lancet, 357, 1063-1068.
6. Austin, P.C. (2008a). A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003. Statistics in Medicine, 27, 2037-2049.
7. Austin, P.C. (2008b). The performance of different propensity-score methods for estimating relative risks. Journal of Clinical Epidemiology, 61, 537-545.
8. Austin, P.C., Grootendorst, P., & Anderson, G.M. (2007). A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: A Monte Carlo study. Statistics in Medicine, 26, 734-753.
9. Austin, P.C., Grootendorst, P., Normand, S.L., & Anderson, G.M. (2007). Conditioning on the propensity score can result in biased estimation of common measures of treatment effect: A Monte Carlo study. Statistics in Medicine, 26, 754-768.
10. Ayanian, J.Z., Landrum, M.B., Guadagnoli, E., & Gaccione, P. (2002). Specialty of ambulatory care physicians and mortality among elderly patients after myocardial infarction. New England Journal of Medicine, 347, 1678-1686.
11. Babaev, A., Frederick, P.D., Pasta, D.J., Every, N., Sichrovsky, T., & Hochman, J.S. (2005). Trends in management and outcomes of patients with acute myocardial infarction complicated by cardiogenic shock. Journal of the American Medical Association, 294, 448-454.
12. Baser, O. (2006). Too much ado about propensity score models? Comparing methods of propensity score matching. Value in Health, 9, 377-385.
13. Baser, O. (2009). Too much ado about instrumental variable approach: Is the cure worse than the disease? Value in Health, 12, 1201-1209.
14. Becker, S.O., & Caliendo, M. (2007). Sensitivity analysis for average treatment effects. Stata Journal, 7, 71-83.
15. Becker, S.O., & Ichino, A. (2002). Estimation of average treatment effects based on propensity scores. Stata Journal, 2, 358-377.
16. Bhatt, D.L., Roe, M.T., Peterson, E.D., Li, Y., Chen, A.Y., Harrington, R.A., et al. (2004). Utilization of early invasive management strategies for high-risk patients with non-ST-segment elevation acute coronary syndromes: Results from the CRUSADE quality improvement initiative. Journal of the American Medical Association, 292, 2096-2104.
17. Bound, J., Jaeger, D.A., & Baker, R.M. (1995). Problems with instrumental variables estimation when the correlation between the instruments and the endogenous explanatory variable is weak. Journal of the American Statistical Association, 90, 443-450.
18. Bradley, C.J., Neumark, D., Luo, Z., Bednarek, H., & Schenk, M. (2005). Employment outcomes of men treated for prostate cancer. Journal of National Cancer Institute, 97, 958-965.
19. Cochran, W.G. (1968). The effectiveness of adjustment by subclassification in removing bias in observational studies. Biometrics, 24, 295-313.
20. Cook, T.D., & Campbell, D.T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Chicago: Rand McNally.
21. D’Agostino, R.B. (1998). Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Statistics in Medicine, 17, 2265-2281.
22. DiPrete, T., & Gangl, M. (2004). Assessing bias in the estimation of causal effects: Rosenbaum bounds on matching estimators and instrumental variables estimation with imperfect instruments. Sociological Methodology, 34, 271-310.
23. Ferguson, T.B., Coombs, L.P., & Peterson, E.D. (2002). Preoperative beta-blocker use and mortality and morbidity following CABG surgery in North America. Journal of the American Medical Association, 287, 2221-2227.
24. Frolkis, J.P., Pothier, C.E., Blackstone, E.H., & Lauer, M.S. (2003). Frequent ventricular ectopy after exercise as a predictor of death. New England Journal of Medicine, 348, 781-790.
25. Gardiner, J.C., Luo, Z.H., & Roman, L.A. (2009). Fixed effects, random effects and GEE: What are the differences? Statistics in Medicine, 28, 221-239.
26. Girou, E., Brun-Buisson, C., Taille, S., Lemaire, F., & Brochard, L. (2003). Secular trends in nosocomial infections and mortality associated with noninvasive ventilation in patients with exacerbation of COPD and pulmonary edema. Journal of the American Medical Association, 290, 2985-2991.
27. Greenland, S. (2000). An introduction to instrumental variables for epidemiologists. International Journal of Epidemiology, 29, 722-729.
28. Greenland, S., Robins, J.M., & Pearl, J. (1999). Confounding and collapsibility in causal inference. Statistical Science, 14, 29-46.
29. Grzybowski, M., Clements, E.A., Parsons, L., Welch, R., Tintinalli, A.T., Ross, M.A., et al. (2003). Mortality benefit of immediate revascularization of acute ST-segment elevation myocardial infarction in patients with contraindications to thrombolytic therapy: A propensity analysis. Journal of the American Medical Association, 290, 1891-1898.
30. Gu, X.S., & Rosenbaum, P.R. (1993). Comparison of multivariate matching methods: Structures, distances, and algorithms. Computational and Graphical Statistics, 2, 405-420.
31. Gum, P.A., Thamilarasan, M., Watanabe, J., Blackstone, E.H., & Lauer, M.S. (2001). Aspirin use and all-cause mortality among patients being evaluated for known or suspected coronary artery disease: A propensity analysis. Journal of the American Medical Association, 286, 1187-1194.
32. Hansen, B.B. (2008). The essential role of balance tests in propensity-matched observational studies: Comments on “A Critical Appraisal of Propensity-Score Matching in the Medical Literature Between 1996 and 2003” by Peter Austin, Statistics in Medicine. Statistics in Medicine, 27, 2050-2054, 2066-2069.
33. Hannan, E.L., Racz, M.J., Walford, G., Jones, R.H., Ryan, T.J., Bennett, E., et al. (2005). Long-term outcomes of coronary-artery bypass grafting versus stent implantation. New England Journal of Medicine, 352, 2174-2183.
34. Heckman, J.J. (2005a). Rejoinder: Response to Sobel. Sociological Methodology, 35, 135-162.
35. Heckman, J.J. (2005b). The scientific model of causality. Sociological Methodology, 35, 1-97.
36. Heckman, J.J., Ichimura, H., Smith, J., & Todd, P. (1998). Characterizing selection bias using experimental data. Econometrica, 66, 1017-1098.
37. Heckman, J.J., Ichimura, H., & Todd, P.E. (1997). Matching as an econometric evaluation estimator: Evidence from evaluating a job training program. Review of Economic Studies, 64, 605-654.
38. Heckman, J.J., & Navarro-Lozano, S. (2004). Using matching, instrumental variables, and control functions to estimate economic choice models. Review of Economics and Statistics, 86, 30-57.
39. Heckman, J.J., & Smith, J.A. (1995). Assessing the case for social experiments. Journal of Economic Perspectives, 9, 85-110.
40. Heckman, J.J., & Vytlacil, E.J. (2007). Econometric evaluation of social programs, part II: Using the marginal treatment effect to organize alternative economic estimators to evaluate social programs and to forecast their effects in new environments. In J. J. Heckman & E. Leamer (Eds.), Handbook of econometrics (Vol. 6B, chap. 71) Amsterdam: Elsevier.
41. Hsueh, E.C., Essner, R., Foshag, L.J., Ye, X., Wang, H.J., & Morton, D.L. (2004). Prolonged survival after complete resection of metastases from intraocular melanoma. Cancer, 100, 122-129.
42. Kravitz, R.L., Duan, N., & Braslow, J. (2004). Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages. Milbank Quarterly, 82, 661-687.
43. Ichino, A., Mealli, F., & Nannicini, T. (2008). From temporary help jobs to permanent employment: What can we learn from matching estimators and their sensitivity? Journal of Applied Econometrics, 23, 305-327.
44. Imbens, G.W. (2004). Nonparametric estimation of average treatment effects under exogeneity: A review. Review of Economics and Statistics, 86, 4-29.
45. Imbens, G.W., & Angrist, J.D. (1994). Identification and estimation of local average treatment effects. Econometrica, 62, 467-475.
46. Imbens, G.W., & Wooldridge, J.M. (2009). Recent developments in the econometrics of program evaluation. Journal of Economic Literature, 47, 5-86.
47. Landrum, M.B., & Ayanian, J.Z. (2001). Causal effect of ambulatory specialty care on mortality following myocardial infarction: A comparison of propensity score and instrumental variable analyses. Health Services and Outcomes Research Methodology, 2, 221-245.
48. Leuven, E., & Sianesi, B. (2003). PSMATCH2: Stata module to perform full Mahalanobis and propensity score matching, common support graphing, and covariate imbalance testing. Chestnut Hill, MA: Boston College, Department of Economics.
49. Lin, D.Y., Psaty, B.M., & Kronmal, R.A. (1998). Assessing the sensitivity of regression results to unmeasured confounders in observational studies. Biometrics, 54, 948-963.
50. Lindenauer, P.K., Pekow, P., Wang, K.J., Mamidi, D.K., Gutierrez, B., & Benjamin, E.M. (2005). Perioperative beta-blocker therapy and mortality after major noncardiac surgery. New England Journal of Medicine, 353, 349-361.
51. Manski, C.F. (1990). Nonparametric bounds on treatment effects. American Economic Review, 80, 319-323.
52. Mehta, R.L., Pascual, M.T., Soroko, S., & Chertow, G.M. (2002). Diuretics, mortality, and nonrecovery of renal function in acute renal. Journal of the American Medical Association, 288, 2547-2553.
53. Mitra, N., Schnabel, F.R., Neugut, A.I., & Heitjan, D.F. (2001). Estimating the effect of an intensive surveillance program on stage of breast carcinoma at diagnosis: A propensity score analysis. Cancer, 91, 1709-1715.
54. Mukamal, K.J., Maclure, M., Muller, J.E., Sherwood, J.B., & Mittleman, M.A. (2001). Prior alcohol consumption and mortality following acute myocardial infarction. Journal of the American Medical Association, 285, 1965-1970.
55. Nannicini, T. (2007). Simulation-based sensitivity analysis for matching estimators. Stata Journal, 7, 334-3350.
56. Newby, L.K., Kristinsson, A., Bhapkar, M.V., Aylward, P.E., Dimas, A.P., Klein, W.W., et al. (2002). Early statin initiation and outcomes in patients with acute coronary syndromes. Journal of the American Medical Association, 287, 3087-3095.
57. Orosz, G.M., Magaziner, J., Hannan, E.L., Morrison, R.S., Koval, K., Gilbert, M., et al. (2004). Association of timing of surgery for hip fracture and patient outcomes. Journal of the American Medical Association, 291, 1738-1743.
58. Pearl, J. (2009). Causality: Models, reasoning, and inference. New York: Cambridge University Press.
59. Petersen, L.A., Normand, S.T., Daley, J., & McNeil, B.J. (2000). Outcome of myocardial infarction in Veterans Health Administration patients as compared with Medicare patients. New England Journal of Medicine, 343, 1934-1941.
60. Polanczyk, C.A., Rohde, L.E., Goldman, L., Cook, E.F., Thomas, E.J., Marcantonio, E.R., et al. (2001). Right heart catheterization and cardiac complications in patients undergoing noncardiac surgery: An observational study. Journal of the American Medical Association, 286, 309-314.
61. Rao, S.V., Jollis, J.G., Harrington, R.A., Granger, C.B., Newby, L.K., Armstrong, P.W., et al. (2004). Relationship of blood transfusion and clinical outcomes in patients with acute coronary syndromes. Journal of the American Medical Association, 292, 1555-1562.
62. Rosenbaum, P.R. (1987). Sensitivity analysis for certain permutation inferences in matched observational studies. Biometrika, 74, 13-26.
63. Rosenbaum, P.R. (2002). Observational Studies. New York: Springer.
64. Rosenbaum, P.R., & Rubin, D.B. (1983a). Assessing sensitivity to an unobserved binary covariate in an observational study with binary outcome. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 45, 212-218.
65. Rosenbaum, P.R., & Rubin, D.B. (1983b). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41-55.
66. Rosenbaum, P.R., & Rubin, D.B. (1984). Reducing bias in observational studies using subclassification on the propensity score. Journal of the American Statistical Association, 79, 516-524.
67. Rubin, D.B. (2004). On principles for modeling propensity scores in medical research. Pharmacoepidemiology and Drug Safety, 13, 855-857.
68. Rubin, D.B. (2006). Matched sampling for causal effects. New York: Cambridge University Press.
69. Schneeweiss, S., Walker, A.M., Glynn, R.J., Maclure, M., Dormuth, C., & Soumerai, S.B. (2002). Outcomes of reference pricing for angiotensin-converting-enzyme inhibitors. New England Journal of Medicine, 346, 822-829.
70. Smith, J.A., & Todd, P.E. (2005). Does matching overcome LaLonde’s critique of nonexperimental estimators? Journal of Econometrics, 125, 305-353.
71. Sobel, M.E. (2005). Discussion: the scientific model of causality. Sociological Methodology, 35, 99-133.
72. Staiger, D., & Stock, J.H. (1997). Instrumental variables regression with weak instruments. Econometrica, 65, 557-586.
73. Stenestrand, U., & Wallentin, L. (2002). Early revascularisation and 1-year survival in 14-day survivors of acute myocardial infarction: A prospective cohort study. Lancet, 359, 1805-1811.
74. Stürmer, T., Joshi, M., Glynn, R.J., Avorn, J., Rothman, K.J., & Schneeweiss, S. (2006). A review of the applications of propensity score methods yielded increasing use, advantages in specific settings, but not substantially different estimates compared with conventional multivariable methods. Journal of Clinical Epidemiology, 59, 437-447.
75. Vikram, H.R., Buenconsejo, J., Hasbun, R., & Quagliarello, V.J. (2003). Impact of valve surgery on 6-month mortality in adults with complicated, left-sided native valve endocarditis: A propensity analysis. Journal of the American Medical Association, 290, 3207-3214.
76. Vincent, J.L., Baron, J.F., Reinhart, K., Gattinoni, L., Thijs, L., Webb, A., et al. (2002). Anemia and blood transfusion in critically ill patients. Journal of the American Medical Association, 288, 1499-1507.
77. Wang, P.S., Schneeweiss, S., Avorn, J., Fischer, M.A., Mogun, H., Solomon, D.H., et al. (2005). Risk of death in elderly users of conventional vs. atypical antipsychotic medications. New England Journal of Medicine, 353, 2335-2341.
78. Wassertheil-Smoller, S., Psaty, B., Greenland, P., Oberman, A., Kotchen, T., Mouton, C., et al. (2004). Association between cardiovascular outcomes and antihypertensive drug treatment in older women. Journal of the American Medical Association, 292, 2849-2859.
79. Weitzen, S., Lapane, K.L., Toledano, A.Y., Hume, A.L., & Mor, V. (2004). Principles for modeling propensity scores in medical research: A systematic literature review. Pharmacoepidemiology & Drug Safety, 13, 841-853.
80. Wooldridge, J.M. (2002). Econometric analysis of cross section and panel data. Cambridge: MIT Press.
81. Wooldridge, J.M. (2005). Violating ignorability of treatment by controlling for too many factors. Econometric Theory, 21, 1026-1028.