Reporting of covariate selection and balance assessment in propensity score analysis is suboptimal: A systematic review

Journal of Clinical Epidemiology

Volumn 68, Issue 2, 2015, Pages 122-131

  Ali, M Sanni ^a,b   Groenwold, Rolf H H ^a,b   Belitser, Svetlana V ^a   Pestman, Wiebe R ^c   Hoes, Arno W ^b   Roes, Kit C B ^b   Boer, Anthonius De ^a   Klungel, Olaf H ^a,b  

^a UTRECHT UNIVERSITY   (Netherlands)

^b UNIVERSITY MEDICAL CENTER UTRECHT   (Netherlands)

^c UNIVERSITY OF LEUVEN   (Belgium)

Author keywords

Balance; Confounding; Pharmacoepidemiology; Propensity score; Reporting; Variable selection

Indexed keywords

ALGORITHM; CONFOUNDING VARIABLE; COVARIANCE; HUMAN; HYPOTHESIS; LOGISTIC REGRESSION ANALYSIS; MEDICAL LITERATURE; MEDICAL RESEARCH; PRIORITY JOURNAL; PROBABILITY; PROPENSITY SCORE; REVIEW; SENSITIVITY ANALYSIS; STATISTICAL ANALYSIS; STATISTICAL MODEL; STATISTICAL SIGNIFICANCE; SYSTEMATIC REVIEW; TREATMENT OUTCOME; VALIDITY; ANALYSIS OF VARIANCE; METHODOLOGY; PATIENT SELECTION; SELECTION BIAS;

ANALYSIS OF VARIANCE; DATA INTERPRETATION, STATISTICAL; HUMANS; PATIENT SELECTION; PROPENSITY SCORE; RESEARCH DESIGN; SELECTION BIAS;

EID: 84920572572     PISSN: 08954356     EISSN: 18785921     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2014.08.011     Document Type: Review

References (60)

1. P.R. Rosenbaum, and D.B. Rubin The central role of the propensity score in observational studies for causal effects Biometrika 70 1983 41 55
2. P.R. Rosenbaum, and D.B. Rubin Reducing bias in observational studies using subclassification on the propensity score J Am Stat Assoc 79 1984 516 524
3. M.A. Brookhart, S. Schneeweiss, K.J. Rothman, R.J. Glynn, J. Avorn, and T. Stürmer Variable selection for propensity score models Am J Epidemiol 163 2006 1149
4. A.R. Patrick, S. Schneeweiss, M.A. Brookhart, R.J. Glynn, K.J. Rothman, and J. Avorn The implications of propensity score variable selection strategies in pharmacoepidemiology: an empirical illustration Pharmacoepidemiol Drug Saf 20 2011 551 559
5. J. Hill Discussion of research using propensity-score matching: comments on 'a critical appraisal of propensity-score matching in the medical literature between 1996 and 2003' by Peter Austin, Statistics in Medicine Stat Med 27 2008 2055 2061
6. M. Lunt Selecting an appropriate caliper can be essential for achieving good balance with propensity score matching Am J Epidemiol 179 2014 226 235
7. M.S. Ali, R.H. Groenwold, and O.H. Klungel Propensity score methods and unobserved covariate imbalance: comments on "squeezing the balloon" Health Serv Res 49 2014 1074 1082
8. P.C. Austin A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003 Stat Med 27 2008 2037 2049
9. S. Weitzen, K.L. Lapane, A.Y. Toledano, A.L. Hume, and V. Mor Principles for modeling propensity scores in medical research: a systematic literature review Pharmacoepidemiol Drug Saf 13 2004 841 853
10. B.R. Shah, A. Laupacis, J.E. Hux, and P.C. Austin Propensity score methods gave similar results to traditional regression modeling in observational studies: a systematic review J Clin Epidemiol 58 2005 550 559
11. P.C. Austin Propensity-score matching in the cardiovascular surgery literature from 2004 to 2006: a systematic review and suggestions for improvement J Thorac Cardiovasc Surg 134 2007 1128 1135
12. F. D'ascenzo, E. Cavallero, G. Biondi-Zoccai, C. Moretti, P. Omedè, and M. Bollati Use and misuse of multivariable approaches in interventional cardiology studies on drug-eluting stents: a systematic review J Interv Cardiol 25 2012 611 621
13. M.A. Brookhart, T. Stürmer, R.J. Glynn, J. Rassen, and S. Schneeweiss Confounding control in healthcare database research: challenges and potential approaches Med Care 48 2010 S114
14. Pearl J. On a class of bias-Amplifying variables that endanger effect estimates. In: Grünwald P, Spirtes P, Eds. Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence (UAI 2010). 2010; Corvallis, OR: Association for Uncertainty in Artificial Intelligence; 201: 425-432.
15. J.A. Myers, J.A. Rassen, J.J. Gagne, K.F. Huybrechts, S. Schneeweiss, and K.J. Rothman Effects of adjusting for instrumental variables on bias and precision of effect estimates Am J Epidemiol 174 2011 1213 1222
16. J. Pearl Invited commentary: understanding bias amplification Am J Epidemiol 174 2011 1223 1227
17. P.C. Austin Goodness-of-fit diagnostics for the propensity score model when estimating treatment effects using covariate adjustment with the propensity score Pharmacoepidemiol Drug Saf 17 2008 1202 1217
18. P.C. Austin Assessing balance in measured baseline covariates when using many-to-one matching on the propensity-score Pharmacoepidemiol Drug Saf 17 2008 1218 1225
19. S.V. Belitser, E.P. Martens, W.R. Pestman, R.H.H. Groenwold, A. Boer, and O.H. Klungel Measuring balance and model selection in propensity score methods Pharmacoepidemiol Drug Saf 20 2011 1115 1129
20. R.H.H. Groenwold, F. Vries, A. Boer, W.R. Pestman, F.H. Rutten, and A.W. Hoes Balance measures for propensity score methods: a clinical example on beta-Agonist use and the risk of myocardial infarction Pharmacoepidemiol Drug Saf 20 2011 1130 1137
21. M.S. Ali, R.H. Groenwold, W.R. Pestman, S.V. Belitser, K.C. Roes, and A.W. Hoes Propensity score balance measures in pharmacoepidemiology: a simulation study Pharmacoepidemiol Drug Saf 23 2014 802 811
22. S. Weitzen, K.L. Lapane, A.Y. Toledano, A.L. Hume, and V. Mor Weaknesses of goodness-of-fit tests for evaluating propensity score models: the case of the omitted confounder Pharmacoepidemiol Drug Saf 14 2005 227 238
23. P.C. Austin Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples Stat Med 28 2009 3083 3107
24. D. Westreich, S.R. Cole, M.J. Funk, M.A. Brookhart, and T. Stürmer The role of the c-statistic in variable selection for propensity score models Pharmacoepidemiol Drug Saf 20 2011 317 320
25. M.E. Falagas, V.D. Kouranos, R. Arencibia-Jorge, and D.E. Karageorgopoulos Comparison of SCImago journal rank indicator with journal impact factor FASEB J 22 2008 2623 2628
26. B. Gonzalez-Pereira, V.P. Guerrero-Bote, and F. Moya-Anegón A new approach to the metric of journals' scientific prestige: the SJR indicator J Informetr 4 2010 379 391
27. L. Bornmann, W. Marx, A.Y. Gasparyan, and G.D. Kitas Diversity, value and limitations of the journal impact factor and alternative metrics Rheumatol Int 32 2012 1861 1867
28. D.E. Ho, K. Imai, G. King, and E.A. Stuart Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference Polit Anal 15 2007 199 236
29. A. Ash, and M. Shwartz R2: a useful measure of model performance when predicting a dichotomous outcome Stat Med 18 1999 375 384
30. J.A. Hanley, and B.J. McNeil A method of comparing the areas under receiver operating characteristic curves derived from the same cases Radiology 148 1983 839 843
31. D.W. Hosmer Jr., and S. Lemeshow Applied logistic regression 2nd ed. 2004 John Wiley & Sons New York, NY
32. C. Drake Effects of misspecification of the propensity score on estimators of treatment effect Biometrics 49 1993 1231 1236
33. B.W. Silverman Density estimation for statistics and data analysis 1986 Chapman & Hall/CRC London, UK
34. M.A. Stephens Use of the Kolmogorov-Smirnov, Cramér-Von Mises and related statistics without extensive tables J R Stat Soc Ser B Meth 32 1970 115 122
35. W.R. Pestman Mathematical statistics: an introduction 2nd ed. 1998 Walter De Gruyter Inc Berlin, Germany
36. J.L. Fleiss, B. Levin, and M.C. Paik Statistical methods for rates and proportions 2nd ed. 2013 John Wiley & Sons Hoboken, NJ
37. J. Hartung, and G. Knapp Statistical inference in adaptive group sequential trials with the standardized mean difference as effect size Sequential Anal 30 2011 94 113
38. J. Cohen Statistical power analysis for the behavioral sciences 2nd ed. 1988 Lawrence Erlbaum Associates Publishers Hillsdale, NJ
39. E.P. Martens, W.R. Pestman, A. De Boer, S.V. Belitser, and O.H. Klungel Systematic differences in treatment effect estimates between propensity score methods and logistic regression Int J Epidemiol 37 2008 1142 1147
40. R.J. Glynn, S. Schneeweiss, and T. Stürmer Indications for propensity scores and review of their use in pharmacoepidemiology Basic Clin Pharmacol Toxicol 98 2006 253 259
41. J.A. Myers, J.A. Rassen, J.J. Gagne, K.F. Huybrechts, S. Schneeweiss, and K.J. Rothman Myers et al. Respond to "understanding bias amplification" Am J Epidemiol 174 2011 1228 1229
42. E.A. Stuart Developing practical recommendations for the use of propensity scores: discussion of 'a critical appraisal of propensity score matching in the medical literature between 1996 and 2003' by Peter Austin, Statistics in Medicine Stat Med 27 2008 2062 2065
43. E. von Elm, D.G. Altman, M. Egger, S.J. Pocock, P.C. Gøtzsche, and J.P. Vandenbroucke The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies Prev Med 45 2007 247 251
44. ENCePP Guide on Methodological Standards in Pharmacoepidemiology. EMA/95098/2010. Available at www.encepp.eu/standards-And-guidances. Accessed June 22, 2013
45. K.M. Mortimer, R. Neugebauer, M. Van Der Laan, and I.B. Tager An application of model-fitting procedures for marginal structural models Am J Epidemiol 162 2005 382 388
46. D. Westreich, J. Lessler, and M.J. Funk Propensity score estimation: neural networks, support vector machines, decision trees (CART), and meta-classifiers as alternatives to logistic regression J Clin Epidemiol 63 2010 826 833
47. B.K. Lee, J. Lessler, and E.A. Stuart Weight trimming and propensity score weighting PLoS One 6 2011 e18174
48. S. Setoguchi, S. Schneeweiss, M.A. Brookhart, R.J. Glynn, and E.F. Cook Evaluating uses of data mining techniques in propensity score estimation: a simulation study Pharmacoepidemiol Drug Saf 17 2008 546 555
49. M.S. Ali, R.H. Groenwold, W.R. Pestman, S.V. Belitser, A.W. Hoes, and A. de Boer Time-dependent propensity score and collider-stratification bias: an example of beta2-Agonist use and the risk of coronary heart disease Eur J Epidemiol 28 2013 291 299
50. K. Imai, G. King, and E.A. Stuart Misunderstandings between experimentalists and observationalists about causal inference J R Stat Soc Ser A Stat Soc 171 2008 481 502
51. J.M. Franklin, J.A. Rassen, D. Ackermann, D.B. Bartels, and S. Schneeweiss Metrics for covariate balance in cohort studies of causal effects Stat Med 33 2014 1685 1699
52. M.Á. Hernán, B. Brumback, and J.M. Robins Marginal structural models to estimate the causal effect of zidovudine on the survival of HIV-positive men Epidemiology 11 2000 561 570
53. J.M. Robins, M.Á. Hernán, and B. Brumback Marginal structural models and causal inference in epidemiology Epidemiology 11 2000 550 560
54. M.A. Hernán, S. Hernández-Díaz, and J.M. Robins A structural approach to selection bias Epidemiology 15 2004 615 625
55. T. Kurth, A.M. Walker, R.J. Glynn, K.A. Chan, J.M. Gaziano, and K. Berger Results of multivariable logistic regression, propensity matching, propensity adjustment, and propensity-based weighting under conditions of nonuniform effect Am J Epidemiol 163 2006 262 270
56. S. Greenland, and J. Pearl Adjustments and their consequences-collapsibility analysis using graphical models Int Stat Rev 79 2010 401 426
57. P.C. Austin, P. Grootendorst, and G.M. Anderson A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study Stat Med 26 2007 734 753
58. J.A. Rassen, A.A. Shelat, J. Myers, R.J. Glynn, K.J. Rothman, and S. Schneeweiss One-to-many propensity score matching in cohort studies Pharmacoepidemiol Drug Saf 21 2012 69 80
59. P.C. Austin Optimal caliper widths for propensity-score matching when estimating differences in means and differences in proportions in observational studies Pharm Stat 10 2011 150 161
60. P.C. Austin A comparison of 12 algorithms for matching on the propensity score Stat Med 33 2013 1057 1069