Evaluating treatment effectiveness under model misspecification: A comparison of targeted maximum likelihood estimation with bias-corrected matching

Statistical Methods in Medical Research

Volumn 25, Issue 5, 2016, Pages 2315-2336

  Kreif, Noémi ^a   Gruber, Susan ^b   Radice, Rosalba ^c,d   Grieve, Richard ^a   Sekhon, Jasjeet S ^c  

^a LONDON SCHOOL OF HYGIENE AND TROPICAL MEDICINE   (United Kingdom)

^b HARVARD SCHOOL OF PUBLIC HEALTH   (United States)

^c UNIVERSITY OF LONDON   (United Kingdom)

^d UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

bias corrected matching; double robustness; machine learning; model misspecification; targeted maximum likelihood estimation; treatment effectiveness

Indexed keywords

ARTICLE; CASE STUDY; CLINICAL EFFECTIVENESS; CONFIDENCE INTERVAL; EVALUATION STUDY; HUMAN; INTERMETHOD COMPARISON; MACHINE LEARNING; MATHEMATICAL VARIABLE; MAXIMUM LIKELIHOOD METHOD; NONLINEAR SYSTEM; OSTEOARTHRITIS; PATIENT-REPORTED OUTCOME; PREDICTION; PROPENSITY SCORE; QUALITY OF LIFE; REGRESSION ANALYSIS; SIMULATION; STATISTICAL BIAS; STATISTICAL MODEL; TOTAL HIP PROSTHESIS; TREATMENT OUTCOME; VARIANCE; AGED; COMPARATIVE STUDY; COMPUTER SIMULATION; HIP PROSTHESIS; MALE; STATISTICAL ANALYSIS;

AGED; BIAS; COMPUTER SIMULATION; CONFIDENCE INTERVALS; DATA INTERPRETATION, STATISTICAL; HIP PROSTHESIS; HUMANS; LIKELIHOOD FUNCTIONS; MACHINE LEARNING; MALE; MODELS, STATISTICAL; OSTEOARTHRITIS; QUALITY OF LIFE; TREATMENT OUTCOME;

EID: 84989888849     PISSN: 09622802     EISSN: 14770334     Source Type: Journal    
DOI: 10.1177/0962280214521341     Document Type: Article

References (84)

1. Rubin DB,. On the limitations of comparative effectiveness research. Stat Med 2010; 29: 1991-1995.
2. Tunis SR, Benner J, McClellan M,. Comparative effectiveness research: Policy context, methods development and research infrastructure. Stat Med 2010; 29: 1963-1976.
3. Fung V, Brand RJ, Newhouse JP, et al. Using Medicare data for comparative effectiveness research: Opportunities and challenges. Am J Manag Care 2011; 17: 489-496.
4. Imbens GM, Wooldridge JM,. Recent developments in the econometrics of program evaluation. J Econ Lit 2009; 47: 5-86.
5. Ho DE, Imai K, King G, et al. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis 2007; 15: 199-236.
6. Basu A, Manca A,. Regression estimators for generic health-related quality of life and quality-adjusted life years. Med Decis Making. Epub ahead of print 18 October 2011. DOI: 10.1177/0272989X11416988.
7. Dehejia RH, Wahba S,. Propensity score-matching methods for nonexperimental causal studies. Rev Econ Stat 2002; 84: 151-161.
8. Westreich D, Lessler J, Funk M,. Propensity score estimation: Neural networks, support vector machines, decision trees (CART), and meta-classifiers as alternatives to logistic regression. J Clin Epidemiol 2010; 63: 826-833.
9. Robins J, Rotnitzky A, Zhao LP,. Estimation of regression coefficients when some regressors are not always observed. J Am Stat Assoc 1994; 89: 846-866.
10. Robins J, Sued M, Lei-Gomez Q, et al. Comment: Performance of double-robust estimators when "inverse probability" weights are highly variable. Stat Sci 2007; 22: 544-559.
11. Westreich D, Cole SR,. Invited commentary: Positivity in practice. Am J Epidemiol 2010; 171: 674-677.
12. Kang JDY, Schafer JL,. Demystifying double robustness: A comparison of alternative strategies for estimating a population mean from incomplete data. Stat Sci 2007; 22: 523-539.
13. Freedman D, Berk RA,. Weighting regression by propensity score. Eval Rev 2008; 32: 392-409.
14. van der Laan MJ,. Targeted maximum likelihood based causal inference: Part I. Int J Biostat 2010; 6: 1-42. DOI: 10.2202/1557-4679.1211-1-42. DOI: 10.2202/1557-4679.1211.
15. van der Laan MJ, Rubin D,. Targeted maximum likelihood learning. U.C. Berkeley Division of Biostatistics Working Paper Series. Working Paper 213, October 2006, http://biostats.bepress.com/ucbbiostat/paper213.
16. Porter KE, Gruber S, van der Laan MJ, et al. The relative performance of targeted maximum likelihood estimators. Int J Biostat 2011; 7: 1-34. DOI: 10.2202/1557-4679.1308-1-34. DOI: 10.2202/1557-4679.1308.
17. Stitelman OM, van der Laan MJ,. Collaborative targeted maximum likelihood for time to event data. Int J Biostat 2010; 6: 1-46-1-46.
18. Gruber S, van der Laan M,. A targeted maximum likelihood estimator of a causal effect on a bounded continuous outcome. Int J Biostat 2010; 6: Article 26. DOI: 10.2202/1557-4679.1260-Article 26. DOI: 10.2202/1557-4679.1260.
19. Rubin DB,. The use of matched sampling and regression adjustment to remove bias in observational studies. Biometrics 1973; 29: 185-203.
20. Abadie A, Imbens GW,. Bias-corrected matching estimators for average treatment effects. J Bus Econ Stat 2011; 29: 1-11.
21. Busso M, DiNardo J and McCrary J. New evidence on the finite sample properties of propensity score reweighting and matching estimators, 2011. Contract No. 3998.
22. Rubin DB, Thomas N,. Combining propensity score matching with additional adjustments for prognostic covariates. J Am Stat Assoc 2000; 95: 573-585.
23. Rubin DB,. Using multivariate matched sampling and regression adjustment to control bias in observational studies. J Am Stat Assoc 1979; 74: 318-328.
24. Lee BK, Lessler J, Stuart EA,. Improving propensity score weighting using machine learning. Stat Med 2010; 29: 337-346.
25. Setoguchi S, Schneeweiss S, Brookhart M, et al. Evaluating uses of data mining techniques in propensity score estimation: A simulation study. Pharmacoepidemiol Drug Saf 2008; 17: 546-555.
26. Austin PC,. Using ensemble-based methods for directly estimating causal effects: An investigation of tree-based G-computation. Multivar Behav Res 2012; 47: 115-135.
27. Ridgeway G, McCaffrey DF,. Comment: Demystifying double robustness: A comparison of alternative strategies for estimating a population mean from incomplete data. Stat Sci 2007; 22: 540-543.
28. Austin PC,. A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003. Stat Med 2008; 27: 2037-2049.
29. Caliendo M, Kopeinig S,. Some practical guidance for the implementation of propensity score matching. J Econ Surv 2008; 22: 31-72.
30. PROMs. Provisional Monthly Patient Reported Outcome Measures (PROMs) in England. April 2009-April 2010: Pre- and post-operative data: Experimental statistics. The Health and Social Care Information Centre, 2010.
31. Ousey K, Cook L,. Understanding patient reported outcome measures (PROMs). Br J Commun Nurs 2011; 16: 80-82.
32. Greenland S, Pearl J, Robins JM,. Confounding and collapsibility in causal inference. Stat Sci 1999; 14: 29-46.
33. Cole SR, Frangakis CE,. The consistency statement in causal inference: A definition or an assumption? Epidemiology 2009; 20: 3-5.
34. Imbens GW,. Nonparametric estimation of average treatment effects under exogeneity: A review. Rev Econ Stat 2004; 86: 4-29.
35. Robins J,. A new approach to causal inference in mortality studies with a sustained exposure period-application to control of the healthy worker survivor effect. Math Modelling 1986; 7: 1393-1512.
36. Imbens G and Wooldridge JM. New developments in econometrics. Lecture Notes, CEMMAP, UCL2009b.
37. Rubin DB,. Estimating causal effects from large data sets using propensity scores. Ann Intern Med 1997; 127: 757-763.
38. van der Laan MJ, Polley EC, Hubbard AE,. Super learner. Stat Appl Genet Mol Biol 2007; 6:. DOI: 10.2202/1544-6115.1309 -. DOI: 10.2202/1544-6115.1309.
39. Polley EC and van der Laan MJ. Super Learner in prediction. U.C. Berkeley Division of Biostatistics Working Paper Series. Working Paper 266, May 2010, http://biostats.bepress.com/ucbbiostat/paper266.
40. van der Laan MJ and Dudoit S. Unified cross-validation methodology for selection among estimators and a general cross-validated adaptive epsilon-net estimator: Finite sample oracle inequalities and examples. U.C. Berkeley Division of Biostatistics Working Paper Series. Working Paper 130, November 2003, http://biostats.bepress.com/ucbbiostat/paper130.
41. Rosenbaum PR, Rubin DB,. The central role of the propensity score in observational studies for causal effects. Biometrika 1983; 70: 41-55.
42. Hirano K, Imbens GW,. Estimation of causal effects using propensity score weighting: An application to data on right heart catheterization. Health Serv Outcomes Res Methodol 2001; 2: 259-278.
43. Waernbaum I,. Model misspecification and robustness in causal inference: Comparing matching with doubly robust estimation. Stat Med 2011; 31: 1572-1581.
44. Abadie A, Imbens GW,. Large sample properties of matching estimators for average treatment effects. Econometrica 2006; 74: 235-267.
45. Hirano K, Imbens GW, Ridder G,. Efficient estimation of average treatment effects using the estimated propensity score. Econometrica 2003; 71: 1161-1189.
46. Radice R, Grieve R, Ramsahai R, et al. Evaluating treatment effectiveness in patient subgroups: A comparison of propensity score methods with an automated matching approach. Int J Biostat 2012; 8: 1-43-1-43.
47. Cole SR, Hernán MA,. Constructing inverse probability weights for marginal structural models. Am J Epidemiol 2008; 168: 656-664.
48. Elliott MR,. Model averaging methods for weight trimming. J Off Stat 2008; 24: 517-540.
49. Crump RK, Hotz VJ, Imbens GW, et al. Dealing with limited overlap in estimation of average treatment effects. Biometrika 2009; 96: 187-199.
50. McCaffrey D, Ridgeway G, Morral A,. Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychol Methods 2004; 9: 403-425.
51. Stuart EA,. Matching methods for causal inference: A review and a look forward. Stat Sci 2010; 25: 1-144-1-144.
52. Bang H, Robins JM,. Doubly robust estimation in missing data and causal inference models. Biometrics 2005; 61: 962-972.
53. Kang JDY, Schafer JL,. Rejoinder: Demystifying double robustness: A comparison of alternative strategies for estimating a population mean from incomplete data. Stat Sci 2007; 22: 574-580.
54. Basu A, Polsky D, Manning W,. Estimating treatment effects on healthcare costs under exogeneity: Is there a 'magic bullet'? Health Serv Outcomes Res Methodol 2011; 11: 1-26.
55. van der Laan MJ, Gruber S,. Collaborative double robust targeted maximum likelihood estimation. Int J Biostat 2010; 6: 1-68. DOI: 10.2202/1557-4679.1181-1-68. DOI: 10.2202/1557-4679.1181.
56. Gudex C, Kind P and Williams A. A social tariff for EuroQol: results from a UK general population survey. Centre for Health Economics, University of York, 1995, pp. 1-24.
57. Rotnitzky A, Lei Q, Sued M, et al. Improved double-robust estimation in missing data and causal inference models. Biometrika 2012; 99: 439-456.
58. Gruber S, van der Laan MJ,. Targeted minimum loss based estimation of causal effects on an outcome with known conditional bounds. Int J Biostat 2012. 8. DOI: 10.1515/1557-4679.1370.
59. Moore KL, van der Laan MJ,. Covariate adjustment in randomized trials with binary outcomes: Targeted maximum likelihood estimation. Stat Med 2009; 28: 39-64.
60. Hampel FR,. The influence curve and its role in robust estimation. J Am Stat Assoc 1974; 69: 383-393.
61. Gruber S, van der Laan MJ,. tmle: An R package for targeted maximum likelihood estimation. J Stat Software 2012; 51: 1-35.
62. Abadie A, Herr JL, Imbens GW, et al. NNMATCH: Stata module to compute nearest-neighbor bias-corrected estimators. Chestnut Hill, MA, USA: Boston College Department of Economics, 2004, http://fmwww.bc.edu/repec/bocode/n/nnmatch.hlp.
63. Austin PC,. Statistical criteria for selecting the optimal number of untreated subjects matched to each treated subject when using many-to-one matching on the propensity score. Am J Epidemiol 2010; 172: 1092-1097.
64. NICE. Health Technology Appraisal. Total hip replacement and resurfacing arthroplasty for the treatment of pain or disability resulting from end stage arthritis of the hip (Review of technology appraisal guidance 2 and 44). Final scope, 2012, http://www.nice.org.uk/nicemedia/live/13690/61348/61348.pdf (accessed 31 July 2012).
65. EuroQol Group. EuroQol-a new facility for the measurement of health-related quality of life. Health Policy 1990; 16: 199-208.
66. Pennington M, Grieve R, Sekhon JS, et al. Cemented, cementless and hybrid prostheses for total hip replacement: A cost-effectiveness analysis. Br Med J 2013. 346: f1026. DOI: 10.1136/bmj.f1026.
67. Carpenter J, Kenward M,. Multiple imputation and its application, John Wiley & Sons, 2012.
68. Rubin D,. Multiple imputation after 18+ years. J Am Stat Assoc 1996; 91: 473-489.
69. Tooze JA, Grunwald GK, Jones RH,. Analysis of repeated measures data with clumping at zero. Stat Methods Med Res 2002; 11: 341-355.
70. Buntin MB, Zaslavsky AM,. Too much ado about two-part models and transformation?: Comparing methods of modeling Medicare expenditures. J Health Econ 2004; 23: 525-542.
71. R Development Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, 2011.
72. Polley EC and van der Laan MJ. R package "SuperLearner" 2010, http://cran.r-project.org/web/packages/SuperLearner/index.html (accessed 15 June 2012).
73. Kooperberg C. Polspline: Polynomial spline routines, 2010 CRAN, Software Package, http://cran.r-project.org/web/packages/polspline/ (accessed 15 June 2012).
74. Ridgeway G, McCaffrey D, Morral A, et al. twang: Toolkit for weighting and analysis of nonequivalent groups. CRAN, Software Package, http://cran.r-project.org/web/packages/twang/index.html.
75. Davison A, Hinkley D,. Bootstrap methods and their application, New York: Cambridge University Press, 1997.
76. Rubin DB,. The design versus the analysis of observational studies for causal effects: Parallels with the design of randomized trials. Stat Med 2007; 26: 20-36.
77. Sauerbrei W, Royston P, Binder H,. Selection of important variables and determination of functional form for continuous predictors in multivariable model building. Stat Med 2007; 26: 5512-5528.
78. Fan J, Li R,. Variable selection via nonconcave penalized likelihood and its oracle properties. J Am Stat Assoc 2001; 96: 1348-1360.
79. Austin PC,. The performance of different propensity score methods for estimating marginal odds ratios. Stat Med 2013; 32: 2837-2849.
80. Brookhart MA, Schneeweiss S, Rothman KJ, et al. Variable selection for propensity score models. Am J Epidemiol 2006; 163: 1149-1156.
81. Senn S, Graf E, Caputo A,. Stratification for the propensity score compared with linear regression techniques to assess the effect of treatment or exposure. Stat Med 2007; 26: 5529-5544.
82. Guo H and Dawid AP (eds). Sufficient covariates and linear propensity analysis. In: International conference on artificial intelligence and statistics, 2010, pp. 281-288.
83. Gruber S, van der Laan MJ,. An application of collaborative targeted maximum likelihood estimation in causal inference and genomics. Int J Biostat 2010; 6: 1-29. DOI: 10.2202/1557-4679.1182-1-29. DOI: 10.2202/1557-4679.1182.
84. Sekhon JS,. Matching: Multivariate and propensity score matching with automated balance search. J Stat Software 2011; 42: 1-52-1-52.