Estimating causal effects: considering three alternatives to difference-in-differences estimation

Health Services and Outcomes Research Methodology

Volumn 16, Issue 1-2, 2016, Pages 1-21

  O’Neill, Stephen ^a   Kreif, Noémi ^a   Grieve, Richard ^a   Sutton, Matthew ^b   Sekhon, Jasjeet S ^c  

^a LONDON SCHOOL OF HYGIENE AND TROPICAL MEDICINE   (United Kingdom)

^b UNIVERSITY OF MANCHESTER   (United Kingdom)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Difference in differences; Matching; Pay for performance; Policy evaluation; Synthetic control method

Indexed keywords

ARTICLE; CAUSAL MODELING; CONTROLLED STUDY; DIFFERENCE IN DIFFERENCE ESTIMATION; LAGGED DEPENDENT VARIABLE; MATCHING ON PAST OUTCOME; MONTE CARLO METHOD; MORTALITY RATE; OUTCOME ASSESSMENT; PRIORITY JOURNAL; SENSITIVITY AND SPECIFICITY; STATISTICAL ANALYSIS; STATISTICAL BIAS; SYNTHETIC CONTROL METHOD;

EID: 84969988705     PISSN: 13873741     EISSN: 15729400     Source Type: Journal    
DOI: 10.1007/s10742-016-0146-8     Document Type: Article

References (50)

1. Abadie, A., Diamond, A., Hainmueller, J.: Synthetic control methods for comparative case studies: estimating the effect of California’s Tobacco Control Program. J. Am. Stat. Assoc. 105(490), 493–505 (2010)
2. Abadie, A., Gardeazabal, J.: The economic costs of conflict: a case-control study for the Basque country. Am. Econ. Rev. 93, 112–132 (2003)
3. Abadie, A.: Semiparametric difference-in-differences estimators. Rev. Econ. Stud. 72(1), 1–19 (2005)
4. Acemoglu, D., Johnson, S., Kermani, A., Kwak, J., Mitton, T: The Value of connections in turbulent times: evidence from the United States. National Bureau of Economic Research (2013)
5. Achen, C.H.: Why lagged dependent variables can supress the explanatory power of other independent variables. In: Presented at the Annual Meeting of Political Methodology, Los Angeles (2000)
6. Allen, T., Fichera, E., Sutton, M.: Can payers use prices to improve quality? Evidence from English hospitals. Health Econ. (2014). doi:10.1002/hec.3121
7. Angrist, J.D., Pischke, J.S.: Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton University Press, Princeton (2009)
8. Ashenfelter, O.: Estimating the effect of training programs on earnings. Rev. Econ. Stat. 60(1), 47–57 (1978)
9. Ashenfelter, O., Card, D.: Using the longitudinal structure of earnings to estimate the effect of training programs. Rev. Econ. Stat. 67(4), 648–660 (1985)
10. Balaev, M.: Improving models of democracy: the example of lagged effects of economic development, education, and gender equality. Soc. Sci. Res. 46, 169–183 (2014)
11. Bell, B., Blundell, R., Van Reenen, J.: Getting the unemployed back to work: an evaluation of the new deal proposals. Int. Tax Public Finance 6(3), 339–360 (1999)
12. Bertrand, M., Duflo, E., Mullainathan, S.: How much should we trust differences-in-differences estimates? Q. J. Econ. 119(1), 249–275 (2004)
13. Bollinger, C.R., Miner, J.: On the robustness of coefficient estimates to the inclusion of proxy variables. J. Econ. Meth. 4(1), 101–122 (2015)
14. Blundell, R., Costa-Dias, M.: Alternative approaches to evaluation in empirical microeconomics. J. Hum. Resour. 44(3), 565–640 (2009)
15. Carpenter, C.S., Stehr, M.: The effects of mandatory seatbelt laws on seatbelt use, motor vehicle fatalities, and crash-related injuries among youths. J. Health Econ. 27, 642–662 (2008)
16. Cole, S.R., Hernán, M.A.: Constructing inverse probability weights for marginal structural models. Am. J. Epidemiol. 168(6), 656–664 (2008)
17. Diamond, A., Sekhon, J.S.: Genetic matching for estimating causal effects: a general multivariate matching method for achieving balance in observational studies. Rev. Econ. Stat. 95(3), 932–945 (2013)
18. Dube, A., Zipperer, B.: Pooled synthetic control estimates for recurring treatment: an application to minimum wage studies, University of Massachusetts. Amherst Working Paper (2013)
19. Eijkenaar, F.: Key issues in the design of pay for performance programs. Eur. J. Health Econ. 14, 117–131 (2013)
20. Emmert, M., Eijkenaar, F., Kemter, H., Esslinger, A.S., Schöffski, O.: Economic evaluation of pay-for-performance in health care: a systematic review. Eur. J. Health Econ. 13, 755–767 (2012)
21. Epstein, A.M.: Will pay for performance improve quality of care? The answer is in the details. New Engl. J. Med. 367, 1852–1853 (2012)
22. Fletcher, J.M., Frisvold, D.E., Tefft, N.: Non-linear effects of soda taxes on consumption and weight outcomes. Health Econ. 24, 566–582 (2015)
23. Health and Social Care Information Centre. Hospital episode statistics homepage. Leeds: Health and Social Care Information Centre. http://www.hscic.gov.uk/hes (2014)
24. Heckman, J.J., Ichimura, H., Todd, P.E.: Matching as an econometric evaluation estimator: evidence from evaluating a job training programme. Rev. Econ. Stud. 64(4), 605–654 (1997)
25. Ho, D.E., Imai, K., King, G., Stuart, E.A.: Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Polit. Anal. 15(3), 199–236 (2007)
26. Imbens, G.W.: Nonparametric estimation of average treatment effects under exogeneity: a review. Rec. Econ. Stat. 86(1), 4–29 (2004)
27. Imbens, G.W., Wooldridge, J.M.: Recent developments in the econometrics of program evaluation. J. Econ. Lit. 47(1), 5–86 (2009)
28. Jones, A.M., Rice, N.: Econometric evaluation of health policies. In: Glied, S., Smith, P. (eds.) The Oxford handbook of health economics. Oxford University Press, Oxford (2011)
29. Karlsberg-Schaffer, S., Sussex, J., Feng, Y.: Incentives to Follow Best Practice in Health Care. Office of Health Economics, London (2015)
30. Kayser, M.A., Wlezien, C.: Performance pressure: patterns of partisanship and the economic vote. Eur. J. Polit. Res. 50(3), 365–394 (2011)
31. Keele, L., Kelly, N.J.: Dynamic models for dynamic theories: the ins and outs of lagged dependent variables. Polit. Anal. 14(2), 186–205 (2006)
32. Kreif, N., Grieve, R., Hangartner, D., Turner, A.J., Nikolova, S., Sutton, M.: Examination of the synthetic control method for evaluating health policies with multiple treated units. Health Econ. (2015). doi:10.1002/hec.3258
33. Kristensen, S.R., McDonald, R., Sutton, M.: Should pay-for-performance schemes be locally designed? Evidence from the commissioning for quality and innovation (CQUIN) framework. J. Health Serv. Res. Policy 18(2 Supplement), 38–49 (2013)
34. Lagarde, M., Wright, M., Nossiter, J., Mays, N.: Challenges of payment-for performance in health care and other public services—design, implementation and evaluation. Policy Innovation Research Unit, London (2013)
35. Lubotsky, D., Wittenberg, M.: Interpretation of regressions with multiple proxies. Rev. Econ. Stat. 88, 531–548 (2007)
36. McDonald, R., Allen, T., Zaidi, S., Fichera, E., Todd, S., Sutton, M., Konteh, F., Hussein, K., Roe, J.: A qualitative and quantitative evaluation of the introduction of best practice tariffs: An evaluation report commissioned by the Department of Health. University of Nottingham and University of Manchester, Nottingham (2012)
37. Meacock, R., Kristensen, S.R., Sutton, M.: The cost-effectiveness of using financial incentives to improve provider quality: a framework and application. Health Econ. 23, 1–13 (2014)
38. Ramsahai, R., Grieve, R., Sekhon, J.S.: Extending iterative matching methods: an approach to improving covariate balance that allows prioritisation. Health Serv. Outcomes Res. Method. 11, 95–114 (2011)
39. Rubin, D.: Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 66, 688–701 (1974)
40. Ryan, A.M., Burgess, J., Dimick, J.B.: Why we shouldn’t be indifferent to specification in difference-in-differences analysis. Health Serv. Res. (2014). doi:10.1111/1475-6773.12270
41. Shiga, T., Wajima, Z., Ohe, Y.: Is operative delay associated with increased mortality of hip fracture patients? Systematic review, meta-analysis, and meta-regression. Can. J. Anesth. 55(3), 146–154 (2008)
42. Smith, J.A., Todd, P.E.: Does matching overcome Lalonde’s critique of nonexperimental estimators? J. Econom. 125(1–2), 305–353 (2005)
43. Steventon, A., Tunkel, S., Blunt, I., Bardsley, M.: Effect of telephone health coaching (Birmingham OwnHealth) on hospital use and associated costs: cohort study with matched controls. Brit. Med. J. 347, f4585 (2013)
44. Stuart, E.A.: Matching methods for causal inference: a review and a look forward. Stat. Sci. 25(1), 1–21 (2010)
45. Stuart, E.A., Huskamp, H.A., Duckworth, D., Simmons, J., Song, Z., Chernew, M.E., Barry, C.L.: Using propensity scores in difference-in-differences models to estimate the effects of a policy change. Health Serv. Outcomes Res. Method. 14, 166–182 (2014)
46. Sutton, M., Nikolova, S., Boaden, R., Lester, H., McDonald, R., Roland, M.: Reduced mortality with hospital pay for performance in England. New Engl. J. Med. 367(19), 1821–1828 (2012)
47. Tibshirani, R.: Regression Shrinkage and Selection via the Lasso. J. R. Stat. Soc. Ser. B Stat. Methodol. 58(1), 267–288 (1996)
48. Wagstaff, A., Moreno-Serra, R.: Europe and Central Asia’s great post-communist social health insurance experiment: aggregate impacts on health sector outcomes. J. Health Econ. 28(2), 322–340 (2009)
49. Wen, H., Hockenberry, J.M., Cummins, J.R.: The effect of medical marijuana laws on adolescent and adult use of marijuana, alcohol, and other substances. J. Health Econ. 42, 64–80 (2015)
50. Xu, Y.: Generalized synthetic control method for causal inference—with time series cross sectional data. Massachusetts Institute of Technology Political Science Department Working Paper No. 2015-1 (2015)