Examination of the Synthetic Control Method for Evaluating Health Policies with Multiple Treated Units

Health Economics (United Kingdom)

Volumn 25, Issue 12, 2016, Pages 1514-1528

  Kreif, Noémi ^a   Grieve, Richard ^a   Hangartner, Dominik ^b   Turner, Alex James ^c   Nikolova, Silviya ^d   Sutton, Matt ^c  

^a LONDON SCHOOL OF HYGIENE AND TROPICAL MEDICINE   (United Kingdom)

^b LONDON SCHOOL OF ECONOMICS   (United Kingdom)

^c UNIVERSITY OF MANCHESTER   (United Kingdom)

^d UNIVERSITY OF LEEDS   (United Kingdom)

Author keywords

difference in differences; pay for performance; policy evaluation; synthetic control method

Indexed keywords

ACUTE HEART INFARCTION; ARTICLE; CORONARY ARTERY BYPASS GRAFT; COST EFFECTIVENESS ANALYSIS; DATA STRUCTURE; DIFFERENCE IN DIFFERENCE ESTIMATION; ERROR; HEALTH CARE POLICY; HEALTH CARE QUALITY; HEALTH CARE SYSTEM; HEALTH ECONOMICS; HEART FAILURE; HIP SURGERY; IDIOSYNCRATIC ERROR; INFORMATION; KNEE SURGERY; LEAVE ONE OUT ROBUSTNESS; METHODOLOGY; MORTALITY; PARAMETERS; PATIENT CARE; PAY FOR PERFORMANCE INITIATIVE; PNEUMONIA; PRIORITY JOURNAL; REIMBURSEMENT; RISK ADJUSTED HOSPITAL MORTALITY; ROBUSTNESS; SENSITIVITY ANALYSIS; STATISTICAL ANALYSIS; SYNTHETIC CONTROL METHOD; ECONOMICS; HOSPITAL MORTALITY; HUMAN; STATISTICAL MODEL; TRENDS;

HEALTH POLICY; HOSPITAL MORTALITY; HUMANS; MODELS, STATISTICAL; REIMBURSEMENT, INCENTIVE;

EID: 84949843641     PISSN: 10579230     EISSN: 10991050     Source Type: Journal    
DOI: 10.1002/hec.3258     Document Type: Article

References (48)

1. Abadie A. 2005. Semiparametric difference-in-differences estimators. The Review of Economic Studies 72: 1–19.
2. Abadie A. 2013. Recent Advances in Programme Evaluation Institute of Fiscal Studies, course handouts.
3. Abadie A, Diamond A, Hainmueller J. 2010. Synthetic control methods for comparative case studies: Estimating the effect of California's tobacco control program. Journal of the American Statistical Association 105(490): 493–505.
4. Abadie A, Diamond A, Hainmueller J. 2015. Comparative politics and the synthetic control method. American Journal of Political Science 59(2): 495–510.
5. Abadie A, Gardeazabal J. 2003. The economic costs of conflict: A case study of the Basque Country. American Economic Review 93: 113–132.
6. Acemoglu D, Johnson S, Kermani A., Kwak J, Mitton T. 2013. The Value of Connections in Turbulent Times: Evidence from the United States. National Bureau of Economic Research.
7. Athey S, Imbens G W. 2006. Identification and inference in nonlinear difference-in-differences models. Econometrica 74: 431–497.
8. Bertrand M, Duflo E, Mullainathan S. 2002. How much should we trust differences-in-differences estimates? National Bureau of Economic Research.
9. Billmeier A, Nannicini T. 2013. Assessing economic liberalization episodes: a synthetic control approach. Review of Economics and Statistics 95: 983–1001.
10. Blundell R, Dias MC. 2009. Alternative approaches to evaluation in empirical microeconomics. Journal of Human Resources 44: 565–640.
11. Brewer M, Crossley T, Joyce R. 2013. Inference with difference-in-differences revisited.
12. Cameron AC, Gelbach JB, Miller DL. 2011. Robust inference with multiway clustering. Journal of Business & Economic Statistics 29: 238–249.
13. Campbell SM, Reeves D, Kontopantelis E, Sibbald B, Roland M. 2009. Effects of pay for performance on the quality of primary care in England. New England Journal of Medicine 361: 368–378.
14. Coffman M, Noy I. 2012. Hurricane Iniki: measuring the long-term economic impact of a natural disaster using synthetic control. Environment and Development Economics 17: 187–205.
15. Courtemanche CJ, Zapata D. 2014. Does universal coverage improve health? The Massachusetts experience. Journal of Policy Analysis and Management 33: 36–69.
16. Craig P, Cooper C, Gunnell D, Haw S, Lawson K, Macintyre S, Ogilvie D, Petticrew M, Reeves B, Sutton M, Thompson S. 2012. Using natural experiments to evaluate population health interventions: new Medical Research Council guidance. Journal of Epidemiology and Community Health 66: 1182–1186.
17. Currie J, Stabile M, Jones L. 2014. Do stimulant medications improve educational and behavioral outcomes for children with ADHD? Journal of Health Economics.
18. Deci EL, Koestner R, Ryan RM. 1999. A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological Bulletin 125: 627.
19. Doran T, Kontopantelis E, Valderas JM, Campbell S, Roland M, Salisbury C, Reeves D. 2011. Effect of financial incentives on incentivised and non-incentivised clinical activities: longitudinal analysis of data from the UK Quality and Outcomes Framework. BMJ 342: d3590.
20. Dranove D, Jin GZ. 2010. Quality disclosure and certification: theory and practice. Journal of Economic Literature 48: 935–963.
21. Dube A, Zipperer B. 2013. Pooled synthetic control estimates for recurring treatment: an application to minimum wage studies, University of Massachusetts. Amherst Working Paper.
22. Échevin D, Fortin B. 2014. Physician payment mechanisms, hospital length of stay and risk of readmission: evidence from a natural experiment. Journal of Health Economics 36: 112–124.
23. Edgington E, Onghena P. 2007. Randomization Tests, CRC Press: Boca Raton, FL.
24. Eijkenaar F. 2013. Key issues in the design of pay for performance programs. The European Journal of Health Economics 14: 117–131.
25. Emmert M, Eijkenaar F, Kemter H, Esslinger AS, Schöffski O. 2012. Economic evaluation of pay-for-performance in health care: a systematic review. The European Journal of Health Economics 13: 755–767.
26. Epstein AM. 2012. Will pay for performance improve quality of care? The answer is in the details. New England Journal of Medicine 367: 1852–1853.
27. Eren O, Ozbeklik IS. 2011. Right-to-Work Laws and State-Level Economic Outcomes: Evidence from the Case Studies of Idaho and Oklahoma Using Synthetic Control Method. unpublished manuscript last accessed on September, 4, 2012.
28. Fryer RG. 2011. Teacher incentives and student achievement: evidence from New York City public schools. National Bureau of Economic Research.
29. Gravelle H, Sutton M, Ma A. 2010. Doctor behaviour under a pay for performance contract: treating, cheating and case finding?. The Economic Journal 120: F129–F156.
30. Health and Social Care Information Centre. 2014. Hospital episode statistics homepage [Online]. Available: http://www.hscic.gov.uk/hes.
31. Hinrichs P. 2012. The effects of affirmative action bans on college enrollment, educational attainment, and the demographic composition of universities. Review of Economics and Statistics 94: 712–722.
32. Hodges JL, Lehmann EL. 1963. Estimates of location based on rank tests. The Annals of Mathematical Statistics 598–611.
33. Jones AM, Rice N. 2011. Econometric evaluation of health policies. In The Oxford Handbook of Health Economics, Glied S, Smith P (eds). Oxford University Press: Oxford.
34. Kristensen SR, Meacock R, Turner AJ, Boaden R, Mcdonald R, Roland M, Sutton M. 2014. Long-term effect of hospital pay for performance on mortality in England. New England Journal of Medicine 371: 540–548.
35. Lagarde M, Wright M, Nossiter J, Mays N. 2013. Challenges of payment-for-performance in health care and other public services – design, implementation and evaluation. Policy Innovation and Research Unit.
36. Lo J. 2013. How Do public health expansions vary by income strata? Evidence from Illinois' all kids program. Inquiry 50: 27–46.
37. Meacock R, Kristensen SR, Sutton M. 2014. The cost-effectiveness of using financial incentives to improve provider quality: a framework and application. Health Economics 23: 1–13.
38. Okeke EN. 2009. An empirical investigation of why doctors migrate and women fail to go for screening. The University of Michigan.
39. Rosenbaum PR. 2003. Exact confidence intervals for nonconstant effects by inverting the signed rank test. The American Statistician 57: 132–138.
40. Ryan AM. 2009. Effects of the Premier Hospital Quality Incentive Demonstration on Medicare patient mortality and cost. Health Services Research 44: 821–842.
41. Ryan AM, Burgess JF, Dimick JB. 2014. Why we shouldn't be indifferent to specification in difference-in-differences models. Health Services Research, under revision.
42. Schaffer ME. 2012. XTIVREG2: Stata module to perform extended IV/2SLS, GMM and AC/HAC, LIML and k-class regression for panel data models [Online]. Available from: http://EconPapers.repec.org/RePEc:boc:bocode:s456501.
43. Sekhon JS. 2011. Matching: multivariate and propensity score matching with automated balance search. Journal of Statistical Software 42(7): 1–52.
44. Statacorp. 2011. Stata Statistical Software: Release 12. StataCorp LP: College Station, TX.
45. Sutton M, Nikolova S, Boaden R, Lester H, Mcdonald R, Roland M. 2012. Reduced mortality with hospital pay for performance in England. New England Journal of Medicine 367: 1821–1828.
46. Williams SC, Schmaltz SP, Morton DJ, Koss RG, Loeb JM. 2005. Quality of care in US hospitals as reflected by standardized measures, 2002–2004. New England Journal of Medicine 353: 255–264.
47. Woolhandler S, Ariely D, Himmelstein, DU. 2012. Why pay for performance may be incompatible with quality improvement. BMJ 345: e5015.
48. Xu Y. 2015. Generalized synthetic control method for causal inference with time-series cross-sectional data.