Why we should not be indifferent to specification choices for difference-in-differences

Health Services Research

Volumn 50, Issue 4, 2015, Pages 1211-1235

  Ryan, Andrew M ^a   Burgess, James F ^b   Dimick, Justin B ^c  

^a UNIVERSITY OF MICHIGAN SCHOOL OF PUBLIC HEALTH   (United States)

^b BOSTON UNIVERSITY SCHOOL OF PUBLIC HEALTH   (United States)

^c UNIVERSITY OF MICHIGAN MEDICAL SCHOOL   (United States)

Author keywords

econometrics; health economics; health policy; Hospitals; quality of care

Indexed keywords

ARTICLE; CLUSTER ANALYSIS; DIFFERENCE IN DIFFERENCES MODEL; HEALTH CARE PLANNING; HEALTH CARE POLICY; HEALTH CARE QUALITY; HUMAN; MONTE CARLO METHOD; PROPENSITY SCORE; RANDOMIZED CONTROLLED TRIAL (TOPIC); SIMULATION; STATISTICAL ANALYSIS; STATISTICAL MODEL; TREND STUDY; DECISION MAKING; HEALTH SERVICES RESEARCH; HOSPITAL MANAGEMENT; ORGANIZATION AND MANAGEMENT; PROBABILITY; PROCEDURES;

CHOICE BEHAVIOR; HEALTH SERVICES RESEARCH; HOSPITAL ADMINISTRATION; HUMANS; MODELS, STATISTICAL; MONTE CARLO METHOD; PROBABILITY; QUALITY OF HEALTH CARE;

EID: 84937969176     PISSN: 00179124     EISSN: 14756773     Source Type: Journal    
DOI: 10.1111/1475-6773.12270     Document Type: Article

References (66)

1. Abadie, A., A. Diamond, and, J. Hainmueller,. 2010. " Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California's Tobacco Control Program." Journal of the American Statistical Association 490 (105): 493-505.
2. Ai, C. R., and, E. C. Norton,. 2003. " Interaction Terms in Logit and Probit Models." Economics Letters 80 (1): 123-9.
3. Anger, S., M. Kvasnicka, and, T. Siedler,. 2011. " One Last Puff? Public Smoking Bans and Smoking Behavior." Journal of Health Economics 30 (3): 591-601.
4. Angrist, J. D., and, J.-S. Pischke,. 2008. Mostly Harmless Econometrics: An Empiricist's Companion. Princeton, NJ: Princeton University Press.
5. Angrist, J. D., and, J.-S. Pischke,. 2010. " The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics." Journal of Economic Perspectives 24 (2): 3-30.
6. Athey, S., and, G. W. Imbens,. 2006. " Identification and Inference in Nonlinear Difference-in-Differences Models." Econometrica 74 (2): 431-97.
7. Azrin, S. T., H. A. Huskamp, V. Azzone, H. H. Goldman, R. G. Frank, M. A. Burnam, S. L. Normand, M. S. Ridgely, A. S. Young, C. L. Barry, A. B. Busch, and, G. Moran,. 2007. " Impact of Full Mental Health and Substance Abuse Parity for Children in the Federal Employees Health Benefits Program." Pediatrics 119 (2): e452-9.
8. Baxter, R., G. T. Ray, and, B. H. Fireman,. 2010. " Effect of Influenza Vaccination on Hospitalizations in Persons Aged 50 Years and Older." Vaccine 28 (45): 7267-72.
9. Bertrand, M., E. Duflo, and, S. Mullainathan,. 2002. " How Much Should We Trust Differences-in-Differences Estimates? " National Bureau of Economic Research Working Paper Series No. 8841.
10. Besley, T., and, R. Burgess,. 2004. " Can Labor Regulation Hinder Economic Performance? Evidence from India." Quarterly Journal of Economics 119 (1): 91-134.
11. Blum, A. B., L. C. Kleinman, B. Starfield, and, J. S. Ross,. 2012. " Impact of State Laws That Extend Eligibility for Parents' Health Insurance Coverage to Young Adults." Pediatrics 129 (3): 426-32.
12. Breusch, T. S., and, A. R. Pagan,. 1979. " A Simple Test for Heteroscedasticity and Random Coefficient Variation." Econometrica 47 (5): 1287.
13. Cantor, J. C., A. C. Monheit, D. DeLia, and, K. Lloyd,. 2012. " Early Impact of the Affordable Care Act on Health Insurance Coverage of Young Adults." Health Services Research 47 (5): 1773-90.
14. Chay, K. Y., P. J. McEwan, and, M. Urquiola,. 2005. " The Central Role of Noise in Evaluating Interventions That Use Test Scores to Rank Schools." American Economic Review 95: 1237-58.
15. Chen, Y., and, J. Meinecke,. 2012. " Do Healthcare Report Cards Cause Providers to Select Patients and Raise Quality of Care? " Health Economics 21 (Suppl 1): 33-55.
16. Cohen, J., 1988. Statistical Power Analysis for the Behavioral Sciences. Hillside, NJ: Lawrence Erlbaum Associates.
17. Cunningham, P. J., J. Hadley, and, J. Reschovsky,. 2002. " The Effects of SCHIP on Children's Health Insurance Coverage: Early Evidence from the Community Tracking Study." Medical Care Research and Review: MCRR 59 (4): 359-83.
18. Dhingra, S. S., M. M. Zack, T. W. Strine, B. G. Druss, and, E. Simoes,. 2013. " Change in Health Insurance Coverage in Massachusetts and Other New England States by Perceived Health Status: Potential Impact of Health Reform." American Journal of Public Health 103 (6): e107-14.
19. Diamond, A., and, J. S. Sekhon,. 2013. " Genetic Matching for Estimating Causal Effects: A General Multivariate Matching Method for Achieving Balance in Observational Studies." Review of Economics and Statistics 95 (3): 932-45.
20. Dimick, J. B., L. H. Nicholas, A. M. Ryan, J. R. Thumma, and, J. D. Birkmeyer,. 2013. " Bariatric Surgery Complications before vs after Implementation of a National Policy Restricting Coverage to Centers of Excellence." Journal of the American Medical Association 309 (8): 792-9.
21. Donald, S. G., and, K. Lang,. 2007. " Inference with Difference in Differences and Other Panel Data." Review of Economics and Statistics 89 (2): 221-33.
22. Dranove, D., D. Kessler, M. McClellan, and, M. Satterthwaite,. 2003. " Is More Information Better? The Effects of "Report Cards" on Health Care Providers." Journal of Political Economy 111 (3): 555-88.
23. Dreyer, N. A., P. Velentgas, K. Westrich, and, R. Dubois,. 2014. " The GRACE Checklist for Rating the Quality of Observational Studies of Comparative Effectiveness: A Tale of Hope and Caution." Journal of Managed Care and Specialty Pharmacy 20 (3): 301-8.
24. Drukker, D. M., 2003. " Testing for Serial Correlation in Linear Panel-Data Models." Stata Journal 3 (2): 168-77.
25. Dusheiko, M., H. Gravelle, R. Jacobs, and, P. Smith,. 2006. " The Effect of Financial Incentives on Gatekeeping Doctors: Evidence from a Natural Experiment." Journal of Health Economics 25 (3): 449-78.
26. Epstein, A. J., S. H. Busch, A. B. Busch, D. A. Asch, and, C. L. Barry,. 2013. " Does Exposure to Conflict of Interest Policies in Psychiatry Residency Affect Antidepressant Prescribing? " Medical Care 51 (2): 199-203.
27. Ernst, M. D., 2004. " Permutation Methods: A Basis for Exact Inference." Statistical Science 19 (4): 676-85.
28. Flum, D. R., S. Kwon, K. MacLeod, B. Wang, R. Alfonso-Cristancho, L. P. Garrison, and, S. D. Sullivan,. 2011. " Bariatric Obesity Outcome Modeling Collaborative. The Use, Safety and Cost of Bariatric Surgery before and after Medicare's National Coverage Decision." Annals of Surgery 254 (6): 860-5.
29. Goldman, H. H., R. G. Frank, M. A. Burnam, H. A. Huskamp, M. S. Ridgely, S. L. Normand, A. S. Young, C. L. Barry, V. Azzone, A. B. Busch, S. T. Azrin, G. Moran, C. Lichtenstein, and, M. Blasinsky,. 2006. " Behavioral Health Insurance Parity for Federal Employees." New England Journal of Medicine 354 (13): 1378-86.
30. Graves, J. A., and, J. Gruber,. 2012. " How Did Health Care Reform in Massachusetts Impact Insurance Premiums? " American Economic Review 102 (3): 508-13.
31. Hausman, J. A., and, D. A. Wise,. 1979. " Attrition Bias in Experimental and Panel Data: The Gary Income Maintenance Experiment." Econometrica 47 (2): 455-73.
32. Jha, A. K., K. E. Joynt, E. J. Orav, and, A. M. Epstein,. 2012. " The Long-Term Effect of Premier Pay for Performance on Patient Outcomes." New England Journal of Medicine 366 (17): 1606-15.
33. Jones, S. S., J. L. Adams, E. C. Schneider, J. S. Ringel, and, E. A. McGlynn,. 2010. " Electronic Health Record Adoption and Quality Improvement in US Hospitals." American Journal of Managed Care 16 (12 Suppl HIT): SP64-71.
34. Joynt, K. E., D. M. Blumenthal, E. J. Orav, F. S. Resnic, and, A. K. Jha,. 2012. " Association of Public Reporting for Percutaneous Coronary Intervention with Utilization and Outcomes among Medicare Beneficiaries with Acute Myocardial Infarction." Journal of the American Medical Association 308 (14): 1460-8.
35. Joynt, K. E., D. Chan, E. J. Orav, and, A. K. Jha,. 2013. " Insurance Expansion In Massachusetts Did Not Reduce Access among Previously Insured Medicare Patients." Health Affairs 32 (3): 571-8.
36. Kantarevic, J., and, B. Kralj,. 2013. " Link between Pay for Performance Incentives and Physician Payment Mechanisms: Evidence from the Diabetes Management Incentive in Ontario." Health Economics 22 (12): 1417-39.
37. Kessler, D. P., W. M. Sage, and, D. J. Becker,. 2005. " Impact of Malpractice Reforms on the Supply of Physician Services." Journal of the American Medical Association 293 (21): 2618-25.
38. Kozhimannil, K. B., M. R. Law, C. Blauer-Peterson, F. Zhang, and, J. F. Wharam,. 2013. " The Impact of High-deductible Health Plans on Men and Women: An Analysis of Emergency Department Care." Medical Care 51 (8): 639-45.
39. Landrum, M. B., S. E. Bronskill, and, S. T. Normand,. 2000. " Analytic Methods for Constructing Cross-Sectional Profiles of Health Care Providers." Health Services Research Methodology 1 (1): 23-47.
40. Leonhardt, K. K., O. Yakusheva, D. Phelan, A. Reeths, T. Hosterman, D. Bonin, and, M. Costello,. 2011. " Clinical Effectiveness and Cost Benefit of Universal versus Targeted Methicillin-Resistant Staphylococcus aureus Screening Upon Admission in Hospitals." Infection Control and Hospital Epidemiology 32 (8): 797-803.
41. Leuven, E., and, B. Sianesi, 2003. " PSMATCH2: Stata Module to Perform Full Mahalanobis and Propensity Score Matching, Common Support Graphing, and Covariate Imbalance Testing " [accessed on November 11, 2014]. Available at http://ideas.repec.org/c/boc/bocode/s432001.html
42. Long, S. K., and, K. Stockley,. 2011. " The Impacts of State Health Reform Initiatives on Adults in New York and Massachusetts." Health Services Research 46 (1 Pt 2): 365-87.
43. McCullough, J. S., J. Christianson, and, B. Leerapan,. 2013. " Do Electronic Medical Records Improve Diabetes Quality in Physician Practices? " American Journal of Managed Care 19 (2): 144-9.
44. Mitchell, J. M., 2008. " Do Financial Incentives Linked to Ownership of Specialty Hospitals Affect Physicians' Practice Patterns? " Medical Care 46 (7): 732-7.
45. Moher, D., A. Liberati, J. Tetzlaff, D. G. Altman, and, P. Group,. 2009. " Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement." Journal of Clinical Epidemiology 62 (10): 1006-12.
46. Nguyen, H. V., 2013. " Do Smoke-Free Car Laws Work? Evidence from a Quasi-Experiment." Journal of Health Economics 32 (1): 138-48.
47. Nguyen, N. T., S. Hohmann, J. Slone, E. Varela, B. R. Smith, and, D. Hoyt,. 2010. " Improved Bariatric Surgery Outcomes for Medicare Beneficiaries after Implementation of the Medicare National Coverage Determination." Archives of Surgery 145 (1): 72-8.
48. Pham-Gia, T., and, T. L. Hung,. 2001. " The Mean and Median Absolute Deviations." Mathematical and Computer Modelling 34 (7-8): 921-36.
49. Rogers, W. H., 1993. " Regression Standard Errors in Clustered Samples." Stata Technical Bulletin 13: 19-23.
50. Ryan, A. M., 2009. " The Effects of the Premier Hospital Quality Incentive Demonstration on Mortality and Hospital Costs in Medicare." Health Services Research 44 (3): 821-42.
51. Ryan, A. M., and, J. Blustein,. 2011. " The Effect of the MassHealth Hospital Pay-for-Performance Program on Quality." Health Services Research 46 (3): 712-28.
52. Ryan, A. M., J. Blustein, and, L. P. Casalino,. 2012a. " Medicare's Flagship Test of Pay-For-Performance Did Not Spur More Rapid Quality Improvement among Low-Performing Hospitals." Health Affairs (Millwood) 31 (4): 797-805.
53. Ryan, A. M., B. K. Nallamothu, and, J. B. Dimick,. 2012b. " Medicare's Public Reporting Initiative on Hospital Quality Had Modest or No Impact on Mortality from Three Key Conditions." Health Affairs (Millwood) 31 (3): 585-92.
54. Ryan, A. M., T. Bishop, S. Shih, and, L. P. Casalino,. 2013. " Small Physician Practices in New York Needed Sustained Help to Realize Gains in Quality from Use of Electronic Health Records." Health Affairs 32 (1): 53-62.
55. Scanlon, D. P., C. S. Hollenbeak, J. Beich, A. M. Dyer, R. A. Gabbay, and, A. Milstein,. 2008. " Financial and Clinical Impact of Team-Based Treatment for Medicaid Enrollees with Diabetes in a Federally Qualified Health Center." Diabetes Care 31 (11): 2160-5.
56. Shen, Y. C., and, S. Zuckerman,. 2005. " The Effect of Medicaid Payment Generosity on Access and Use among Beneficiaries." Health Services Research 40 (3): 723-44.
57. Song, Z., D. G. Safran, B. E. Landon, Y. He, R. P. Ellis, R. E. Mechanic, M. P. Day, and, M. E. Chernew,. 2011. " Health Care Spending and Quality in Year 1 of the Alternative Quality Contract." New England Journal of Medicine 365 (10): 909-18.
58. Suehs, B. T., A. Louder, M. Udall, J. C. Cappelleri, A. V. Joshi, and, N. C. Patel,. 2014. " Impact of a Pregabalin Step Therapy Policy among Medicare Advantage Beneficiaries." Pain Practice 14 (5): 419-26.
59. Sutton, M., S. Nikolova, R. Boaden, H. Lester, R. McDonald, and, M. Roland,. 2012. " Reduced Mortality with Hospital Pay for Performance in England." New England Journal of Medicine 367 (19): 1821-8.
60. Volpp, K. G., A. K. Rosen, P. R. Rosenbaum, P. S. Romano, O. Even-Shoshan, A. Canamucio, L. Bellini, T. Behringer, and, J. H. Silber,. 2007. " Mortality among Patients in VA Hospitals in the First 2 Years Following ACGME Resident Duty Hour Reform." Journal of the American Medical Association 298 (9): 984-92.
61. Werner, R. M., R. T. Konetzka, and, D. Polsky,. 2013. " The Effect of Pay-for-Performance in Nursing Homes: Evidence from State Medicaid Programs." Health Services Research 48 (4): 1393-414.
62. Wharam, J. F., A. J. Graves, F. Zhang, S. B. Soumerai, D. Ross-Degnan, and, B. E. Landon,. 2012. " Two-Year Trends in Cancer Screening among Low Socioeconomic Status Women in an HMO-Based High-Deductible Health Plan." Journal of General Internal Medicine 27 (9): 1112-9.
63. Wooldridge, J., 2002. Econometric Analysis of Cross Section and Panel Data. Cambridge, MA: MIT Press.
64. Wooldridge, J.,. 2009. Introductory Econometrics: A Modern Approach. Mason, OH: Thomson.
65. Zhu, J., P. Brawarsky, S. Lipsitz, H. Huskamp, and, J. S. Haas,. 2010. " Massachusetts Health Reform and Disparities in Coverage, Access and Health Status." Journal of General Internal Medicine 25 (12): 1356-62.
66. Zivin, K., P. N. Pfeiffer, B. R. Szymanski, M. Valenstein, E. P. Post, E. M. Miller, and, J. F. McCarthy,. 2010. " Initiation of Primary Care-Mental Health Integration Programs in the VA Health System: Associations with Psychiatric Diagnoses in Primary Care." Medical Care 48 (9): 843-51.