Accuracy of hospital standardized mortality rates: Effects of model calibration

Medical Care

Volumn 52, Issue 4, 2014, Pages 378-384

  Kipnis, Patricia ^a   Liu, Vincent ^b   Escobar, Gabriel J ^b  

^a DIVISION OF RESEARCH   (United States)

^b Hospital Operations Research Kaiser Permanente Northern California   (United States)

Author keywords

C statistic; Calibration; Hospital report cards; Logistic regression; Monte carlo simulations; Risk adjustment

Indexed keywords

ACCURACY; ARTICLE; CONTROLLED STUDY; ELECTRONIC MEDICAL RECORD; HOSPITAL; HUMAN; INTEGRATED HEALTH CARE SYSTEM; MONTE CARLO METHOD; MORTALITY; PRIORITY JOURNAL; RISK ASSESSMENT; SAMPLE SIZE; SENSITIVITY AND SPECIFICITY; STANDARDIZED MORTALITY RATIO; CALIBRATION; REPRODUCIBILITY; STANDARD; STATISTICAL MODEL; STATISTICS;

CALIBRATION; HOSPITAL MORTALITY; HOSPITALS; HUMANS; MODELS, STATISTICAL; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY;

EID: 84904856951     PISSN: 00257079     EISSN: 15371948     Source Type: Journal    
DOI: 10.1097/MLR.0000000000000111     Document Type: Article

References (29)

1. Fund C, eds. Commonwealth Fund Analysis of HQA/Hospital Compare Data. New York, NY: Commonwealth Fund. Available at: http:// www.whynotthebest.org.
2. California Office of Statewide Health Planning and Development. Healthcare Outcomes-Clinical Data Programs. Development COoSHPa; 2013. Available at http://www.oshpd.ca.gov/HQAD/outsornces/ clinical.htm.
3. HealthGrades. Hospital Report Cards: Mortality and Complication Outcomes 2013 Methodology. In: HealthGrades, ed. Hospital Report Cards. Denver, CO: Health Grades, Inc; 2013.
4. Iezzoni AA, Shwartz M. Risk Adjustment for Measuring Health Care Outcomes, Chapter 9. Chicago, IL: Health Administration Press; 1997.
5. Jarman B, Gault S, Alves B, et al. Explaining differences in English hospital death rates using routinely collected data. BMJ. 1999;318: 1515-1520.
6. Jarman B, Aylin P, Browne M. Monitoring changes in hospital standardized mortality ratios. BMJ. 2005;330:329.
7. Krumholz HM, Wang Y, Mattera JA, et al. An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with heart failure. Circulation. 2006; 113: 1693-1701.
8. Centers for Medicare and Medicaid Services. Hospital Compare. Available at: http://www.hospitalcompare.hhs.gov. Accessed August 10, 2013.
9. Bardach N, Hibbard JH, Dudley RA. Users of Public Reports of Hospital Quality: Who, What, Why, and How?. Rockville, MD: Agency for Healthcare Research and Quality; 2011.
10. Hibbard JH, Green J, Sofaer S, et al. An experiment shows that a welldesigned report on costs and quality can help consumers choose highvalue health care. Health Aff. 2012;31:560-568.
11. Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21:128-138.
12. Austin PC. The relationship between the C-statistic of a risk-adjustment model and the accuracy of hospital report cards: a Monte Carlo study. Med Care. 2013;51:275-284.
13. Dimick JB, Welch HG, Birkmeyer JD. Surgical mortality as an indicator of hospital quality: the problem with small sample size. JAMA. 2004; 292:847-851.
14. Escobar GJ, Fireman BH, Palen TE, et al. Risk adjusting community-acquired pneumonia hospital outcomes using automated databases. Am J Manag Care. 2008;14:158-166.
15. Escobar GJ, Gardner M, Greene JG, et al. Risk-adjusting hospital mortality using a comprehensive electronic record in an integrated healthcare delivery system. Med Care. 2013;51:446-453.
16. Liu VTB, Ragins AI, Kipnis P, et al. An electronic simplified acute physiology score-based risk adjustment score for critical illness in an integrated healthcare system. Crit Care Med. 2013;41:41-48.
17. Liu V, Kipnis P, Gould MK, et al. Length of stay predictions: Improvements through the use of automated laboratory and comorbidity variables. Med Care. 2010;48:739-744.
18. Escobar GJ, Greene JD, Gardner MN, et al. Intra-hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6:74-80.
19. Liu V, Kipnis P, Rizk NW, et al. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med. 2012;7:224-230.
20. Escobar G, Greene J, Scheirer P, et al. Risk adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46:232-239.
21. Nagelkerke N. A note on a general definition of the coefficient of determination. Biometrika. 1991;78:691-692.
22. Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York, NY: Springer; 2009.
23. Breslow NE, Day NE. Statistical methods in cancer research. Volume II-the design and analysis of cohort studies. IARC Sci Publ. 1987;82: 1-406.
24. Kramer AA, Zimmerman JE. Assessing the calibration of mortality benchmarks in critical care: The Hosmer-Lemeshow test revisited. Crit Care Med. 2007;35:2052-2056.
25. Krumholz HM, Spertus JA, Shahian DM, et al. Measuring performance for treating heart attacks and heart failure: The case for outcomes measurement. Health Aff. 2007;26:75-85.
26. Kipnis P, Escobar GJ, Draper D. Effect of choice of estimation method on inter-hospital mortality rate comparisons. Med Care. 2010;48: 456-485.
27. Krumholz HM, Brindis RG, Brush JE, et al. Standards for statistical models used for public reporting of health outcomes: an American Heart Association Scientific Statement ftom the Quality of Care and Outcomes Research Interdisciplinary Writing Group: cosponsored by the Council on Epidemiology and Prevention and the Stroke Council. Endorsed by the American College of Cardiology Foundation. Circulation. 2006;113:456-462.
28. Iezzoni LI. "Black box" medical information systems: A technology needing assessment. JAMA. 1991;265:3006-3007.
29. Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007;115:928-935.