Composite Measures of Health Care Provider Performance: A Description of Approaches

Milbank Quarterly

Volumn 93, Issue 4, 2015, Pages 788-825

  Shwartz, Michael ^a,b   Restuccia, Joseph D ^a,b   Rosen, Amy K ^a,c  

^a BOSTON UNIVERSITY   (United States)

^b VETERANS AFFAIRS MEDICAL CENTER   (United States)

^c BOSTON UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

composite measures; performance measurement

Indexed keywords

ACCURACY; ARTICLE; CONSTRUCT VALIDITY; HEALTH CARE DELIVERY; HEALTH CARE ORGANIZATION; HEALTH CARE POLICY; HEALTH CARE QUALITY; HEALTH PROGRAM; HEALTH SURVEY; HUMAN; MEASUREMENT ACCURACY; OUTCOME ASSESSMENT; PERFORMANCE MEASUREMENT SYSTEM; PRIORITY JOURNAL; RELIABILITY; SCORING SYSTEM; SOCIAL STATUS; BENCHMARKING; ECONOMICS; PERSONNEL MANAGEMENT; PRIMARY HEALTH CARE; PROCEDURES; REIMBURSEMENT; UNITED STATES;

BENCHMARKING; HUMANS; PHYSICIAN INCENTIVE PLANS; PRIMARY HEALTH CARE; QUALITY ASSURANCE, HEALTH CARE; QUALITY INDICATORS, HEALTH CARE; REIMBURSEMENT, INCENTIVE; UNITED STATES;

EID: 84962090347     PISSN: 0887378X     EISSN: 14680009     Source Type: Journal    
DOI: 10.1111/1468-0009.12165     Document Type: Article

References (48)

1. Institute of Medicine. To Err Is Human: Building a Safer Health System. Washington, DC: National Academies Press; 2000.
2. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001.
3. Institute of Medicine. Performance Measurement: Accelerating Improvement. Washington, DC: National Academies Press; 2006.
4. Centers for Medicare and Medicaid Services. National provider call: Hospital Value-Based Purchasing: fiscal year 2015 overview for beneficiaries, providers, and stakeholders. 2013. http://www.cms.gov/outreach-and-education/outreach/npc/downloads/hospvbp-fy15-npc-final-03052013-508.pdf. Accessed November 23, 2014.
5. National Quality Forum. Composite performance measure evaluation guidance. 2013. http://www.qualityforum.org/Publications/2013/04/Composite-Performance-Measure-Evaluation-Guidance.aspx. Accessed November 23, 2014.
6. Peterson ED, DeLong ER, Masoudi FA, et al., ACCF/AHA 2010 position statement on composite measures for healthcare performance assessment: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Performance Measures. Circulation. 2010; 121: 1780-1791.
7. Rajaram R, Barnard C, Bilimoria KY,. Concerns about using the Patient Safety Indicator-90 Composite in pay-for-performance programs. JAMA. 2015; 313: 897-898.
8. Shwartz M, Ash AS,. Composite measures: matching the method to the purpose. 2008. Agency for Healthcare Research and Quality, National Quality Measures Clearinghouse. http://www.qualitymeasures.ahrq.gov/expert/expert-commentary.aspx?id=16464. Accessed November 23, 2014.
9. Babbie E,. The Practice of Social Research. Washington, DC: Wadsworth; 1995.
10. Battelle. Patient Safety Measures (PSI) Parameter Estimates: Version 4.5 (With Corrected PSI #90). Columbus, OH: Battelle; 2013.
11. Nolan T, Berwick DM,. All-or-none measurement raises the bar on performance. JAMA. 2006; 295: 1168-1170.
12. Institute for Healthcare Improvement. Implement the IHI Central Line Bundle. Undated. http://www.ihi.org/resources/Pages/Changes/ImplementtheCentralLineBundle.aspx. Accessed April 11, 2015.
13. O'Brien SM, Shahian DM, DeLong ER, et al., Quality measurement in adult cardiac surgery: part 2-statistical considerations in composite measure scoring and provider rating. Ann Thorac Surg. 2007; 83: S13-S26.
14. Minnesota Department of Human Services. Health Care Delivery System (HCDS) quality measurement. 2012. http://www.dhs.state.mn.us/main/groups/business-partners/documents/pub/dhs16-177107.pdf. Accessed April 10, 2015.
15. Austin JM, Andrea GD, Birkmeyer JD, et al., Safety in numbers: the development of Leapfrog's Composite Safety Score for U.S. hospitals. J Patient Safety. 2013; 9: 1-9.
16. Ranking by rank. Significance. 2014; 11: 2-3.
17. Centers for Medicare and Medicaid Services. Health Insurance Marketplace: 2015 beta testing of the Quality Rating System and Qualified Health Plan Enrollee Experience Survey: technical guidance for 2015. 2014. http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityInitiativesGenInfo/Downloads/2015-QRS-and-QHP-Enrollee-Survey-Technical-Guidance-Final.pdf. Accessed April 11, 2015.
18. Reeves D, Campbell SM, Adams J, Shekelle PG, Kontopantelis E, Roland MO,. Combining multiple measures of clinical quality: an evaluation of different analytic approaches. Med Care. 2007; 45: 489-496.
19. Eapen ZJ, Fonarow GC, Dai D, et al., Comparison of composite measure methodologies for rewarding quality of care: an analysis from the American Heart Association's Get With the Guidelines Program. Circ Cardiovasc Qual Outcomes. 2011; 4: 610-618.
20. Jacobs R, Goddard M, Smith PC,. How robust are hospital ranks based on composite performance measures? Med Care. 2005; 43: 1177-1184.
21. Radley DC, McCarthy D, Lippin JA, Hayes SL, Schoen C,. Aiming Higher: Results from a Scorecard on State Health System Performance. New York, NY: Commonwealth Fund; 2014.
22. Centers for Medicare and Medicaid Services. CMS fact sheet: Nursing Home Compare Five-Star Quality Rating System. Undated. http://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/CertificationandComplianc/downloads/consumerfactsheet.pdf. Accessed April 11, 2015.
23. Shwartz M, Peköz EA, Burgess JR, Christiansen CL, Rosen AK, Berlowitz D,. A probability metric for identifying high performing facilities: an application for pay-for-performance. Med Care. 2014; 52: 1030-1036.
24. Spiegelhalter D, Sherlaw-Johnson C, Bardsley M, Blunt I, Wood C, Grigg O,. Statistical methods for healthcare regulation: rating, screening and surveillance. Appl Stat. 2012; 175 (Pt. 1): 1-47.
25. Goldstein E, Farquhar M, Crofton C, Darby C, Garfinkel S,. Measuring hospital care from the patients' perspective: an overview of the CAHPS Hospital Survey development process. Health Serv Res. 2005; 40: 1977-1995.
26. O'Malley AJ, Zaslavsky AM, Hays RD, Hepner KA, Keller S, Cleary PD,. Exploratory factor analysis of the CAHPS hospital pilot survey responses across and within medical, surgical, and obstetrics services. Health Serv Res. 2005; 40: 2078-2095.
27. Glickman SW, Boulding W, Roos JMT, Staelin R, Peterson ED, Schulman K,. Alternative pay-for-performance scoring methods: implications for quality improvement and patient outcomes. Med Care. 2009; 47: 1062-1068.
28. Shwartz M, Ren J, Peköz EA, Wang X, Cohen AB, Restuccia JD,. Estimating a composite measure of hospital quality from the hospital compare database: differences when using a Bayesian hierarchical latent variable model versus denominator-based weights. Med Care. 2008; 46: 778-785.
29. Van Doorn-Klomberg AL, Braspenning JCC, Feskens RCW, Bouma M, Campbell SM, Reeves D,. Precision of individual and composite performance scores: the ideal number of indicators in an indcator set. Med Care. 2013; 51: 115-121.
30. Nunnelly J, Bernstein I,. Psychometric Theory. 3rd ed. New York, NY: McGraw-Hill; 1994.
31. Scholle SH, Roski J, Adams JL, et al., Benchmarking physician performance: reliability of individual and composite measures. Am J Manage Care. 2008; 14: 829-838.
32. Kaplan SH, Griffith JL, Price LL, Pawlson LG, Greenfield S,. Improving the reliability of physician performance assessment: identifying the "physician effect" on quality and creating composite measures. Med Care. 2009; 47: 378-387.
33. Shwartz M, Cohen AB, Restuccia JD, et al., How well can we identify the high-performing hospital? Med Care Res Rev. 2011; 68: 290-310.
34. Ash AS, Fienberg SE, Louis TA, Normand S-LT, Stukel TA, Utts J,. Statistical issues in assessing hospital performance: commissioned by the Committee of Presidents of Statistical Societies. 2012. http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/HospitalQualityInits/Downloads/Statistical-Issues-in-Assessing-Hospital-Performance.pdf. Accessed November 23, 2014.
35. Efron B, Morris C,. Stein's paradox in statistics. Sci Am. 1977; 236: 119-127.
36. Christiansen CL, Morris CN,. Improving the statistical approach to health care provider profiling. Ann Intern Med. 1997; 127: 764-768.
37. Normand, S-LT, Glickman M, Gastsonis C,. Statistical methods for profiling providers of medical care: issues and applications. J Am Stat Assoc. 1997; 92: 803-814.
38. Burgess JF Jr, Christiansen CL, Michalak SE, Morris CN,. Medical profiling: improving standards and risk adjustments using hierarchical models. J Health Econ. 2000; 19: 291-309.
39. Greenland S., Principles of multilevel modeling. Int J Epidemiol. 2000; 29: 158-167.
40. Landrum MB, Bronskill SE, Normand S-LT,. Analytic methods for constructing cross-sectional profiles of health care providers. Health Serv Outcomes Res Methodol. 2000; 1: 23-47.
41. Arling GT, Lewis T, Kane RL, Mueller C, Flood S,. Improving quality assessment through multilevel modeling: the case of Nursing Home Compare. Health Serv Res. 2007; 42: 1177-1199.
42. Staiger D, Dimick JB, Baser O, Fan Z, Birkmeyer JD,. Empirically derived composite measures of surgical performance. Med Care. 2009; 47: 226-233.
43. Shwartz M, Peköz EA, Christiansen CL, Burgess JF, Berlowitz D,. Shrinkage estimators for a composite measure of quality conceptualized as a formative construct. Health Serv Res. 2013; 48: 271-289.
44. Horwitz L, Partovian C, Lin Z, et al., Hospital-wide (all condition) 30-day risk-standardized readmission measure: draft measure methodology report. 2011. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/MMS/downloads/MMSHospital-WideAll-ConditionReadmissionRate.pdf. Accessed April 5, 2015.
45. Dimick JB, Staiger DO, Basur O, Birkmeyer JD,. Composite measures for predicting surgical mortality in the hospital. Health Aff. 2009; 28: 1189-1198.
46. Dimick JB, Staiger DO, Osborne NH, Nicholas LH, Birkmeyer JD,. Composite measures for rating hospital quality with major surgery. Health Aff. 2012; 47: 1861-1879.
47. Chen LM, Staiger DO, Birkmeyer JD, Ryan AM, Zhang W, Dimick JB,. Composite quality measures for common inpatient medical conditions. Med Care. 2013; 51: 832-837.
48. Silber JH, Rosenbaum PR, Brachet TJ, et al., The Hospital Compare mortality model and the volume-outcome relationship. Health Serv Res. 2010; 45 (5, pt. 1): 1148-1167.