Hospital-wide mortality as a quality metric: Conceptual and methodological challenges

American Journal of Medical Quality

Volumn 27, Issue 2, 2012, Pages 112-123

  Shahian, David M ^a   Iezzoni, Lisa I ^a   Meyer, Gregg S ^a   Kirle, Leslie ^b   Normand, Sharon Lise T ^a  

^a MASSACHUSETTS GENERAL HOSPITAL   (United States)

^b Kirle Consulting   (United States)

Author keywords

hospital wide mortality; performance metrics

Indexed keywords

ARTICLE; EPIDEMIOLOGY; HEALTH CARE QUALITY; HUMAN; METHODOLOGY; MORTALITY; ORGANIZATION AND MANAGEMENT; RISK ASSESSMENT; SAMPLE SIZE; STANDARD;

BIAS (EPIDEMIOLOGY); HOSPITAL MORTALITY; HUMANS; QUALITY INDICATORS, HEALTH CARE; QUALITY OF HEALTH CARE; RISK ADJUSTMENT; SAMPLE SIZE;

EID: 84858251813     PISSN: 10628606     EISSN: 1555824X     Source Type: Journal    
DOI: 10.1177/1062860611412358     Document Type: Article

References (99)

1. Krakauer H,Bailey RC,Skellan KJ, et al.Evaluation of the HCFA model for the analysis of mortality following hospitalization.Health Serv Res. 1992;27:317-335.
2. Iezzoni LIRisk Adjustment for Measuring Health Care Outcomes. 3rd ed.Chicago, IL: Health Administration Press; 2003.Chicago, IL: Health Administration Press; 2003:.
3. Jarman B,Gault S,Alves B, et al.Explaining differences in English hospital death rates using routinely collected data.BMJ. 1999;318:1515-1520.
4. Jarman B,Bottle A,Aylin P,Browne M.Monitoring changes in hospital standardised mortality ratios.BMJ. 2005;330:329.
5. Jarman B.In defence of the hospital standardized mortality ratio.Healthc Pap. 2008;8:37-42.
6. Heijink R,Koolman X,Pieter D,van der Veen A,Jarman B,Westert G.Measuring and explaining mortality in Dutch hospitals; the hospital standardized mortality rate between 2003 and 2005.BMC Health Serv Res. 2008;8:73.
7. Canadian Institute for Health Information. Hospital standardized mortality ratio (HSMR). http://www.cihi.ca/CIHI-ext-portal/pdf/internet/HSMR_TECH_NOTES_201005_en. Accessed February 11, 2011.
8. Penfold RB,Dean S,Flemons W,Moffatt M.Do hospital standardized mortality ratios measure patient safety? HSMRs in the Winnipeg Regional Health Authority.Healthc Pap. 2008;8:8-24.
9. Wen E,Sandoval C,Zelmer J,Webster G.Understanding and using the hospital standardized mortality ratio in Canada: challenges and opportunities.Healthc Pap. 2008;8:26-36.
10. McKinley J,Gibson D,Ardal S.Hospital standardized mortality ratio: the way forward in Ontario.Healthc Pap. 2008;8:43-49.
11. Zahn C,Baker M,MacNaughton J,Flemming C,Bell R.Hospital standardized mortality ratio is a useful burning platform.Healthc Pap. 2008;8:50-53.
12. Figler S.Data may reveal real issues.Healthc Pap. 2008;8:54-56.
13. Brien SE,Ghali WA.CIHI's hospital standardized mortality ratio: friend or foe?.Healthc Pap. 2008;8:57-61.
14. Hawkes N.Patient coding and the ratings game.BMJ. 2010;340:c2153.
15. Black N.Assessing the quality of hospitals.BMJ. 2010;340:c2066.
16. Lilford R,Pronovost P.Using hospital mortality rates to judge hospital performance: a bad idea that just won't go away.BMJ. 2010;340:c2016.
17. Agency for Healthcare Research and Quality. Mortality measurement. http://www.ahrq.gov/qual/mortality/. Accessed February 11, 2011.
18. Shahian DM,Wolf RE,Iezzoni LI,Kirle L,Normand SL.Variability in the measurement of hospital-wide mortality rates.N Engl J Med. 2010;363:2530-2539.
19. Commission for Healthcare Audit and Inspection. Investigation into Mid-Staffordshire NHS Foundation Trust. http://www.cqc.org.uk/_db/_documents/Investigation_into_Mid_Staffordshire_NHS_Foundation_Trust.pdf. Accessed February 11, 2011.
20. Shojania KG,Forster AJ.Hospital mortality: when failure is not a good measure of success.CMAJ. 2008;179:153-157.
21. Shahian DM,Blackstone EH,Edwards FH, et al.Cardiac surgery risk models: a position article.Ann Thorac Surg. 2004;78:1868-1877.
22. Shahian DM,Normand SL,Torchiana DF, et al.Cardiac surgery report cards: comprehensive review and statistical critique.Ann Thorac Surg. 2001;72:2155-2168.
23. Shahian DM,Torchiana DF,Normand SL.Implementation of a cardiac surgery report card: lessons from the Massachusetts experience.Ann Thorac Surg. 2005;80:1146-1150.
24. Shahian DM,Torchiana DF,Shemin RJ,Rawn JD,Normand SL.Massachusetts cardiac surgery report card: implications of statistical methodology.Ann Thorac Surg. 2005;80:2106-2113.
25. Krumholz HM,Wang Y,Mattera JA, et al.An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with an acute myocardial infarction.Circulation. 2006;113:1683-1692.
26. Krumholz HM,Wang Y,Mattera JA, et al.An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with heart failure.Circulation. 2006;113:1693-1701.
27. Krumholz HM,Brindis RG,Brush JE, et al.Standards for statistical models used for public reporting of health outcomes: an American Heart Association Scientific Statement from the Quality of Care and Outcomes Research Interdisciplinary Writing Group: cosponsored by the Council on Epidemiology and Prevention and the Stroke Council. Endorsed by the American College of Cardiology Foundation.Circulation. 2006;113:456-462.
28. Park RE,Brook RH,Kosecoff J, et al.Explaining variations in hospital death rates. Randomness, severity of illness, quality of care.JAMA. 1990;264:484-490.
29. Best WR,Cowper DC.The ratio of observed-to-expected mortality as a quality of care indicator in non-surgical VA patients.Med Care. 1994;32:390-400.
30. Jencks SF,Daley J,Draper D,Thomas N,Lenhart G,Walker J.Interpreting hospital mortality data. The role of clinical risk adjustment.JAMA. 1988;260:3611-3616.
31. Pitches DW,Mohammed MA,Lilford RJ.What is the empirical evidence that hospitals with higher-risk adjusted mortality rates provide poorer quality care? A systematic review of the literature.BMC Health Serv Res. 2007;7:91.
32. Thomas JW,Hofer TP.Research evidence on the validity of risk-adjusted mortality rate as a measure of hospital quality of care.Med Care Res Rev. 1998;55:371-404.
33. Birkmeyer JD,Dimick JB,Birkmeyer NJ.Measuring the quality of surgical care: structure, process, or outcomes?.J Am Coll Surg. 2004;198:626-632.
34. Dimick JB,Welch HG,Birkmeyer JD.Surgical mortality as an indicator of hospital quality: the problem with small sample size.JAMA. 2004;292:847-851.
35. Thomas JW,Holloway JJ,Guire KE.Validating risk-adjusted mortality as an indicator for quality of care.Inquiry. 1993;30:6-22.
36. Thomas JW,Hofer TP.Accuracy of risk-adjusted mortality rate as a measure of hospital quality of care.Med Care. 1999;37:83-92.
37. Hofer TP,Hayward RA.Identifying poor-quality hospitals. Can hospital mortality rates detect quality problems for medical diagnoses?.Med Care. 1996;34:737-753.
38. Zalkind DL,Eastaugh SR.Mortality rates as an indicator of hospital quality.Hosp Health Serv Adm. 1997;42:3-15.
39. Dubois RW,Rogers WH,Moxley JH,Draper D,Brook RH.Hospital inpatient mortality. Is it a predictor of quality?.N Engl J Med. 1987;317:1674-1680.
40. Lilford R,Mohammed MA,Spiegelhalter D,Thomson R.Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma.Lancet. 2004;363:1147-1154.
41. Blumberg MS.Comments on HCFA hospital death rate statistical outliers. Health Care Financing Administration.Health Serv Res. 1987;21:715-739.
42. Austin PC,Naylor CD,Tu JV.A comparison of a Bayesian vs. a frequentist method for profiling hospital performance.J Eval Clin Pract. 2001;7:35-45.
43. Iezzoni LI.The risks of risk adjustment.JAMA. 1997;278:1600-1607.
44. Rothberg MB,Morsi E,Benjamin EM,Pekow PS,Lindenauer PK.Choosing the best hospital: the limitations of public quality reporting.Health Aff (Millwood). 2008;27:1680-1687.
45. Shapiro MF,Park RE,Keesey J,Brook RH.The effect of alternative case-mix adjustments on mortality differences between municipal and voluntary hospitals in New York City.Health Serv Res. 1994;29:95-112.
46. Mohammed MA,Deeks JJ,Girling A, et al.Evidence of methodological bias in hospital standardised mortality ratios: retrospective database study of English hospitals.BMJ. 2009;338:b780.
47. Shahian DM,O'Brien SM,Filardo G, et al.The Society of Thoracic Surgeons 2008 cardiac surgery risk models: part 1-coronary artery bypass grafting surgery.Ann Thorac Surg. 2009;88 (1 suppl): S2-22.
48. Hannan EL,Kilburn H,Lindsey ML,Lewis R.Clinical versus administrative data bases for CABG surgery. Does it matter?.Med Care. 1992;30:892-907.
49. Hannan EL,Racz MJ,Jollis JG,Peterson ED.Using Medicare claims data to assess provider quality for CABG surgery: does it work well enough?.Health Serv Res. 1997;31:659-678.
50. Shahian DM,Silverstein T,Lovett AF,Wolf RE,Normand SL.Comparison of clinical and administrative data sources for hospital coronary artery bypass graft surgery report cards.Circulation. 2007;115:1518-1527.
51. Best WR,Khuri SF,Phelan M, et al.Identifying patient preoperative risk factors and postoperative adverse events in administrative databases: results from the Department of Veterans Affairs National Surgical Quality Improvement Program.J Am Coll Surg. 2002;194:257-266.
52. Romano PS,Roos LL,Luft HS,Jollis JG,Doliszny K.A comparison of administrative versus clinical data: coronary artery bypass surgery as an example. Ischemic Heart Disease Patient Outcomes Research Team.J Clin Epidemiol. 1994;47:249-260.
53. Romano PS.Asking too much of administrative data?.J Am Coll Surg. 2003;196:337-338.
54. Mack MJ,Herbert M,Prince S,Dewey TM,Magee MJ,Edgerton JR.Does reporting of coronary artery bypass grafting from administrative databases accurately reflect actual clinical outcomes?.J Thorac Cardiovasc Surg. 2005;129:1309-1317.
55. Chassin MR,Park RE,Lohr KN,Keesey J,Brook RH.Differences among hospitals in Medicare patient mortality.Health Serv Res. 1989;24:1-31.
56. Osswald BR,Tochtermann U,Schweiger P, et al.Minimal early mortality in CABG-simply a question of surgical quality?.Thorac Cardiovasc Surg. 2002;50:276-280.
57. Osswald BR,Blackstone EH,Tochtermann U,Thomas G,Vahl CF,Hagl S.The meaning of early mortality after CABG.Eur J Cardiothorac Surg. 1999;15:401-407.
58. Kfoury AG,French TK,Horne BD, et al.Incremental survival benefit with adherence to standardized heart failure core measures: a performance evaluation study of 2958 patients.J Card Fail. 2008;14:95-102.
59. Normand SLT,Glickman ME,Gatsonis CA.Statistical methods for profiling providers of medical care: issues and applications.J Am Stat Assoc. 1997;92:803-814.
60. Normand SLT,Shahian DM.Statistical and clinical aspects of hospital outcomes profiling.Stat Sci. 2007;22:206-226.
61. Shahian DM,Normand SL.Comparison of "risk-adjusted" hospital outcomes.Circulation. 2008;117:1955-1963.
62. Ivanov J,Tu JV,Naylor CD.Ready-made, recalibrated, or remodeled? Issues in the use of risk indexes for assessing mortality after coronary artery bypass graft surgery.Circulation. 1999;99:2098-2104.
63. Rosen HM,Green BA.The HCFA excess mortality lists: a methodological critique.Hosp Health Serv Adm. 1987;32:119-127.
64. Brennan TA,Leape LL,Laird NM, et al.Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I.N Engl J Med. 1991;324:370-376.
65. Hayward RA,Hofer TP.Estimating hospital deaths due to medical errors: preventability is in the eye of the reviewer.JAMA. 2001;286:415-420.
66. Holloway RG,Quill TE.Mortality as a measure of quality: implications for palliative and end-of-life care.JAMA. 2007;298:802-804.
67. Austin PC,Alter DA,Anderson GM,Tu JV.Impact of the choice of benchmark on the conclusions of hospital report cards.Am Heart J. 2004;148:1041-1046.
68. Austin PC.A comparison of Bayesian methods for profiling hospital performance.Med Decis Making. 2002;22:163-172.
69. Austin PC,Anderson GM.Optimal statistical decisions for hospital report cards.Med Decis Making. 2005;25:11-19.
70. Austin PC,Alter DA,Tu JV.The use of fixed- and random-effects models for classifying hospitals as mortality outliers: a Monte Carlo assessment.Med Decis Making. 2003;23:526-539.
71. Austin PC,Tu JV,Alter DA.Comparing hierarchical modeling with traditional logistic regression analysis among patients hospitalized with acute myocardial infarction: should we be analyzing cardiovascular outcomes data differently?.Am Heart J. 2003;145:27-35.
72. Normand SL,Wolf RE,Ayanian JZ,McNeil BJ.Assessing the accuracy of hospital clinical performance measures.Med Decis Making. 2007;27:9-20.
73. Localio AR,Hamory BH,Fisher AC,TenHave TR.The public release of hospital and physician mortality data in Pennsylvania. A case study.Med Care. 1997;35:272-286.
74. Gelman A,Hill JData Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge, England: Cambridge University Press; 2007:.
75. Green J,Wintfeld N.Report cards on cardiac surgeons. Assessing New York State's approach.N Engl J Med. 1995;332:1229-1232.
76. Omoigui NA,Miller DP,Brown KJ, et al.Outmigration for coronary bypass surgery in an era of public dissemination of clinical outcomes.Circulation. 1996;93:27-33.
77. Schneider EC,Epstein AM.Influence of cardiac-surgery performance reports on referral practices and access to care-a survey of cardiovascular specialists.N Engl J Med. 1996;335:251-256.
78. Dranove D,Kessler D,McClellan M,Satterthwaite M.Is more information better? The effects of "report cards" on health care providers.J Political Econ. 2003;111:555-588.
79. Burack JH,Impellizzeri P,Homel P,Cunningham JN.Public reporting of surgical mortality: a survey of New York State cardiothoracic surgeons.Ann Thorac Surg. 1999;68:1195-1200.
80. Moscucci M,Eagle KA,Share D, et al.Public reporting and case selection for percutaneous coronary interventions: an analysis from two large multicenter percutaneous coronary intervention databases.J Am Coll Cardiol. 2005;45:1759-1765.
81. Apolito RA,Greenberg MA,Menegus MA, et al.Impact of the New York State Cardiac Surgery and Percutaneous Coronary Intervention Reporting System on the management of patients with acute myocardial infarction complicated by cardiogenic shock.Am Heart J. 2008;155:267-273.
82. Resnic FS,Welt FG.The public health hazards of risk avoidance associated with public reporting of risk-adjusted outcomes in coronary intervention.J Am Coll Cardiol. 2009;53:825-830.
83. Narins CR,Dozier AM,Ling FS,Zareba W.The influence of public reporting of outcome data on medical decision making by physicians.Arch Intern Med. 2005;165:83-87.
84. Ash A.Identifying poor-quality hospitals with mortality rates. Often there's more noise than signal.Med Care. 1996;34:735-736.
85. Sutton AJ,Higgins JP.Recent developments in meta-analysis.Stat Med. 2008;27:625-650.
86. Rothman KJ,Greenland S,Lash TModern Epidemiology. Philadelphia, PA: Lippincott Williams & Wilkins; 2008:.
87. Egger M,Smith GD,Altman DGSystematic Reviews in Health Care: Meta-Analysis in Context. 2nd ed.London, England: BMJ Publishing; 2001.London, England: BMJ Publishing; 2001:.
88. Morgenstern H.Ecologic studies in epidemiology: concepts, principles, and methods.Annu Rev Public Health. 1995;16:61-81.
89. Snijders TAB,Bosker RJMultilevel Analysis. An Introduction to Basic and Advanced Multilevel Modeling. London, England: Sage; 1999:.
90. Pearl JCausality: Models, Reasoning, and Inference. Cambridge, England: Cambridge University Press; 2000:.
91. Hox JMultilevel Analysis: Technique and Applications. Mahwah, NJ: Lawrence Erlbaum; 2002:.
92. Lindley DV,Novick MR.The role of exchangeability in inference.Ann Stat. 1981;9:45-58.
93. Rosenthal GE,Shah A,Way LE,Harper DL.Variations in standardized hospital mortality rates for six common medical diagnoses: implications for profiling hospital quality.Med Care. 1998;36:955-964.
94. Rosenthal GE.Weak associations between hospital mortality rates for individual diagnoses: implications for profiling hospital quality.Am J Public Health. 1997;87:429-433.
95. NardoMSaisanaMSaltelliATarantolaSHoffmanAGiovanniniE. Handbook on Constructing Composite Indicators: Methodology and User Guide (2005 OECD Statistics Working Paper). http://www.oecd.org/officialdocuments/displaydocumentpdf/?cote=std/doc(2005)3&doclanguage=en. Accessed May 10, 2011.
96. Nunnally JC,Bernstein IHPsychometric Theory. 3rd ed.New York, NY: McGraw-Hill; 1994.New York, NY: McGraw-Hill; 1994:.
97. O'Brien SM,Shahian DM,Delong ER, et al.Quality measurement in adult cardiac surgery: part 2-statistical considerations in composite measure scoring and provider rating.Ann Thorac Surg. 2007;83 (4 suppl): S13-S26.
98. Shahian DM,Edwards FH,Ferraris VA, et al.Quality measurement in adult cardiac surgery: part 1-conceptual framework and measure selection.Ann Thorac Surg. 2007;83 (4 suppl): S3-S12.
99. Performance Measurement: Accelerating Improvement. Washington, DC: National Academies Press; 2006:.