League tables and their limitations: Statistical issues in comparisons of institutional performance

Journal of the Royal Statistical Society. Series A: Statistics in Society

Volumn 159, Issue 3, 1996, Pages 385-443

  Goldstein, Harvey ^a   Spiegelhalter, David J ^b  

^a UNIVERSITY OF LONDON   (United Kingdom)

^b MRC BIOSTATISTICS UNIT   (United Kingdom)

Author keywords

Markov chain Monte Carlo methods; Multilevel modelling; Physician profiling; Ranking; Risk shrinkage estimators; Stratification; Value added

Indexed keywords

EID: 0039862135     PISSN: 09641998     EISSN: None     Source Type: Journal    
DOI: 10.2307/2983325     Document Type: Article

References (58)

1. Aitkin, M. and Longford, N. (1986) Statistical modelling issues in school effectiveness studies (with discussion). J. R. Statist. Soc. A, 149, 1-42.
2. Annals of Thoracic Surgery (1994) Using outcomes data to improve clinical practice: building on models from cardiac surgery. Ann. Thor. Surg., 58, 1808-1884.
3. Audit Commission (1995) Local Authority Performance Indicators, vol. 1, Education, Social Services and Total Expenditure. London: Her Majesty's Stationery Office.
4. Bernardo, J. M. and Smith, A. F. M. (1994) Bayesian Theory. Chichester: Wiley.
5. Besag, J., Green, P. J., Higdon, D. and Mengersen, K. (1995) Bayesian computation and stochastic systems (with discussion). Statist. Sci., 10, 3-66.
6. British Medical Journal (1995) Hospital league tables derided. Br. Med. J., 311, 200.
7. Bumiller, E. (1995) Death-rate rankings shake New York cardiac surgeons. New York Times, Sept. 6th.
8. Charlton, J. R. H., Hartley, R. M., Silver, R. and Holland, W. W. (1983) Geographical variation in mortality from conditions amenable to medical intervention in England and Wales. Lancet, i, 691-696.
9. Clayton, D. G. and Kaldor, J. (1987) Empirical Bayes estimates of age-standardised relative risks for use in disease mapping. Biometrics, 43, 671-681.
10. de Courcy-Wheeler, R. H. B., Wolfe, C. D. A., Fitzgerald, A., Spencer, M., Goodman, J. D. S. and Gamsu, H. R. (1995) Use of the CRIB (clinical risk index for babies) score in prediction of neonatal mortality and morbidity. Arch. Dis. Chldhd, 73, F32-F36.
11. Department for Education (1994) GCSE and GCS A/AS level performance of candidates attempting two or more GCE A/AS levels in 1992/93. Statistical Bulletin 9/94. Department for Education, London.
12. _(1995a) GCSE to GCE A/AS Value Added: Briefing for Schools and Colleges. London: Department for Education.
13. _(1995b) GCSE and GCE A/AS level performance of candidates attempting two or more GCE A/AS levels in 1993/94. Statistical Bulletin 4/95. Department for Education, London.
14. Department of Health (1994) Public Health Common Data Set 1993. England, vol. 1. Guildford: Institute of Public Health.
15. Fitz-Gibbon, C. (1992) School effects at A level: genesis of an information system. In School Effectiveness Research Policy and Practice (eds D. Reynolds and P. Cuttance). London: Cassell.
16. Gatsonis, C. A., Epstein, A. M., Newhouse, J. P., Normand, S.-L. and McNeil, B. J. (1995) Variations in the utilisation of coronary angiography for elderly patients with an acute myocardial infarction: an analysis using hierarchical logistic regression. Med. Care, 33, 625-642.
17. Gatsonis, C. A., Normand, S.-L., Hiu, C. and Morris, C. (1993) Geographic variation of procedure utilisation: hierarchical model approach. Med. Care, 31, YS54-YS59.
18. Gelfand, A. E. and Smith, A. F. M. (1990) Sampling based approaches to calculating marginal densities. J. Am. Statist. Ass., 85, 398-409.
19. Gilks, W. R., Thomas, A. and Spiegelhalter, D. J. (1994) A language and program for complex Bayesian modelling. Statistician, 43, 169-177.
20. Goldstein, H. (1995) Multilevel Statistical Models. London: Arnold.
21. Goldstein, H. and Healy, M. J. R. (1995) The graphical presentation of a collection of means. J. R. Statist. Soc. A, 158, 175-177.
22. Goldstein, H. and Sammons, P. (1996) The influence of secondary and junior schools on sixteen year examination performance. In School Effectiveness and School Improvement. To be published.
23. Goldstein, H. and Thomas, S. (1996) Using examination results as indicators of school and college performance. J. R. Statist. Soc. A, 159, 149-163.
24. Gray, J., Jesson, D. and Goldstein, H. (1995) Changes in GCSE Examination Performance using Value Added Analysis over a Five Year Period in One Local Education Authority. Cambridge: Homerton College.
25. Green, J. and Wintfeld, N. (1995) Report cards on cardiac surgeons: assessing New York State's approach. New Engl. J. Med., 332, 1229-1232.
26. Hannan, E. L., Kilburn, H., Racz, M., Shields, E. and Chassin, M. R. (1994) Improving the outcomes of coronary artery bypass surgery in New York State. J. Am. Med. Ass., 271, 761-776.
27. Iezzoni, L. I. (1994) (ed.) Risk Adjustment for Measuring Health Care Outcomes. Ann Arbor: Health Administration Press.
28. Jencks, S., Daley, J., Draper, D., Thomas, N., Lenhart, G. and Walker, J. (1988) Interpreting hospital mortality data: the role of clinical risk adjustment. J. Am. Med. Ass., 260, 3611-3616.
29. Leyland, A. H. and Boddy, F. A. (1995) Measuring performance in hospital care: the example of length of stay in gynaecology. Eur. J. Publ. Hlth, to be published.
30. Leyland, A. H., Pritchard, C. W., McLoone, P. and Boddy, F. A. (1991) Measures of performance in Scottish maternity hospitals. Br. Med. J., 303, 389-393.
31. Lowry, S. (1988) Focus on performance indicators. Br. Med. J., 296, 992-994.
32. McArdle, C. S. and Hole, D. (1991) Impact and variability among surgeons on postoperative morbidity and mortality and ultimate survival. Br. Med. J., 302, 1501-1505.
33. McColl, A. J. and Gulliford, M. C. (1993) Population Health Outcome Indicators for the NHS: a Feasibility Study. London: Royal College of Physicians.
34. McKee, M. and Hunter, D. (1994) What can comparisons of hospital death rates tell us about the quality of care? In Outcomes into Clinical Practice (ed. T. Delamothe), pp. 108-115. London: British Medical Journal Press.
35. _(1995) Mortality league tables: do they inform or mislead? Qual. Hlth Care, 4, 5-12.
36. McPherson, K., Wennberg, J. E., Hovind, O. B. and Clifford, P. (1982) Small-area variations in the use of common surgical procedures: an international comparison of New England, England and Norway. New Engl. J. Med., 307, 1310-1314.
37. Morris, C. (1983) Parametric empirical Bayes inference: theory and applications (with discussion). J. Am. Statist. Ass., 79, 47-65.
38. National Health Service Executive (1995) The NHS Performance Guide 1994-1995. Leeds: National Health Service Executive.
39. National Health Service Management Executive (1992) The Health of the Nation: a Strategy for Health in England. London: Her Majesty's Stationery Office.
40. _(1993) Local Target Setting - a Discussion Paper. Leeds: National Health Service Management Executive.
41. New York State Department of Health (1993) Coronary Artery Bypass Surgery in New York State, 1990-1992. Albany: New York State Department of Health.
42. Normand, S.-L., Glickman, M. E. and Gatsonis, C. A. (1995) Statistical methods for profiling providers of medical care: issues and applications. Report HCP-1995-1. Department of Health Care Policy, Harvard Medical School, Boston.
43. Orchard, C. (1994) Comparing healthcare outcomes. Br. Med. J., 308, 1493-1496.
44. Organisation for Economic Co-operation and Development (1992) Education at a Glance. Paris: Organisation for Economic Co-operation and Development.
45. Rasbash, J. and Woodhouse, G. (1995) MLn Command Reference. London: Institute of Education.
46. Raudenbush, S. W. and Willms, J. D. (1995) The estimation of school effects. J. Educ. Behav. Statist., to be published.
47. Rogers, W. H., Draper, D., Kahn, K. L., Keeler, E. B., Rubenstein, L. V., Kosecoff, J. and Brook, R. H. (1990) Quality of care before and after implementation of the DRG-based prospective payment system. J. Am. Med. Ass., 264, 1989-1994.
48. Rowan, K. M., Kerr, J. H., McPherson, K., Short, A. and Vessey, M. P. (1993) Intensive Care Society's APACHE II study in Britain and Ireland-II: outcome comparisons of intensive care units after adjustment for case mix by the American APACHE II method. Br. Med. J., 307, 977-981.
49. Sanders, W. L. and Horn, S. (1994) The Tennessee Value-Added Assessment System (TVAAS): mixed model methodology in educational assessment. J. Persnnl Eval. Educ., 8, 299-311.
50. Scottish Office (1994) Clinical Outcome Indicators-1993. Edinburgh: Clinical Resource and Audit Group.
51. Silber, J. H., Rosenbaum, P. R. and Ross, R. N. (1995) Comparing the contributions of groups of predictors: which outcomes vary with hospital rather than patient characteristics? J. Am. Statist. Ass., 90, 7-18.
52. Smith, P. (1990) The use of performance indicators in the public sector. J. R. Statist. Soc. A, 153, 53-72.
53. Sowden, A. J., Deeks, J. J. and Sheldon, T. A. (1995) Volume and outcome in coronary artery bypass graft surgery: true association or artefact? Br. Med. J., 311, 151-155.
54. Spiegelhalter, D. J., Thomas, A., Best, N. G. and Gilks, W. R. (1995) BUGS: Bayesian Inference using Gibbs Sampling, Version 0.30. Cambridge: Medical Research Council Biostatistics Unit.
55. Steedman, J. (1980) Progress in Secondary Schools. London: National Children's Bureau.
56. Thomas, N., Longford, N. T. and Rolph, J. E. (1994) Empirical Bayes methods for estimating hospital-specific mortality rates. Statist. Med., 13, 889-903.
57. Woodhouse, G., Yang, M., Goldstein, H. and Rasbash, J. (1996) Adjusting for measurement error in multilevel analysis. J. R. Statist. Soc. A, 159, 201-212.
58. Yates, J. M. and Davidge, M. G. (1984) Can you measure performance? Br. Med. J., 288, 1935-1936.