A hierarchical modelling framework for identifying unusual performance in health care providers

Journal of the Royal Statistical Society. Series A: Statistics in Society

Volumn 170, Issue 4, 2007, Pages 865-890

  Ohlssen, David I ^a,b   Sharples, Linda D ^a   Spiegelhalter, David J ^a  

^a MRC BIOSTATISTICS UNIT   (United Kingdom)

^b NOVARTIS PHARMA AG   (Switzerland)

Author keywords

Cross validation; Divergent performance; Institutional comparisons; Multiple comparisons; Random effects

Indexed keywords

EID: 34848872593     PISSN: 09641998     EISSN: 1467985X     Source Type: Journal    
DOI: 10.1111/j.1467-985X.2007.00487.x     Document Type: Article

References (61)

1. Aitkin, M. Longford, N. (1986) Statistical modelling issues in school effectiveness studies (with discussion). J. R. Statist. Soc. A, 149, 1 42.
2. Austin, P. (2002) A comparison of Bayesian methods for profiling hospital performance. Med. Decisn Mak., 22, 163 172.
3. Barnett, B. Lewis, T. (1978) Outliers in Statistics. London : Wiley.
4. Bayarri, M. J. Berger, J. O. (2000) P values for composite null models. J. Am. Statist. Ass., 95, 1127 1142.
5. Bayarri, M. J. Berger, J. O. (2004) The interplay of Bayesian and frequentist analysis. Statist. Sci., 19, 58 80.
6. Bayarri, M. Morales, J. (2003) Bayesian measures of surprise for outlier detection. J. Statist. Planng Inf., 111, 3 22.
7. Benjamini, Y. Hochberg, Y. (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Statist. Soc. B, 57, 289 300.
8. Benjamini, Y. Yekutieli, D. (2001) The control of the false discovery rate in multiple testing under dependency. Ann. Statist., 29, 1165 1188.
9. Box, G. E. P. (1980) Sampling and Bayes' inference in scientific modelling and robustness (with discussion). J. R. Statist. Soc. A, 143, 383 430.
10. Bristol Royal Infirmary Inquiry (2001) The Inquiry into the management of care of children receiving complex heart surgery. (Available from http://www.bristol-inquiry.org.uk Gelman, A. (1998) General methods for monitoring convergence of iterative simulations. J. Computnl Graph. Statist., 7, 434 455.
11. Browne, W. J. Draper, D. (2006) A comparison of Bayesian and likelihood-based methods for fitting multilevel models (with discussion). Bayes. Anal., 1, 473 514.
12. Burgess, J., Christiansen, C. L., Michalak, S. Morris, C. (2000) Medical profiling: improving standards and risk adjustment using hierarchical models. J. Hlth Econ., 19, 291 309.
13. Commission for Health Improvement (2001) Report on the investigation into heart and lung transplantation at St George's Healthcare NHS Trust. Commission for Health Improvement, London Available from http://www.chi.nhs.uk/eng/ organisations/london/st_georges Smith, A. F. M. (1998) Quantitative refinements for comparisons of institutional performance. J. R. Statist. Soc. A, 161, 5 12.
14. DeLong, E. R., Peterson, E. D., DeLong, D. M., Muhlbaier, L. H., Hackett, S. Mark, D. B. (1997) Comparing risk-adjustment methods for provider profiling. Statist. Med., 16, 2645 2664.
15. Do, K.-A., Müller, P. Tang, F. (2005) A Bayesian mixture model for differential gene expression. Appl. Statist., 54, 627 644.
16. Draper, D. Gittoes, M. (2004) Statistical analysis of performance indicators in UK higher education. J. R. Statist. Soc. A, 167, 449 474.
17. DuMouchel, W. (1999) Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. Am. Statistn, 53, 177 190.
18. Efron, B., Tibshirani, R., Storey, J. D. Tusher, V. (2001) Empirical Bayes analysis of a microarray experiment. J. Am. Statist. Ass., 96, 1151 1160.
19. Farewell, V. T. (2002) Discussion on 'Commissioned analysis of surgical performance using routine data: lessons from the Bristol inquiry' (by D. J. Spiegelhalter, P. Aylin, N. G. Best, S. J. W. Evans and G. D. Murray). J. R. Statist. Soc. A, 165, 223.
20. Gelman, A. (2006) Prior distributions for variance parameters in hierarchical models. Bayes. Anal., 1, 515 534.
21. Gelman, A., Meng, X. L. Stern, H. (1996) Posterior predictive assessment of model fitness via realized discrepancies. Statist. Sin., 6, 733 760.
22. Genovese, C. R., Lazar, N. A. Nichols, T. (2002) Thresholding of statistical maps in functional neuroimaging using the false discovery rate. Neuroimage, 15, 870 878.
23. Goldstein, H. Spiegelhalter, D. J. (1996) League tables and their limitations: statistical issues in comparisons of institutional performance (with discussion). J. R. Statist. Soc. A, 159, 385 443.
24. Goldstein, H. Thomas, S. (1996) Using examination results as indicators of school and college performance. J. R. Statist. Soc. A, 159, 149 163.
25. Grigg, O., Farewell, V. Spiegelhalter, D. (2003) Use of risk-adjusted CUSUM and RSPRT charts for monitoring in medical contexts. Statist. Meth. Med. Res., 12, 147 170.
26. Healthcare Commission (2004) NHS performance ratings 2003/2004. Health Commission, London. (Available from http://www.healthcarecommission.org.uk/ InformationForServiceProviders/PerformanceRatings.)
27. Jasra, A., Holmes, C. Stephens, D. (2005) Markov chain Monte Carlo methods and the label switching problem in Bayesian mixture modelling. Statist. Sci., 20, 50 67.
28. Justel, A. Pena, D. (1996) Gibbs sampling will fail in outlier problems with strong masking. J. Computnl Graph. Statist., 5, 176 189.
29. Langford, I. H. Lewis, T. (1998) Outliers in multilevel data (with discussion). J. R. Statist. Soc. A, 161, 121 160.
30. Longford, N. T. (2001) Simulation-based diagnostics in random-coefficient models. J. R. Statist. Soc. A, 164, 259 273.
31. Longford, N. T. (2002) Discussion on 'Commissioned analysis of surgical performance using routine data: lessons from the Bristol inquiry' (by D. J. Spiegelhalter, P. Aylin, N. G. Best, S. J. W. Evans and G. D. Murray). J. R. Statist. Soc. A, 165, 223 224.
32. Marshall, C., Best, N., Bottle, A. Aylin, P. (2004) Statistical issues in the prospective monitoring of health outcomes across multiple units. J. R. Statist. Soc. A, 167, 541 559.
33. Marshall, E. C. Spiegelhalter, D. J. (1998a) Comparing institutional performance using Markov chain Monte Carlo methods. In Statistical Analysis of Medical Data: New Developments (eds B. Everitt G. Dunn). London : Arnold.
34. Marshall, E. C. Spiegelhalter, D. J. (1998b) Reliability of league tables of in vitro fertilisation clinics: retrospective analysis of birth rates. Br. Med. J., 316, 1701 1704.
35. Marshall, E. C. Spiegelhalter, D. J. (2003) Approximate cross-validatory predictive checks in disease mapping models. Statist. Med., 22, 1649 1660.
36. McCullagh, P. Nelder, J. A. (1989) Generalized Linear Models, 2nd edn. London : Chapman and Hall.
37. Mehrotra, D. V. Heyse, J. F. (2004) Use of the false discovery rate for evaluating clinical safety data. Statist. Meth. Med. Res., 13, 227 238.
38. Muller, P., Parmigiani, G. Rice, K. (2006) FDR and Bayesian multiple comparisons rules. Working Paper 115. Department of Biostatistics, Johns Hopkins University, Baltimore.
39. Nashef, S. A. M., Roques, F., Michel, P., Gauducheau, E., Lemeshow, S., Salamon, R. the EuroSCORE Study Group (1999) European system for cardiac operative risk evaluation (EuroSCORE). Eur. J. Card. Thorac. Surg., 16, 9 13.
40. New York State Department of Health (2004) Adult Cardiac Surgery in New York State 1998-2000. Albany : New York State Department of Health.
41. Normand, S., Glickman, M. Gatsonis, C. (1997) Statistical methods for profiling providers of medical care: issues and applications. J. Am. Statist. Ass., 92, 803 814.
42. Ohlssen, D. I., Sharples, L. D. Spiegelhalter, D. J. (2007) Flexible random-effects models using Bayesian semi-parametric models: applications to institutional comparisons. Statist. Med., 26, 2088 2112.
43. Paddock, S., Ridgeway, G., Lin, R. Louis, T. (2006) Flexible distributions for triple-goal estimates in two-stage hierarchical models. Computnl Statist. Data Anal., 50, 3243 3262.
44. Richardson, S. Green, P. J 1997) On Bayesian analysis of mixtures with an unknown number of components (with discussion). J. R. Statist. Soc. B, 59, 731 792 correction, 60 (1998 661.
45. Robins, J. M., van der Vaart, A. Ventura, V. (2000) Asymptotic distribution of P values in composite null models. J. Am. Statist. Ass., 95, 1143 1156.
46. Scott, J. G. Berger, J. O. (2006) An exploration of aspects of Bayesian multiple testing. J. Statist. Planng Inf., 136, 2144 2162.
47. Scottish Executive Health Department (2003) Clinical outcome indicators. Clinical Outcomes Working Group, Edinburgh. (Available from http://www.show. scot.nhs.uk/indicators/Outcomes/Main.htm.)
48. Senn, S. (2002) Discussion on 'Commissioned analysis of surgical performance using routine data: lessons from the Bristol inquiry' (by D. J. Spiegelhalter, P. Aylin, N. G. Best, S. J. W. Evans and G. D. Murray). J. R. Statist. Soc. A, 165, 224.
49. Sharples, L. (1990) Identification and accommodation of outliers in general hierarchical models. Biometrika, 77, 445 453.
50. Smith, J. P. Naylor, R. A. (2001) Dropping out of university: a statistical analysis of the probability of withdrawal for UK university students. J. R. Statist. Soc. A, 64, 389 405.
51. Spiegelhalter, D. J. (1999) Surgical audit: statistical lessons from Nightingale and Codman. J. R. Statist. Soc. A, 162, 45 58.
52. Spiegelhalter, D. J. (2005) Funnel plots for comparing institutional performance. Statist. Med., 24, 1185 1202.
53. Spiegelhalter, D. J., Aylin, P., Best, N. G., Evans, S. J. W. Murray, G. D. (2002a) Commissioned analysis of surgical performance using routine data: lessons from the Bristol inquiry (with discussion). J. R. Statist. Soc. A, 165, 191 231.
54. Spiegelhalter, D. J., Best, N. G., Carlin, B. P. van der Linde, A. (2002b) Bayesian measures of model complexity and fit (with discussion). J. R. Statist. Soc. B, 64, 583 639.
55. Steiner, S., Cook, R. Farewell, V. (1999) Monitoring paired binary surgical outcomes using cumulative sum charts. Statist. Med., 18, 69 86.
56. Storey, J. Tibshirani, R. (2003) Statistical significance for genome-wide studies. Proc. Natn. Acad. Sci. USA, 100, 9440 9445.
57. Thomas, N., Longford, N. Rolph, J. (1994) Empirical bayes methods for estimating hospital-specific mortality rates. Statist. Med., 13, 889 903.
58. Verdinelli, I. Wasserman, L. (1991) Bayesian analysis of outlier problems using the Gibbs sampler. Statist. Comput., 1, 105 117.
59. Weiss, R., Cho, M. Yanuzzi, M. (1999) On Bayesian calculations for mixture likelihoods and priors. Statist. Med., 18, 1555 1570.
60. Wright, S. P. (1992) Adjusted p-values for simultaneous inference. Biometrics, 48, 1005 1013.
61. Yau, K. K. W. Kuk, A. Y. C. (2002) Robust estimation in generalized linear mixed models. J. R. Statist. Soc. B, 64, 101 117.