Recalibration of risk prediction models in a large multicenter cohort of admissions to adult, general critical care units in the United Kingdom

Critical Care Medicine

Volumn 34, Issue 5, 2006, Pages 1378-1388

  Harrison, David A ^a   Brady, Anthony R ^a   Parry, Gareth J ^b   Carpenter, James R ^c   Rowan, Kathy ^a  

^a and London School of Hygiene Tropical Medicine   (United Kingdom)

^b UNIVERSITY OF SHEFFIELD   (United Kingdom)

^c LONDON SCHOOL OF HYGIENE AND TROPICAL MEDICINE   (United Kingdom)

Author keywords

Critical care; Intensive care units; Models; Risk adjustment; Severity of illness index; Statistical; Validation studies

Indexed keywords

APACHE; ARTICLE; CALIBRATION; COHORT ANALYSIS; DISEASE SEVERITY; INTENSIVE CARE UNIT; PRIORITY JOURNAL; PROSPECTIVE STUDY; RISK ASSESSMENT; THEORETICAL MODEL; UNITED KINGDOM; CLINICAL TRIAL; DECISION SUPPORT SYSTEM; DIAGNOSIS RELATED GROUP; FEMALE; HOSPITALIZATION; HUMAN; INTENSIVE CARE; MALE; MIDDLE AGED; MULTICENTER STUDY; REPRODUCIBILITY; STANDARD; STATISTICAL MODEL; VALIDATION STUDY;

APACHE; CALIBRATION; CRITICAL CARE; DECISION SUPPORT TECHNIQUES; DIAGNOSIS-RELATED GROUPS; ENGLAND; FEMALE; HUMANS; INTENSIVE CARE UNITS; MALE; MIDDLE AGED; MODELS, STATISTICAL; NORTHERN IRELAND; PROSPECTIVE STUDIES; REFERENCE STANDARDS; REPRODUCIBILITY OF RESULTS; RISK ADJUSTMENT; SEVERITY OF ILLNESS INDEX; WALES;

EID: 33646585120     PISSN: 00903493     EISSN: None     Source Type: Journal    
DOI: 10.1097/01.CCM.0000216702.94014.75     Document Type: Article

References (42)

1. Black N: Why we need observational studies to evaluate the effectiveness of health care. BMJ 1996; 312:1215-1218
2. UK Neonatal Staffing Study Group: A prospective evaluation of patient volume, staffing and workload in relation to risk-adjusted outcomes in a random, stratified sample of all UK neonatal intensive care units. Lancet 2002; 359:99-107
3. Knaus WA, Draper EA, Wagner DP, et al: APACHE II: A severity of disease classification system. Crit Care Med 1985; 13:818-829
4. Rowan KM: Outcome Comparisons of Intensive Care Units in Great Britain and Ireland Using the APACHE II Method [DPhil thesis]. Oxford, University of Oxford, 1992
5. Knaus WA, Wagner DP, Draper EA, et al: The APACHE III prognostic system: risk prediction of hospital mortality for critically ill hospitalized adults. Chest 1991; 100:1619-1636
6. Le Gall JR, Lemeshow S, Saulnier F: A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. JAMA 1993; 270:2957-2963
7. Lemeshow S, Teres D, Klar J, et al: Mortality probability models (MPM II) based on an international cohort of intensive care unit patients. JAMA 1993; 270:2478-2486
8. Livingston BM, MacKirdy FN, Howie JC, et al: Assessment of the performance of five intensive care scoring models within a large Scottish database. Crit Care Med 2000; 28:1820-1827
9. Beck DH, Smith GB, Pappachan JV, et al: External validation of the SAPS II, APACHE II and APACHE III prognostic models in South England: A multicentre study. Intensive Care Med 2003; 29:249-256
10. Fery-Lemonnier E, Landais P, Loirat P, et al: Evaluation of severity scoring systems in ICUs: Translation, conversion and definition ambiguities as a source of inter-observer variability in APACHE II, SAPS and OSF. Intensive Care Med 1995; 21:356-360
11. Hanley JA, McNeil BJ: The meaning and use of the area under the receiver operating characteristics (ROC) curve. Radiology 1982; 143:29-36
12. Hosmer DW, Lemeshow S: Goodness of fit tests for the multiple logistic regression model. Communications in Statistics 1980; A9:1043-1069
13. Van Houwelingen JC, Le Cessie S: Predictive value of statistical models. Stat Med 1990; 9:1303-1325
14. Miller ME, Hui SL, Tierney WM: Validation techniques for logistic regression models. Stat Med 1991; 10:1213-1226
15. Hosmer DW, Hosmer T, Le Cessie S, et al: A comparison of goodness-of-fit tests for the logistic regression model. Stat Med 1997; 16:965-980
16. Rowan K: Intensive Care Society has set up a centre for national audit. BMJ 1996; 313:1007-1008
17. Intensive Care National Audit & Research Centre (ICNARC). ICNARC Case Mix Programme Dataset Specification, Version 2.0. London, ICNARC, 1997
18. Harrison DA, Brady AR, Rowan K: Case mix, outcome and length of stay for admissions to adult, general critical care units in England, Wales and Northern Ireland: the Intensive Care National Audit & Research Centre Case Mix Programme Database. Crit Care 2004; 8:R99-R111; available online at http://ccforum.com/content/8/2/R99 (accessed June 20, 2005)
19. Harrell FE, Califf RM, Pryor DB, et al: Evaluating the yield of medical tests. JAMA 1982; 247:2543-2546
20. Bamber D: The area above the ordinal dominance graph and the area below the receiver operating characteristic graph. J Math Psychol 1975; 12:387-415
21. Shapiro AR: The evaluation of clinical predictions. N Engl J Med 1977; 296:1509-1514
22. Brier GW: Verification of forecasts expressed in terms of probability. Monthly Weather Review 1950; 75:1-3
23. Spiegelhalter DJ: Probabilistic prediction in patient management and clinical trials. Stat Med 1986; 5:421-433
24. Yates JF: External correspondence: Decomposition of the mean probability score. Organ Behav Hum Perform 1982; 30:132-156
25. Cox DR: Two further applications of a model for binary regression. Biometrika 1958; 45:562-565
26. Wunsch H, Brady AR, Rowan K: Impact of exclusion criteria from severity of disease scoring methods on outcomes in intensive care. J Crit Care 2004; 19:67-74
27. Young JD, Goldfrad C, Rowan K: Development and testing of a hierarchical method to code the reason for admission to intensive care units: The ICNARC Coding Method. Br J Anaesth 2001; 87:543-548
28. Steyerberg EW, Borsboom GJJM, van Houwelingen HC, et al: Validation and updating of predictive logistic regression models: A study on sample size and shrinkage. Stat Med 2004; 23:2567-2586
29. Efron B, Tibshirani RJ: An Introduction to the Bootstrap. London, Chapman & Hall, 1993
30. Beck DH, Smith GB, Pappachan JV: The effects of two methods for customising the original SAPS II model for intensive care patients from South England. Anaesthesia 2002; 57:785-793
31. Moreno R, Miranda DR, Fidler V, et al: Evaluation of two outcome prediction models on an independent database. Crit Care Med 1998; 26:50-61
32. Moreno R, Morais P: Outcome prediction in intensive care: Results of a prospective, multicentre, Portuguese study. Intensive Care Med 1997; 23:177-186
33. Apolone G, Bertolini G, D'Amico R, et al: The performance of SAPS II in a cohort of patients admitted to 99 Italian ICUs: Results from GiViTI: Gruppo Italiano per la Valutazione degli interventi in Terapia Intensiva. Intensive Care Med 1996; 22:1368-1378
34. Sicignano A, Giudici D: Customization of SAPS II for the assessment of severity in Italian ICU patients. ARCHIDIA: Archivio Diagnostico. Minerva Anestesiol 2000; 66:139-145
35. Metnitz PG, Valentin A, Vesely H, et al: Prognostic performance and customisation of the SAPS II: Results of a multicenter Austrian study: Simplified Acute Physiology Score. Intensive Care Med 1999; 25:192-197
36. Zimmerman JE, Wagner DP, Draper EA, et al: Evaluation of acute physiology and chronic health evaluation III predictions of hospital mortality in an independent database. Crit Care Med 1998; 26:1317-1326
37. Bastos PG, Sun X, Wagner DP, et al: Application of the APACHE III prognostic system in Brazilian intensive care units: A prospective multicenter study. Intensive Care Med 1996; 22:564-570
38. Vázquez Mata G, del Mar Jiménez Quintana M, Rivera Fernández R, et al: Objetivación de la gravedad mediante el sistema APACHE-III aplicado en España. Med Clin (Barc) 2001; 117:446-451
39. Rivera-Fernández R, Vázquez-Mata G, Bravo M, et al: The APACHE III prognostic system: Customized mortality predictions for Spanish ICU patients. Intensive Care Med 1998; 24:574-581
40. Livingston BM, Mackenzie SJ, MacKirdy FN, et al., on behalf of the Scottish Intensive Care Society Audit Group: Should the presedation Glasgow Coma Scale value be used when calculating Acute Physiology and Chronic Health Evaluation scores for sedated patients? Crit Care Med 2000; 28:389-394
41. Brady AR, Harrison D, Black S, et al., on behalf of the UK PICOS study group: Assessment and optimization of mortality prediction tools for admissions to pediatric intensive care in the United Kingdom. Pediatrics (In Press)
42. The SAPS III Outcome Research Group. Available online at http://www.saps3.org (accessed June 20, 2005)