Designing optimal mortality risk prediction scores that preserve clinical knowledge

Journal of Biomedical Informatics

Volumn 56, Issue , 2015, Pages 145-156

  Arzeno, Natalia M ^a   Lawson, Karla A ^b   Duzinski, Sarah V ^b   Vikalo, Haris ^a  

^a The University of Texas at Austin   (United States)

^b DELL CHILDREN S MEDICAL CENTER OF CENTRAL TEXAS   (United States)

Author keywords

Continuous risk score; ICU; Nonlinear features; Optimizable risk score; PRISM III; Prognostic model; SOFA

Indexed keywords

ADDITIVES; FORECASTING; HOSPITAL DATA PROCESSING; MATHEMATICAL TRANSFORMATIONS; POPULATION STATISTICS; PRISMS;

AREA UNDER ROC CURVE (AUC); BLOCK COORDINATE DESCENTS; NONLINEAR FEATURES; OPTIMIZATION FRAMEWORK; PREDICTIVE VARIABLES; PROGNOSTIC MODEL; RISK SCORE; SOFA;

INTENSIVE CARE UNITS;

ADOLESCENT; ADULT; ALGORITHM; AREA UNDER THE CURVE; ARTICLE; CHILD; CHILDHOOD MORTALITY; CONTROLLED STUDY; GROUPS BY AGE; HUMAN; INFANT; INTENSIVE CARE UNIT; LENGTH OF STAY; MEASUREMENT PRECISION; NEWBORN; NONLINEAR SYSTEM; PEDIATRIC RISK OF MORTALITY SCORE; POPULATION RISK; PREDICTION; PRIORITY JOURNAL; PROCESS OPTIMIZATION; RECEIVER OPERATING CHARACTERISTIC; RISK ASSESSMENT; RISK FACTOR; SCORING SYSTEM; SEQUENTIAL ORGAN FAILURE ASSESSMENT SCORE; TRAUMATIC BRAIN INJURY; VALIDATION STUDY; BRAIN INJURIES; FACTUAL DATABASE; HOSPITAL MORTALITY; INTENSIVE CARE; MEDICAL INFORMATICS; OUTCOME ASSESSMENT; PREDICTIVE VALUE; PROCEDURES; PROGNOSIS; REGRESSION ANALYSIS; STATISTICAL MODEL;

ADULT; ALGORITHMS; BRAIN INJURIES; CHILD; CRITICAL CARE; DATABASES, FACTUAL; HOSPITAL MORTALITY; HUMANS; INTENSIVE CARE UNITS; INTENSIVE CARE UNITS, PEDIATRIC; MEDICAL INFORMATICS; MODELS, STATISTICAL; OUTCOME ASSESSMENT (HEALTH CARE); PREDICTIVE VALUE OF TESTS; PROGNOSIS; REGRESSION ANALYSIS; RISK ASSESSMENT; ROC CURVE;

EID: 84938596230     PISSN: 15320464     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jbi.2015.05.021     Document Type: Article

References (73)

1. Zimmerman J.E., Kramer A.A., McNair D.S., Malila F.M. Acute physiology and chronic health evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients. Crit. Care Med. 2006, 34(5):1297-1310.
2. Knaus W., Wagner D., Draper E., Zimmerman J., Bergner M., Bastos P., Sirio C., Murphy D., Lotring T., Damiano A. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 1991, 100(6):1619-1636.
3. Moreno R., Metnitz P., Almeida E., Jordan B., Bauer P., Campos R., Iapichino G., Edbrooke D., Capuzzo M., Le Gall J.-R. SAPS 3-From evaluation of the patient to evaluation of the intensive care unit. Part 2: development of a prognostic model for hospital mortality at ICU admission. Inten. Care Med. 2005, 31(10):1345-1355.
4. Le Gall J., Lemeshow S., Saulnier F. A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. JAMA: J. Am. Med. Assoc. 1993, 270(24):2957-2963.
5. Pollack M.M., Patel K.M., Ruttimann U.E. PRISM III: an updated pediatric risk of mortality score. Crit. Care Med. 1996, 24(5):743-752.
6. Pollack M.M., Ruttimann U.E., Getson P.R. Pediatric risk of mortality (PRISM) score. Crit. Care Med. 1988, 16(11):1110-1116.
7. Streiner D.L. Breaking up is hard to do: the heartbreak of dichotomizing continuous data. Can. J. Psychiat. 2002, 47(3):262.
8. Royston P., Altman D.G., Sauerbrei W. Dichotomizing continuous predictors in multiple regression: a bad idea. Statist. Med. 2006, 25(1):127-141.
9. Marcin J.P., Pollack M.M. Review of the methodologies and applications of scoring systems in neonatal and pediatric intensive care. Pediat. Crit. Care Med. 2000, 1(1):20-27.
10. Minne L., Eslami S., de Keizer N., de Jonge E., de Rooij S.E., Abu-Hanna A. Statistical process control for validating a classification tree model for predicting mortality-a novel approach towards temporal validation. J. Biomed. Inf. 2012, 45(1):37-44.
11. Vincent J.L., Moreno R., Takala J., Willatts S., De Mendonça A., Bruining H., Reinhart C.K., Suter P.M., Thijs L.G. The SOFA (sepsis-related organ failure assessment) score to describe organ dysfunction/failure on behalf of the working group. on sepsis-related problems of the european society of intensive care medicine. Inten. Care Med. 1996, 22(7):707-710.
12. Vincent J.L., Ferreira F., Moreno R. Scoring systems for assessing organ dysfunction and survival. Crit. Care Clin. 2000, 16(2):353-366.
13. Scavarda D., Gabaudan C., Ughetto F., Lamy F., Imada V., Lena G., Paut O. Initial predictive factors of outcome in severe non-accidental head trauma in children. Child's Nerv. Syst.: ChNS 2010, 26(11):1555-1561.
14. Gemke R.J., van Vught J. Scoring systems in pediatric intensive care: PRISM III versus PIM. Inten. Care Med. 2002, 28(2):204-207.
15. Brady A.R., Harrison D., Black S., Jones S., Rowan K., Pearson G., Ratcliffe J., Parry G.J. Assessment and optimization of mortality prediction tools for admissions to pediatric intensive care in the united kingdom. Pediatrics 2006, 117(4):e733-742.
16. Karambelkar G., Mane S., Agarkhedkar S., Karambelkar R., Singhania S., Kadam S. The relevance of 24h PRISM III score in predicting mortality in pediatric intensive care unit. Int. J. Pharm. 2012, 3(4):214-219.
17. Cantais E., Paut O., Giorgi R., Viard L., Camboulives J. Evaluating the prognosis of multiple, severely traumatized children in the intensive care unit. Inten. Care Med. 2001, 27(9):1511-1517.
18. Bahloul M., Chaari A., Chabchoub I., Medhyoub F., Dammak H., Kallel H., Ksibi H., Haddar S., Rekik N., Chelly H., Bouaziz M. Outcome analysis and outcome predictors of traumatic head injury in childhood: analysis of 454 observations. J. Emerg., Trauma, Shock 2011, 4(2):198-206.
19. Volakli E.A., Sdougka M., Drossou-Agakidou V., Emporiadou M., Reizoglou M., Giala M. Short-term and long-term mortality following pediatric intensive care. Pediat. Int. 2012, 54(2):248-255.
20. Martha V.F., Garcia P.C.R., Piva J.P., Einloft P.R., Bruno F., Rampon V. Comparison of two prognostic scores (PRISM and PIM) at a pediatric intensive care unit. J. Pediat. 2005, 81(3):259-264.
21. Qureshi A.U., Ali A.S., Ahmad T.M. Comparison of three prognostic scores (PRISM, PELOD and PIM 2) at pediatric intensive care unit under Pakistani circumstances. Abbottabad: JAMC 2007, 19(2):49-53.
22. Laurer H.L., Meaney D.F., Margulies S.S., Mcintosh T.K. Modeling brain injury/trauma. Encyclopedia of the Human Brain 2002, 93-102. Academic Press, New York. V. Ramachandran (Ed.).
23. Maas A.I.R., Harrison-Felix C.L., Menon D., Adelson P.D., Balkin T., Bullock R., Engel D.C., Gordon W., Langlois-Orman J., Lew H.L., Robertson C., Temkin N., Valadka A., Verfaellie M., Wainwright M., Wright D.W., Schwab K. Standardizing data collection in traumatic brain injury. J. Neurot. 2011, 28(2):177-187.
24. Slater A., Shann F. The suitability of the pediatric index of mortality (PIM), PIM2, the pediatric risk of mortality (PRISM), and PRISM III for monitoring the quality of pediatric intensive care in australia and new zealand. Pediat. Crit. Care Med. 2004, 5(5):447-454. ANZICS Paediatric Study Group.
25. Tibby S., Taylor D., Festa M., Hanna S., Hatherill M., Jones G., Habibi P., Durward A., Murdoch I. A comparison of three scoring systems for mortality risk among retrieved intensive care patients. Arch. Dis. Childh. 2002, 87(5):421-425.
26. Bhadoria P., Bhagwat A.G. Severity scoring systems in paediatric intensive care units. Ind. J. Anaesth. 2008, 52(Suppl. (5)):663-675.
27. Thukral A., Lodha R., Irshad M., Arora N.K. Performance of pediatric risk of mortality (PRISM), pediatric index of mortality (PIM), and PIM2 in a pediatric intensive care unit in a developing country. Pediat. Crit. Care Med. 2006, 7(4):356-361.
28. Wells M., Riera-Fanego J.F., Luyt D.K., Dance M., Lipman J. Poor discriminatory performance of the pediatric risk of mortality (PRISM) score in a south african intensive care unit. Crit. Care Med. 1996, 24(9):1507-1513.
29. Goddard J.M. Pediatric risk of mortality scoring overestimates severity of illness in infants. Crit. Care Med. 1992, 20(12):1662-1665.
30. Ponce-Ponce De León A.L., Romero-Gutiérrez G., Valenzuela C.A., González-Bravo F.E. Simplified PRISM III score and outcome in the pediatric intensive care unit. Pediat. Int. 2005, 47(1):80-83.
31. Baker S.P., O'Neill B., Haddon W., Long W.B. The injury severity score: a method for describing patients with multiple injuries and evaluating emergency care. J. Trauma 1974, 14(3):187-196.
32. Slater A., Shann F., Pearson G. PIM2: a revised version of the paediatric index of mortality. Intens. Care Med. 2003, 29(2):278-285.
33. Marcin J.P., Pollack M.M. Triage scoring systems, severity of illness measures, and mortality prediction models in pediatric trauma. Crit. Care Med. 2002, 30(11 Suppl.):S457-467.
34. Shann F., Pearson G., Slater A., Wilkinson K. Paediatric index of mortality (PIM): a mortality prediction model for children in intensive care. Intens. Care Med. 1997, 23(2):201-207.
35. Minne L., Abu-Hanna A., de Jonge E. Evaluation of SOFA-based models for predicting mortality in the ICU: a systematic review. Crit. Care 2008, 12(6):R161.
36. Hwang S.Y., Lee J.H., Lee Y.H., Hong C.K., Sung A.J., Choi Y.C. Comparison of the sequential organ failure assessment, acute physiology and chronic health evaluation II scoring system, and trauma and injury severity score method for predicting the outcomes of intensive care unit trauma patients. Am. J. Emerg. Med. 2012, 30(5):749-753.
37. Toma T., Abu-Hanna A., Bosman R.-J. Discovery and inclusion of SOFA score episodes in mortality prediction. J. Biomed. Inf. 2007, 40(6):649-660.
38. Fueglistaler P., Amsler F., Schüepp M., Fueglistaler-Montali I., Attenberger C., Pargger H., Jacob A.L., Gross T. Prognostic value of sequential organ failure assessment and simplified acute physiology II score compared with trauma scores in the outcome of multiple-trauma patients. Am. J. Surg. 2010, 200(2):204-214.
39. Ho K.M. Combining sequential organ failure assessment (SOFA) score with acute physiology and chronic health evaluation (APACHE) II score to predict hospital mortality of critically ill patients. Anaesth. Intens. Care 2007, 35(4):515-521.
40. Breslow M.J., Badawi O. Severity scoring in the critically ill: part 1-interpretation and accuracy of outcome prediction scoring systems. Chest 2012, 141(1):245-252. http://dx.doi.org/10.1378/chest.11-0330. 10.1378/chest.11-0330.
41. Lemeshow S., Teres D., Klar J., Avrunin J.S., Gehlbach S.H., Rapoport J. Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients. JAMA 1993, 270(20):2478-2486.
42. Higgins T.L., Teres D., Copes W.S., Nathanson B.H., Stark M., Kramer A.A. Assessing contemporary intensive care unit outcome: an updated Mortality Probability Admission Model (MPM0-III). Crit. Care Med. 2007, 35(3):827-835. 10.1097/01.CCM.0000257337.63529.9F.
43. Brinkman S., Abu-Hanna A., de Jonge E., de Keizer N.F. Prediction of long-term mortality in ICU patients: model validation and assessing the effect of using in-hospital versus long-term mortality on benchmarking. Intens. Care Med. 2013, 39(11):1925-1931. 10.1007/s00134-013-3042-5.
44. Keegan M.T., Gajic O., Afessa B. Severity of illness scoring systems in the intensive care unit. Crit. Care Med. 2011, 39(1):163-169. 10.1097/CCM.0b013e3181f96f81.
45. Vincent J.-L., Moreno R. Clinical review: scoring systems in the critically ill. Crit. Care (London, England) 2010, 14(2):207. 10.1186/cc8204.
46. Knox D.B., Lanspa M.J., Pratt C.M., Kuttler K.G., Jones J.P., Brown S.M. Glasgow coma scale score dominates the association between admission sequential organ failure assessment score and 30-day mortality in a mixed intensive care unit population. J. Crit. Care 2014, 29(5):780-785. 10.1016/j.jcrc.2014.05.009.
47. Keegan M.T., Gajic O., Afessa B. Comparison of APACHE III, APACHE IV, SAPS 3, and MPM0iii and influence of resuscitation status on model performance. Chest 2012, 142(4):851-858. 10.1378/chest.11-2164.
48. N. Peek, A. Abu-Hanna. Clinical prognostic methods (online).
49. Courville X.F., Koval K.J., Carney B.T., Spratt K.F. Early prediction of posttraumatic in-hospital mortality in pediatric patients. J. Pediat. Orthoped. 2009, 29(5):439-444.
50. Gortzis L.G., Sakellaropoulos F., Ilias I., Stamoulis K., Dimopoulou I. Predicting ICU survival: a meta-level approach. BMC Health Serv. Res. 2008, 8(1):157. 10.1186/1472-6963-8-157.
51. L.-W. Lehman, M. Saeed, W. Long, J. Lee, R. Mark, Risk stratification of ICU patients using topic models inferred from unstructured progress notes, in: AMIA Annual Symposium Proceedings 2012, 2012, pp. 505-511.
52. Ruttimann U.E., Pollack M.M. A time-series approach to predict outcome from pediatric intensive care. Comp. Biomed. Res., Int. J. 1993, 26(4):353-372.
53. Burd R.S., Madigan D. The impact of injury coding schemes on predicting hospital mortality after pediatric injury. Acad. Emerg. Med. 2009, 16(7):639-645.
54. Saria S., Rajani A.K., Gould J., Koller D., Penn A.A. Integration of early physiological responses predicts later illness severity in preterm infants. Sci. Transl. Med. 2010, 2(48). 48ra65-48ra65.
55. Paetz J. Finding optimal decision scores by evolutionary strategies. Artif. Intell. Med. 2004, 32(2):85-95.
56. J. Sun, J. Hu, Combining knowledge and data driven insights for identifying risk factors using electronic health records, in: AMIA Annual Symposium Proceedings 2012, 2012, pp. 901-910.
57. Pivovarov R., Elhadad N. A hybrid knowledge-based and data-driven approach to identifying semantically similar concepts. J. Biomed. Inf. 2012, 45(3):471-481.
58. Hastie T., Tibshirani R., Friedman J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction 2009, Springer, New York, NY. second ed.
59. Boyd S., Vandenberghe L. Convex Optimization 2009, Cambridge Univ Press.
60. Hilden J., Glasziou P. Regret graphs, diagnostic uncertainty and Youden's index. Statist. Med. 1996, 15(10):969-986.
61. Perkins N.J., Schisterman E.F. The inconsistency of optimal cutpoints obtained using two criteria based on the receiver operating characteristic curve. Am. J. Epidemiol. 2006, 163(7):670-675.
62. Medlock S., Ravelli A.C.J., Tamminga P., Mol B.W.M., Abu-Hanna A. Prediction of mortality in very premature infants: a systematic review of prediction models. PloS One 2011, 6(9):e23441.
63. Brier G.W. Verification of forecasts expressed in terms of probability. Month. Weath. Rev. 1950, 78(1):1-3.
64. Platt J.C. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in Large-Margin Classifiers 1999, 61-74. MIT Press.
65. S.L. Murphy, J.Q. Xu, K.D. Kochanek, Deaths: Preliminary Data for 2010, National Vital Statistics Reports, vol. 60 (4), National Center for Health Statistics, Hyattsville, MD.
66. M. Faul, L. Xu, M.M. Wald, V.G. Coronado, Traumatic brain injury in the united states: emergency department visits, hospitalizations and deaths 2002-2006, 2010.
67. Tibshirani R. Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc., Ser. B 1994, 58:267-288.
68. Zou H., Hastie T. Regularization and variable selection via the elastic net. J. Roy. Statist. Soc., Ser. B 2005, 67:301-320.
69. T. Hastie, R. Tibshirani, G. Sherlock, M. Eisen, P. Brown, D. Botstein, Imputing Missing Data for Gene Expression Arrays, Tech. rep., Stanford University, 1999.
70. Tipping M.E., Bishop C.M. Probabilistic principal component analysis. J. Roy. Statist. Soc., Ser. B (Statist. Methodol.) 2002, 61(3):611-622.
71. Nelson Textbook of Pediatrics 2004, Elsevier. 17th ed. R.E. Behrman, R.M. Kiegman, H.B. Jenson (Eds.).
72. Saeed M., Villarroel M., Reisner A.T., Clifford G., Lehman L.-W., Moody G., Heldt T., Kyaw T.H., Moody B., Mark R.G. Multiparameter intelligent monitoring in intensive care II (MIMIC-II): a public-access intensive care unit database. Crit. Care Med. 2011, 39:952-960.
73. Goldberger A.L., Amaral L.A.N., Glass L., Hausdorff J.M., Ivanov P.C., Mark R.G., Mietus J.E., Moody G.B., Peng C.-K., Stanley H.E. PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation 2000, 101(23):e215-e220.