Development and validation of an electronic medical record-based alert score for detection of inpatient deterioration outside the ICU

Journal of Biomedical Informatics

Volumn 64, Issue , 2016, Pages 10-19

  Kipnis, Patricia ^a,b   Turk, Benjamin J ^b   Wulf, David A ^b   LaGuardia, Juan Carlos ^b   Liu, Vincent ^b,c   Churpek, Matthew M ^d   Romero Brufau, Santiago ^e   Escobar, Gabriel J ^b,c  

^a Kaiser Foundation Health Plan Inc   (United States)

^b DIVISION OF RESEARCH   (United States)

^c KAISER PERMANENTE MEDICAL CENTER   (United States)

^d UNIVERSITY OF CHICAGO   (United States)

^e MAYO CLINIC   (United States)

Author keywords

Critical care; Deterioration; Early warning score; Electronic health records; Patient safety; Physiologic monitoring; Risk score

Indexed keywords

DETERIORATION; HEALTH RISKS; HOSPITALS; MEDICAL COMPUTING; PHYSIOLOGY; REGRESSION ANALYSIS;

CRITICAL CARE; EARLY WARNING SCORE; ELECTRONIC HEALTH RECORD; PATIENT SAFETY; PHYSIOLOGIC MONITORING; RISK SCORE;

INTENSIVE CARE UNITS;

ADULT; AGED; AREA UNDER THE CURVE; ARTICLE; AUTOMATED EARLY WARNING SCORE; COMORBIDITY; DISEASE SEVERITY; ELECTRONIC MEDICAL RECORD; FEMALE; HEALTH SERVICE; HEALTH STATUS; HOSPITALIZATION; HUMAN; INTENSIVE CARE UNIT; LOGISTIC REGRESSION ANALYSIS; MAJOR CLINICAL STUDY; MALE; OUTCOME ASSESSMENT; PREDICTIVE VALUE; PRIORITY JOURNAL; RECEIVER OPERATING CHARACTERISTIC; SCORING SYSTEM; SENSITIVITY AND SPECIFICITY; CALIFORNIA; CRITICAL VALUE; ELECTRONIC HEALTH RECORD; HOSPITAL PATIENT; MIDDLE AGED; VALIDATION STUDY; VITAL SIGN;

ADULT; AGED; CALIFORNIA; ELECTRONIC HEALTH RECORDS; FEMALE; HUMANS; INPATIENTS; INTENSIVE CARE UNITS; LABORATORY CRITICAL VALUES; MALE; MIDDLE AGED; VITAL SIGNS;

EID: 84988703622     PISSN: 15320464     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jbi.2016.09.013     Document Type: Article

References (41)

1. [1] Schein, R.M., Hazday, N., Pena, M., Ruben, B.H., Sprung, C.L., Clinical antecedents to in-hospital cardiopulmonary arrest. Chest 98 (1990), 1388–1392.
2. [2] Hournihan, F., Bishop, G., Hillman, K.M., Dauffurn, K., Lee, A., The medical emergency team: a new strategy to identify and intervene in high-risk surgical patients. Clin. Intens. Care 6 (1995), 269–272.
3. [3] Goldhill, D.R., Worthington, L.M., Mulcahy, A.J., Tarling, M.M., Quality of care before admission to intensive care. Deaths on the wards might be prevented. BMJ, 318, 1999, 195.
4. [4] Goldhill, D.R., Worthington, L., Mulcahy, A., Tarling, M., Sumner, A., The patient-at-risk team: identifying and managing seriously ill ward patients. Anaesthesia 54 (1999), 853–860.
5. [5] Subbe, C.P., Kruger, M., Rutherford, P., Gemmel, L., Validation of a modified early warning score in medical admissions. Q. J. Med. 94 (2001), 521–526.
6. [6] NHS, National Early Warning Score (NEWS), Standardising the assessment of acute-illness severity in the NHS, report of a working party, in: Physicians LRCo, (Ed.), Royal College of Physicians, London, 2012.
7. [7] Smith, G.B., Prytherch, D.R., Meredith, P., Schmidt, P.E., Featherstone, P.I., The ability of the national early warning score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation 84 (2013), 465–470.
8. [8] Prytherch, D.R., Smith, G.B., Schmidt, P.E., Featherstone, P.I., ViEWS–towards a national early warning score for detecting adult inpatient deterioration. Resuscitation 81 (2010), 932–937.
9. [9] Fraser, D.D., Singh, R.N., Frewen, T., The PEWS score: potential calling criteria for critical care response teams in children's hospitals. J. Crit. Care 21 (2006), 278–279.
10. [10] Parshuram, C.S., Hutchison, J., Middaugh, K., Development and initial validation of the bedside paediatric early warning system score. Crit. Care, 13, 2009, R135.
11. [11] McLellan, M.C., Connor, J.A., The cardiac children's hospital early warning score (C-CHEWS). J. Pediatr. Nurs. 28 (2013), 171–178.
12. [12] Rothman, M.J., Rothman, S.I., Beals, J.T., Development and validation of a continuous measure of patient condition using the electronic medical record. J. Biomed. Inform. 46 (2013), 837–848.
13. [13] Churpek, M., Yuen, T., Winslow, C., Robicsek, A., Meltzer, D., Gibbons, R., Edelson, D., Multicenter development and validation of a risk stratification tool for ward patients. Am. J. Respir. Crit. Care Med. 190 (2014), 649–655.
14. [14] Kollef, M.H., Chen, Y., Heard, K., LaRossa, G.N., Lu, C., Martin, N.R., Martin, N., Micek, S.T., Bailey, T., A randomized trial of real-time automated clinical deterioration alerts sent to a rapid response team. J. Hosp. Med. 9 (2014), 424–429.
15. [15] Evans, R.S., Kuttler, K.G., Simpson, K.J., Howe, S., Crossno, P.F., Johnson, K.V., Schreiner, M.N., Lloyd, J.F., Tettelbach, W.H., Keddington, R.K., Tanner, A., Wilde, C., Clemmer, T.P., Automated detection of physiologic deterioration in hospitalized patients. J. Am. Med. Inform. Assoc.: JAMIA 22 (2015), 350–360.
16. [16] Escobar, G.J., LaGuardia, J., Turk, B.J., Ragins, A., Kipnis, P., Draper, D., Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J. Hosp. Med. 7 (2012), 388–395.
17. [17] van Walraven, C., Escobar, G.J., Greene, J.D., Forster, A.J., The Kaiser Permanente inpatient risk adjustment methodology was valid in an external patient population. J. Clin. Epidemiol. 63 (2010), 798–803.
18. [18] Escobar, G., Greene, J., Scheirer, P., Gardner, M., Draper, D., Kipnis, P., Risk adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med. Care 46 (2008), 232–239.
19. [19] Escobar, G.J., Gardner, M., Greene, J.G., Draper, D., Kipnis, P., Risk-adjusting hospital mortality using a comprehensive electronic record in an integrated healthcare delivery system. Med. Care 51 (2013), 446–453.
20. [20] Teradata Integrated Data Warehouses. Available at: < http://www.teradata.com/products-and-services/Integrated-Data-Warehouse-Overview/?LangType=1033&LangSelect=true>.
21. [21] Japkowicz, N., Shaju, S., The class imbalance problem: a systematic study. Intell. Data Anal. 6:5 (2002), 429–449.
22. [22] Galar, M., Fernandez, A., Barrenechea, E., Bustince, H., Herrera, F., A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 42 (2012), 463–484.
23. [23] Liu, X.Y., Wu, J., Zhou, Z.H., Exploratory undersampling for class-imbalance learning. IEEE Trans. Syst. Man Cybernet. Part B, Cybernet.: Publ. IEEE Syst. Man Cybernet. Soc. 39 (2009), 539–550.
24. [24] Bishop, C.M., Pattern Recognition and Machine Learning. 2006, Springer, Berlin.
25. [25] Hornik, K., Meyer, D., Support vector machines in R. J. Stat. Softw. 15 (2006), 1–28.
26. [26] Allison, P.D., Discrete-time methods for the analysis of event histories. Sociol. Methodol. 13 (1982), 61–98.
27. [27] Allison, PD, (eds.) Logistic Regression Using SAS: Theory and Application, second ed., 2012.
28. [28] Landwehr, N., Hall, M., Frank, E., Logistic model trees. Mach. Learn. 59 (2005), 161–205.
29. [29] Rokach, L., Ensemble-based classifiers. Artif. Intell. Rev. 33 (2010), 1–39.
30. [30] Nagelkerke, N., A note on a general definition of the coefficient of determination. Biometrika 78 (1991), 691–692.
31. [31] Harrell, F.E. Jr., Lee, K.L., Califf, R.M., Pryor, D.B., Rosati, R.A., Regression modelling strategies for improved prognostic prediction. Stat. Med. 3 (1984), 143–152.
32. [32] Steyerberg, E.W., Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating New York. 2009, Springer, NY.
33. [33] Saito, T., Rehmsmeier, M., The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS ONE, 10, 2015, e0118432.
34. [34] Romero-Brufau, S., Huddleston, J.M., Escobar, G.J., Liebow, M., Why the C-statistic is not informative to evaluate early warning scores and what metrics to use. Crit. Care, 19, 2015, 285.
35. [35] Escobar, G.J., Greene, J.D., Gardner, M.N., Marelich, G.P., Quick, B., Kipnis, P., Intra-hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J. Hosp. Med. 6 (2011), 74–80.
36. [36] Smith, G.B., Prytherch, D.R., Schmidt, P.E., Featherstone, P.I., Review and performance evaluation of aggregate weighted ‘track and trigger’ systems. Resuscitation 77 (2008), 170–179.
37. [37] Schutt, R., O'Neil, C., Doing Data Science: Straight Talk from the Frontline. 2013, O'Reilly Media, Inc.
38. [38] Lantz, B., Machine Learning With R. 2013, Packt Publishing Ltd.
39. [39] Duckitt, R.W., Buxton-Thomas, R., Walker, J., Cheek, E., Bewick, V., Venn, R., Forni, L.G., Worthing physiological scoring system: derivation and validation of a physiological early-warning system for medical admissions. An observational, population-based single-centre study. Br. J. Anaesth. 98 (2007), 769–774.
40. [40] Pedersen, N.E., Oestergaard, D., Lippert, A., End points for validating early warning scores in the context of rapid response systems: a Delphi consensus study. Acta Anaesthesiol. Scand. 60:5 (2016), 616–622.
41. [41] Bates, D.W., Saria, S., Ohno-Machado, L., Shah, A., Escobar, G., Big data in health care: using analytics to identify and manage high-risk and high-cost patients. Health Aff. 33 (2014), 1123–1131.