Machine Learning and Decision Support in Critical Care

Proceedings of the IEEE

Volumn 104, Issue 2, 2016, Pages 444-466

  Johnson, Alistair E W ^a   Ghassemi, Mohammad M ^a   Nemati, Shamim ^b   Niehaus, Katherine E ^c   Clifton, David ^c   Clifford, Gari D ^d  

^a Massachusetts Institute of Technology Cambridge   (United States)

^b EMORY UNIVERSITY SCHOOL OF MEDICINE   (United States)

^c UNIVERSITY OF OXFORD   (United Kingdom)

^d GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Critical care; Feature extraction; Machine learning; Signal processing

Indexed keywords

DATABASE SYSTEMS; DECISION SUPPORT SYSTEMS; FEATURE EXTRACTION; FORECASTING; HUMAN RESOURCE MANAGEMENT; LEARNING SYSTEMS; MACHINE LEARNING; RISK ASSESSMENT; RISK MANAGEMENT; RISK PERCEPTION; SIGNAL PROCESSING;

ARTIFACT DETECTION; CLINICAL DATABASE; CLINICAL MONITORING; CRITICAL CARE; DECISION SUPPORTS; EVENT PREDICTION; MULTI-MODAL DATA; PREDICTION PROBLEM;

INFORMATION MANAGEMENT;

EID: 84962092181     PISSN: 00189219     EISSN: 15582256     Source Type: Journal    
DOI: 10.1109/JPROC.2015.2501978     Document Type: Article

References (208)

1. J.-L. Vincent, "Critical care-where have we been and where are we going," Crit. Care, vol. 17, p. S2, 2013.
2. P. Pronovost, D. Angus, T. R. Dorman, K. A. Dremsizov, and T. T. Young, "Physician staffing patterns and clinical outcomes in critically Ill patient: A systematic review," JAMA, vol. 288, no. 17, pp. 2151-2162, 2002.
3. R. Kane, T. Shamliyan, C. Mueller, S. Duval, and T. J. Wilt, "The association of registered nurse staffing levels and patient outcomes: Systematic review and meta-analysis," Med. Care, vol. 45, no. 12, pp. 1195-1204, Dec. 2007.
4. S. M. Pastores and V. Kvetan, "Shortage of intensive care specialists in the united states: Recent insights and proposed solutions," Revista Brasileira de terapia intensiva, vol. 27, no. 1, pp. 5-6, 2015.
5. N. A. Halpern and S. M. Pastores, "Critical care medicine in the United States 2000-2005: An analysis of bed numbers, occupancy rates, payer mix, costs," Crit. Care Med., vol. 38, no. 1, pp. 65-71, 2010.
6. O. Badawi et al., "Making big data useful for health care: A summary of the inaugural MIT critical data conference," JMIR Med. Inf., vol. 2, no. 2, p. e22, 2014.
7. G. F. Riley, "Administrative and claims records as sources of health care cost data," Med. Care, vol. 47, pp. S51-S55, 2009.
8. A. E. W. Johnson, A. Kramer, and G. D. Clifford, "Data preprocessing and mortality prediction: The Physionet/CinC 2012 challenge revisited," in Proc. Comput. Cardiol. Conf., 2014, vol. 41, pp. 157-160.
9. Centers for Medicare & Medicaid Services, "The Health Insurance Portability and Accountability Act of 1996 (HIPAA)," 1996. [Online]. Available: http://www.cms. hhs.gov/hipaa/
10. F. Caldicott, "Information: To share or not to share. The Information governance review," 2013.
11. D. C. Ince, L. Hatton, and J. Graham-Cumming, "The case for open computer programs," Nature, vol. 482, no. 7386, pp. 485-488, 2012.
12. R. B. Ness, Joint Policy Committee, "Influence of the HIPAA privacy rule on health research," JAMA, vol. 298, no. 18, pp. 2164-2170, 2007.
13. C. M. O'Keefe, "Privacy and the use of health data-reducing disclosure risk," Electron. J. Health Inf., vol. 3, no. 1, p. 5, 2008.
14. H. Office for Civil Rights, "Standards for privacy of individually identifiable health information. final rule," Fed. Register, vol. 67, no. 157, p. 53181, 2002.
15. P. Doyle, J. Lane, J. Theeuwes, and L. Zayatz, Confidentiality, Disclosure and Data Access: Theory and Practical Applications for Statistical Agencies. Amsterdam, The Netherlands: Elsevier Science, 2004.
16. I. Neamatullah et al., "Automated de-identification of free-text medical records," BMC Med. Inf. Decision Making, vol. 8, no. 1, p. 32, 2008.
17. S. N. Murphy et al., "Serving the enterprise and beyond with informatics for integrating biology and the bedside (i2b2)," J. Amer. Med. Inf. Assoc., vol. 17, no. 2, pp. 124-130, 2010.
18. C. Dwork, "Differential privacy," in Encyclopedia of Cryptography and Security. New York, NY, USA: Springer-Verlag, 2011, pp. 338-340.
19. N. Mohammed, X. Jiang, R. Chen, B. C. Fung, and L. Ohno-Machado, "Privacy-preserving heterogeneous health data sharing," J. Amer. Med. Inf. Assoc., vol. 20, no. 3, pp. 462-469, 2013.
20. A. Goldberger, L. Amaral, and L. Glass, "PhysioBank, PhysioToolkit, PhysioNet: Components of a new research resource for complex physiologic signals," Circulation, vol. 101, no. 23, pp. e215-e220, Jun. 2000.
21. M. Saeed, C. Lieu, G. Raber, and R. G. Mark, "MIMIC II: A massive temporal ICU patient database to support research in intelligent patient monitoring," Comput. Cardiol., vol. 29, pp. 641-644, 2002.
22. M. Saeed et al., "Multiparameter intelligent monitoring in intensive care (MIMIC II): A public-access intensive care unit database," Crit. Care Med., vol. 39, no. 5, pp. 952-960, May 2011.
23. U.S. Food and Drug Administration, "Registration Listing," Jun. 2015. [Online]. Available: http://www.fda.gov/ MedicalDevices/DeviceRegulationand Guidance/HowtoMarketYourDevice/ RegistrationandListing/ucm134495.htm
24. K. Lesh, S. Weininger, J. M. Goldman, B. Wilson, and G. Himes, "Medical device interoperability-assessing the environment," in Proc. Joint Workshop HCMDSS-MDPNP, 2007, pp. 3-12.
25. D. Charles, J. King, V. Patel, and M. F. Furukawa, "Adoption of electronic health record systems among U.S. non-federal acute care hospitals: 2008-2012," ONC Data Brief, no. 9, 2013.
26. N. Black and M. Payne, "Directory of clinical databases: Improving and promoting their use," Quality Safety Health Care, vol. 12, no. 5, pp. 348-352, 2003.
27. C. R. Cooke and T. J. Iwashyna, "Using existing data to address important clinical questions in critical care," Crit. Care Med., vol. 41, no. 3, p. 886, 2013.
28. T. J. Iwashyna, E. W. Ely, D. M. Smith, and K. M. Langa, "Long-term cognitive impairment and functional disability among survivors of severe sepsis," JAMA, vol. 304, no. 16, pp. 1787-1794, 2010.
29. J. M. Finney, A. S. Walker, T. E. Peto, and D. H. Wyllie, "An efficient record linkage scheme using graphical analysis for identifier error detection," BMC Med. Inf. Decision Making, vol. 11, no. 1, p. 7, 2011.
30. M. McShea, R. Holl, O. Badawi, R. R. Riker, and E. Silfen, "The EICU research institute-a collaboration between industry, health-care providers, academia," IEEE Eng. Med. Biol. Mag., vol. 29, no. 2, pp. 18-25, 2010.
31. J. E. Zimmerman, A. A. Kramer, D. S. McNair, and F. M. Malila, "Acute physiology and chronic health evaluation (apache) iv: Hospital mortality assessment for today's critically ill patients," Crit. Care Med., vol. 34, no. 5, pp. 1297-1310, 2006.
32. J. E. Zimmerman and A. A. Kramer, "Outcome prediction in critical care: The acute physiology and chronic health evaluation models," Current Opinion Crit. Care, vol. 14, pp. 491-497, 2008.
33. U.S. Dept. Health Human Services, "ICD 9 CM. The International Classification of Diseases. 9. Rev: Clinical Modification.; 1: Diseases: Tabular List.; 2: Diseases: Alphabetic Index.; 3: Procedures: Tabular List and Alphabetic Index," U.S. Government Printing Office, 1980.
34. K. J. O'malley et al., "Measuring diagnoses: ICD code accuracy," Health Services Res., vol. 40, no. 5p2, pp. 1620-1639, 2005.
35. J. P. Pestian et al., "A shared task involving multi-label classification of clinical free text," in Proc. Workshop BioNLP 2007: Biol. Transl. Clin. Lang. Process., 2007, pp. 97-104.
36. L. Bos and K. Donnelly, "Snomed-CT: The advanced terminology and coding system for ehealth," Stud. Health Technol. Inf., vol. 121, pp. 279-290, 2006.
37. P. L. Elkin et al., "Evaluation of the content coverage of snomed CT: Ability of snomed clinical terms to represent clinical problem lists," Mayo Clin. Proc., vol. 81, no. 6, pp. 741-748, 2006.
38. C. J. McDonald et al., "Loinc, a universal standard for identifying laboratory observations: A 5-year update," Clin. Chem., vol. 49, no. 4, pp. 624-633, 2003.
39. P. Whetzel et al., "Bioportal: Enhanced functionality via new web services from the national center for biomedical ontology to access and use ontologies in software applications," Nucleic Acids Res., vol. 39, pp. w541-w545, Jul. 2011.
40. J. D D'Amore et al., "Are meaningful use stage 2 certified EHRS ready for interoperability? Findings from the smart c-CDa collaborative," J. Amer. Med. Inf. Assoc., vol. 21, no. 6, pp. 1060-1068, 2014.
41. K. Nouira and A. Trabelsi, "Intelligent monitoring system for intensive care units," J. Med. Syst., vol. 36, no. 4, pp. 2309-2318, 2012.
42. M. Imhoff, M. Bauer, U. Gather, and D. Löhlein, "Statistical pattern detection in univariate time series of intensive care on-line monitoring data," Intensive Care Med., vol. 24, no. 12, pp. 1305-1314, 1998.
43. M. West, P. J. Harrison, and H. S. Migon, "Dynamic generalized linear models and Bayesian forecasting," J. Amer. Stat. Assoc., vol. 80, no. 389, pp. 73-83, 1985.
44. C. Becker and U. Gather, "The largest nonidentifiable outlier: A comparison of multivariate simultaneous outlier identification rules," Comput. Stat. Data Anal., vol. 36, no. 1, pp. 119-127, 2001.
45. S. Nizami, J. R. Green, and C. McGregor, "Implementation of artifact detection in critical care: A methodological review," IEEE Rev. Biomed. Eng., vol. 6, pp. 127-142, 2013.
46. C. L. Tsien and J. C. Fackler, "Poor prognosis for existing monitors in the intensive care unit," Crit. Care Med., vol. 25, no. 4, pp. 614-619, 1997.
47. M. C. Chambrin et al., "Multicentric study of monitoring alarms in the adult intensive care unit (ICU): A descriptive analysis," Intensive Care Med., vol. 25, no. 12, pp. 1360-1366, Dec. 1999.
48. Q. Li, R. G. Mark, and G. D. Clifford, "Robust heart rate estimation from multiple asynchronous noisy sources using signal quality indices and a Kalman filter," Physiol. Meas., vol. 29, no. 1, pp. 15-32, Jan. 2008.
49. G. D. Clifford, J. Behar, Q. Li, and I. Rezek, "Signal quality indices and data fusion for determining clinical acceptability of electrocardiograms," Physiol. Meas., vol. 33, no. 9, p. 1419, 2012.
50. J. Pan and W. J. Tompkins, "A real-time QRS detection algorithm," IEEE Trans. Biomed. Eng., vol. 32, no. 3, pp. 230-236, 1985.
51. P. S. Hamilton and W. J. Tompkins, "Quantitative investigation of QRS detection rules using the MIT/BIH arrhythmia database," IEEE Trans. Biomed. Eng., vol. BME-33, no. 12, pp. 1157-1165, 1986.
52. W. Zong, G. Moody, and D. Jiang, "A robust open-source algorithm to detect onset and duration of QRS complexes," in Proc. Comput. Cardiol., 2003, vol. 30, pp. 737-740.
53. J. Behar, J. Oster, Q. Li, and G. D. Clifford, "ECG signal quality during arrhythmia and its application to false alarm reduction," IEEE Trans. Biomed. Eng., vol. 60, no. 6, pp. 1660-1666, 2013.
54. C. Cortes and V. Vapnik, "Support-vector networks," Mach. Learn., vol. 20, no. 3, pp. 273-297, Sep. 1995.
55. Q. Li and G. D. Clifford, "Signal quality and data fusion for false alarm reduction in the intensive care unit," J. Electrocardiol., vol. 45, no. 6, pp. 596-603, 2012.
56. M. E. Tipping, "Sparse Bayesian learning and the relevance vector machine," J. Mach. Learn. Res., vol. 1, pp. 211-244, 2001.
57. E. Morgado et al., "Quality estimation of the electrocardiogram using cross-correlation among leads," Biomed. Eng. Online, vol. 14, no. 1, p. 59, 2015.
58. L. Breiman, J. Friedman, C. J. Stone, and R. A. Olshen, Classification and Regression Trees. Boca Raton, FL, USA: CRC Press, 1984.
59. J. R. Quinlan, C4. 5: Programs for Machine Learning. Amsterdam, The Netherlands: Elsevier, 2014.
60. W. W. Cohen, "Fast effective rule induction," in Proc. 12th Int. Conf. Mach. Learn., 1995, pp. 115-123.
61. A. Barachant, A. Andreev, and M. Congedo, "The Riemannian potato: An automatic and adaptive artifact detection method for online experiments using Riemannian geometry," in Proc. TOBI Workshop lV, 2013, pp. 19-20.
62. C. L. Tsien, I. S. Kohane, and N. McIntosh, "Building ICU artifact detection models with more data in less time," in Proc. AMIA Symp., 2001, p. 706.
63. M. Imhoff, S. Kuhls, U. Gather, and R. Fried, "Smart alarms from medical devices in the OR and ICU," Best Practice Res. Clin. Anaesthesiol., vol. 23, no. 1, pp. 39-50, 2009.
64. M. Cvach, "Monitor alarm fatigue: An integrative review," Biomed. Instrum. Technol., vol. 46, no. 4, pp. 268-277, 2012.
65. B. J. Drew et al., "Insights into the problem of alarm fatigue with physiologic monitor devices: A comprehensive observational study of consecutive intensive care unit patients," 2014.
66. W. Zong, G. Moody, and R. Mark, "Reduction of false arterial blood pressure alarms using signal quality assessment and relationships between the electrocardiogram and arterial blood pressure," Med. Biol. Eng. Comput., vol. 42, no. 5, pp. 698-706, 2004.
67. G. D. Clifford et al., "Computing in Cardiology," 2015; 42:273-276.
68. F. Plesinger, P. Klimes, J. Halamek, and P. Jurak, "False alarms in intensive care unit monitors: Detection of life-threatening arrhythmias using elementary algebra, descriptive statistics and fuzzy logic," in Proc. Comput. Cardiol. Conf., 2015, pp. 1-4.
69. C. H. Antink and S. Leonhardt, "Reducing false arrhythmia alarms using robust interval estimation and machine learning," in Proc. Comput. Cardiol. Conf., 2015, pp. 1-4.
70. S. Fallet, S. Yazdani, and J.-M. Vesin, "A multimodal approach to reduce false arrhythmia alarms in the intensive care unit," in Proc. Comput. Cardiol. Conf., 2015 pp. 1-4.
71. G. D. Clifford, W. J. Long, G. B. Moody, and P. Szolovits, "Robust parameter extraction for decision support using multimodal intensive care data," Philosoph. Trans. A, Math. Phys. Eng. Sci., vol. 367, no. 1887, pp. 411-429, Jan. 2009.
72. P. G. H. Metnitz et al., "SAPS 3-From evaluation of the patient to evaluation of the intensive care unit. Part 1: Objectives, methods and cohort description," Intensive Care Med., vol. 31, no. 10, pp. 1336-1344, Oct. 2005.
73. J. W. Tukey, "Exploratory data analysis," 1977.
74. V. Barnett and T. Lewis, Outliers in Statistical Data. New York, NY, USA: Wiley, 1994, vol. 3.
75. A. Fialho et al., "Disease-based modeling to predict fluid response in intensive care units," Methods Inf. Med., vol. 52, no. 6, pp. 494-502, 2013.
76. N. Aleks et al., "Probabilistic detection of short events, with application to critical care monitoring," in Proc. Adv. Neural Inf. Process. Syst., 2009, pp. 49-56.
77. J. A. Quinn, C. K. Williams, and N. McIntosh, "Factorial switching linear dynamical systems applied to physiological condition monitoring," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 9, pp. 1537-1551, 2009.
78. K. Georgatzis and C. K. Williams, "Discriminative switching linear dynamical systems applied to physiological condition monitoring," 2015. [Online]. Available: http://arxiv.org/abs/1504.06494
79. C. W. Hug, G. D. Clifford, and A. T. Reisner, "Clinician blood pressure documentation of stable intensive care patients: An intelligent archiving agent has a higher association with future hypotension," Crit. Care Med., vol. 39, no. 5, pp. 1006-1014, May 2011.
80. R. Snow, B. O'Connor, D. Jurafsky, and A. Y. Ng, "Cheap and fast-but is it good?: Evaluating non-expert annotations for natural language tasks," in Proc. Conf. Empirical Methods Natural Lang. Process., 2008, pp. 254-263.
81. T. Zhu, A. E. W. Johnson, J. Behar, and G. D. Clifford, "Crowd-sourced annotation of ECG signals using contextual information," Ann. Biomed. Eng., vol. 42, no. 4, pp. 871-884, 2014.
82. B.-U. Kohler, C. Hennig, and R. Orglmeister, "The principles of software QRS detection," IEEE Eng. Med. Biol. Mag., vol. 21, no. 1, pp. 42-57, 2002.
83. S. Jakob et al., "Detection of artifacts in monitored trends in intensive care," Comput. Methods Programs Biomed., vol. 63, no. 3, pp. 203-209, 2000.
84. P. Yang, G. A. Dumont, and J. M. Ansermino, "Sensor fusion using a hybrid median filter for artifact removal in intraoperative heart rate monitoring," J. Clin. Monitor. Comput., vol. 23, no. 2, pp. 75-83, 2009.
85. J. Allen and A. Murray, "Assessing ECG signal quality on a coronary care unit," Physiol. Meas., vol. 17, no. 4, p. 249, 1996.
86. W. Kaiser and M. Findeis, "Novel signal processing methods for exercise ECG," Proc. IJBEM, vol. 2, Special Issue on Electrocardiography, in Ischemic Heart Disease, 2000.
87. L. Chen, T. McKenna, A. Reisner, and J. Reifman, "Algorithms to qualify respiratory data collected during the transport of trauma patients," Physiol. Meas., vol. 27, no. 9, p. 797, 2006.
88. A. E. W. Johnson, J. Behar, F. Andreotti, G. D. Clifford, and J. Oster, "Multimodal heart beat detection using signal quality indices," Physiol. Meas., vol. 36, no. 8, p. 1665, 2015.
89. A. C. Harvey, Forecasting, Structural Time Series Models and the Kalman Filter. Cambridge, U.K.: Cambridge Univ. Press, 1990.
90. D. F. Sittig and M. Factor, "Physiologic trend detection and artifact rejection: A parallel implementation of a multi-state Kalman filtering algorithm," Comput. Methods Programs Biomed., vol. 31, no. 1, p. 1-10, 1990.
91. J. M. Feldman, M. H. Ebrahim, and I. Bar-Kana, "Robust sensor fusion improves heart rate estimation: Clinical evaluation," J. Clin. Monitor., vol. 13, no. 6, pp. 379-384, 1997.
92. M. H. Ebrahim, J. M. Feldman, and I. Bar-Kana, "A robust sensor fusion method for heart rate estimation," J. Clin. Monitor., vol. 13, no. 6, pp. 385-393, 1997.
93. L. Tarassenko et al., "Medical signal processing using the software monitor," in Proc. DERA/IEE Workshop Intell. Sensor Process., 2001, pp. 3/1-3/4.
94. S. Challa and D. Koks, "Bayesian and Dempster-Shafer fusion," Sadhana, vol. 29, no. 2, pp. 145-174, 2004.
95. T. Wartzek, C. Brueser, M. Walter, and S. Leonhardt, "Robust sensor fusion of unobtrusively measured heart rate," IEEE J. Biomed. Health Inf., vol. 18, pp. 654-660, 2013.
96. J. Oster et al., "Semi-supervised ECG beat classification and novelty detection based on switching Kalman filters," IEEE Trans. Biomed. Eng., vol. 62, no. 9, pp. 2125-2134, 2015.
97. D. M. Eddy, "Variations in physician practice: The role of uncertainty," Health Affairs, vol. 3, no. 2, pp. 74-89, 1984.
98. R. J. Little and D. B. Rubin, Statistical Analysis With Missing Data. New York, NY, USA: Wiley, 2014.
99. A. Vesin et al., "Reporting and handling missing values in clinical studies in intensive care units," Intensive Care Med., vol. 39, no. 8, pp. 1396-1404, 2013.
100. J. A. Sterne et al., "Multiple imputation for missing data in epidemiological and clinical research: Potential and pitfalls," BMJ, vol. 338, 2009. [Online]. Available: http:// dx.doi.org/10.1136/bmj.b2393
101. S. J. Shah et al., "Phenomapping for novel classification of heart failure with preserved ejection fraction," Circulation, vol. 131, no. 3, pp. 269-279, Jan. 2015.
102. A. K. Waljee et al., "Comparison of imputation methods for missing laboratory data in medicine," BMJ Open, vol. 3, no. 8, 2013, e002847.
103. S.-H. Kim, H.-J. Yang, S.-H. Kim, and G.-S. Lee, "Physiocover: Recovering the missing values in physiological data of intensive care units," Int. J. Contents, vol. 10, no. 2, pp. 47-58, 2014.
104. Q. Long and B. A. Johnson, "Variable selection in the presence of missing data: Resampling and imputation," Biostatistics, vol. 16, no. 3, pp. 596-610, Jan. 2015.
105. R. P. Moreno et al., "SAPS 3-From evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission," Intensive Care Med., vol. 31, no. 10, pp. 1345-1355, Oct. 2005.
106. J. L. Schafer and J. W. Graham, "Missing data: Our view of the state of the art," Psychol. Methods, vol. 7, no. 2, p. 147, 2002.
107. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood from incomplete data via the em algorithm," J. Roy. Stat. Soc. B, Methodol., vol. 39, pp. 1-38, 1977.
108. D. B. Rubin, "Multiple imputations in sample surveys-a phenomenological Bayesian approach to nonresponse," in Proc. Surv. Res. Methods Sec. Amer. Stat. Assoc., 1978, vol. 1, pp. 20-34.
109. D. B. Rubin, Multiple Imputation for Nonresponse in Surveys. New York, NY, USA: Wiley, 2004.
110. S. Chevret, S. Seaman, and M. Resche-Rigon, "Multiple imputation: A mature approach to dealing with missing data," Intensive Care Med., vol. 41, no. 2, pp. 348-350, 2015.
111. L. Clifton et al., "Gaussian process regression in vital-sign early warning systems," in Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., 2012, pp. 6161-6164.
112. T. A. Lasko, "Nonstationary Gaussian process regression for evaluating clinical laboratory test sampling strategies," in Proc. AAAI Conf. Artif. Intell., Jan. 2015, vol. 2015, pp. 1777-1783.
113. R. P. Dellinger et al., "Surviving sepsis campaign: International guidelines for management of severe sepsis and septic shock, 2012," Intensive Care Med., vol. 39, no. 2, pp. 165-228, 2013.
114. P. K. Lindenauer, T. Lagu, M.-S. Shieh, P. S. Pekow, and M. B. Rothberg, "Association of diagnostic coding with trends in hospitalizations and mortality of patients with pneumonia, 2003-2009," JAMA, vol. 307, no. 13, pp. 1405-1413, 2012.
115. Y. Halpern, Y. Choi, S. Horng, and D. Sontag, "Using anchors to estimate clinical state without labeled data," in Proc. AMIA Annu. Symp., 2014, vol. 2014, p. 606.
116. C. Elkan and K. Noto, "Learning classifiers from only positive and unlabeled data," in Proc. 14th ACM SIGKDD Int. Conf. Knowl. Disc. Data Mining, 2008, pp. 213-220.
117. T. Gjørup, H. S. Kelbaek, D. L. Nielsen, S. Kreiner, and J. Godtfredsen, "Reproducibility of electrocardiographic interpretation in patients with suspected myocardial infarction. A controlled study of the effect of a training trial," Tech. Rep. 1, 1994.
118. R. Bond et al., "Eye tracking technology and the 12-lead electrocardiogram: Where the experts look" in Proc. 39th Annu. Conf. Int. Soc. Computerized Electrocardiogr., 2014.
119. S. M. Salerno, P. C. Alguire, and H. S. Waxman, "Competency in interpretation of 12-lead electrocardiograms: A summary and appraisal of published evidence," Ann. Internal Med., vol. 138, no. 9, pp. 751-760, 2003.
120. G. Clifford, F. Azuaje, and P. McSharry, Advanced Methods and Tools for ECG Data Analysis. Boston, MA, USA: Artech House, 2006.
121. A. P. Dawid and A. M. Skene, "Maximum likelihood estimation of observer error-rates using the EM algorithm," J. Roy. Stat. Soc. C, Appl. Stat., vol. 28, no. 1, pp. 20-28, 1979.
122. V. Raykar et al., "Supervised learning from multiple experts: Whom to trust when everyone lies a bit," in Proc. 26th Annu. Int. Conf. Mach. Learn., 2009, pp. 889-896.
123. V. C. Raykar et al., "Learning from crowds," J. Mach. Learn. Res., pp. 1297-1322, 2010.
124. T. Zhu, J. Behar, T. Papastylianou, and G. D. Clifford, "CrowdLabel: A crowdsourcing platform for electrophysiology," in Proc. Comput. Cardiol. Conf., 2014, vol. 41, pp. 789-792.
125. P. Welinder and P. Perona, "Online crowdsourcing: Rating annotators and obtaining cost-effective labels," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Workshops, 2010, pp. 25-32.
126. P. Welinder, S. Branson, P. Perona, and S. J. Belongie, "The multidimensional wisdom of crowds," in Proc. Adv. Neural Inf. Process. Syst., 2010, pp. 2424-2432.
127. T. Zhu et al., "Fusing continuous-valued medical labels using a Bayesian model," 2015. [Online]. Available: http://arxiv.org/ abs/1503.06619
128. A. Holzinger, C. Röcker, and M. Ziefle, "From smart health to smart hospitals," in Smart Health, Lecture Notes in Computer Science. Berlin, Germany: Springer-Verlag, 2015, vol. 8700, pp. 1-20.
129. G. S. Power and D. A. Harrison, "Why try to predict ICU outcomes" Current Opinion Crit. Care, vol. 20, no. 5, pp. 544-549, 2014.
130. W. A. Knaus, J. E. Zimmerman, D. P. Wagner, E. A. Draper, and D. E. Lawrence, "APACHE-acute physiology and chronic health evaluation: A physiologically based classification system," Crit. Care Med., vol. 9, pp. 591-597, 1981.
131. W. A. Knaus, J. E. Zimmerman, D. P. Wagner, and E. A. Draper, "APACHE II: A severity of disease classification system," Crit. Care Med., vol. 13, pp. 818-829, 1985.
132. W. A. Knaus et al., "The APACHE III prognostic system. risk prediction of hospital mortality for critically ill hospitalized adults," Chest, vol. 100, no. 6, pp. 1619-1636, 1991.
133. J. R. LeGall et al., "A simplified acute physiology score for ICU patients," Crit. Care Med., vol. 12, no. 11, pp. 975-977, 1984.
134. J. R. LeGall, S. Lemeshow, and F. Saulnier, "A new simplified acute physiology score (SAPS-II) based on a European North-American multicenter study," JAMA, vol. 270, no. 24, pp. 2957-2963, Dec. 22, 1993.
135. S. Lemeshow, D. Teres, and H. Pastides, "A method for predicting survival and mortality of ICU patients using objectively derived weights," Crit. Care Med., vol. 13, pp. 519-525, 1985.
136. S. Lemeshow, D. Teres, and J. Klar, "Mortality probability model (MPM II) based on an international cohort of intensive care unit patients," JAMA, vol. 270, pp. 2478-2486, 1993.
137. T. L. Higgins et al., "Assessing contemporary intensive care unit outcome: An updated mortality probability admission model (MPM0-III)," Crit. Care Med., vol. 35, no. 3, pp. 827-835, Mar. 2007.
138. A. E. W. Johnson, A. A. Kramer, and G. D. Clifford, "A new severity of illness scale using a subset of acute physiology and chronic health evaluation data elements shows comparable predictive accuracy," Crit. Care Med., vol. 41, no. 7, pp. 1711-1718, 2013.
139. R. Dybowski, P. Weller, R. Chang, and V. Gant, "Prediction of outcome in critically ill patients using artificial neural network synthesised by genetic algorithm," Lancet, vol. 347, no. 9009, pp. 1146-1150, Apr. 1996.
140. G. Clermont, D. Angus, S. DiRusso, M. Griffin, and W. Linde-Zwirble, "Predicting hospital mortality for patients in the intensive care unit: A comparison of artificial neural networks with logistic regression models," Crit. Care Med., vol. 29, no. 2, pp. 291-296, 2001.
141. L. S. Wong and J. D. Young, "A comparison of ICU mortality prediction using the APACHE II scoring system and artificial neural networks," Anaesthesia, vol. 54, no. 11, pp. 1048-1054, Nov. 1999.
142. I. Silva, G. B. Moody, D. J. Scott, L. A. Celi, and R. G. Mark, "Predicting in-hospital mortality of ICU patients: The PhysioNet/ Computing in Cardiology Challenge 2012," Comput. Cardiol., vol. 39, pp. 245-248, 2012.
143. A. E. W. Johnson et al., "Patient specific predictions in the intensive care unit using a Bayesian ensemble," Comput. Cardiol., vol. 39, pp. 249-252, 2012.
144. L. Citi and R. Barbieri, "Physionet 2012 challenge: Predicting mortality of ICU patients using a cascaded SVM-GLM paradigm," Comput. Cardiol., vol. 39, pp. 257-260, 2012.
145. R. Pirracchio et al., "Mortality prediction in intensive care units with the super ICU learner algorithm (sicula): A population-based study," Lancet Respirat. Med., vol. 3, no. 1, pp. 42-52, 2015.
146. C. C. Mayer et al., "Selection of entropy-measure parameters for knowledge discovery in heart rate variability data," BMC Bioinf., vol. 15, p. S2, 2014.
147. S. Saria et al., "Integration of early physiological responses predicts later illness severity in preterm infants," Sci. Transl. Med., vol. 2, no. 48, pp. 48-65, 2010.
148. M. M. Ghassemi et al., "A data-driven approach to optimized medication dosing: A focus on heparin," Intensive Care Med., vol. 40, no. 9, pp. 1332-1339, 2014.
149. M. M. Ghassemi, M. B. Westover, R. G. Badawi, O. Mark, and S. Nemati, "Personalized medication dosing via sequential regression: A focus on heparin," Amer. J. Respirat. Crit. Care, 2015.
150. S. Nemati and R. Adams, "Identifying outcome-discriminative dynamics in multivariate physiological cohort time series," in Advanced State Space Methods for Neural and Clinical Data. Cambridge, U.K.: Cambridge Univ. Press, 2015, p. 283.
151. S. K. Nachimuthu, A. Wong, and P. J. Haug, "Modeling glucose homeostasis and insulin dosing in an intensive care unit using dynamic Bayesian networks," in Proc. AMIA Annu. Symp., 2010, vol. 2010, p. 532.
152. A. Smith, M. West, K. Gordon, M. Knapp, and I. Trimble, "Monitoring kidney transplant patients," The Statistician, vol. 32, pp. 46-54, 1983.
153. U. Gather, M. Imhoff, and R. Fried, "Graphical models for multivariate time series from intensive care monitoring," Stat. Med., vol. 21, no. 18, pp. 2685-2701, 2002.
154. C. Williams, J. Quinn, and N. Mcintosh, "Factorial switching Kalman filters for condition monitoring in neonatal intensive care," in Proc. Adv. Neural Inf. Process. Syst., 2005, pp. 1513-1520.
155. M. van der Heijden, M. Velikova, and P. J. Lucas, "Learning Bayesian networks for clinical time series analysis," J. Biomed. Inf., vol. 48, pp. 94-105, 2014.
156. Z. Xing, J. Pei, and E. Keogh, "A brief survey on sequence classification," ACM SIGKDD, vol. 12, no. 1, pp. 40-48, 2010.
157. T. W. Liao, "Clustering of time series data-A survey," Pattern Recognit., vol. 38, no. 11, pp. 1857-1874, 2005.
158. J. D. Rennie and T. Jaakkola, "Using term informativeness for named entity detection," in Proc. 28th Annu. Int. ACM SIGIR Conf. Res. Develop. Inf. Retrieval, 2005, pp. 353-360.
159. L. Lehman, T. Kyaw, G. Clifford, and R. Mark, "A temporal search engine for a massive multi-parameter clinical information database," in Proc. Comput. Cardiol., 2007, pp. 637-640.
160. L. Lehman, M. Saeed, G. Moody, and R. Mark, "Similarity-based searching in multi-parameter time series databases," in Proc. Comput. Cardiol., 2008, pp. 653-656.
161. D. M. Sow et al., "Real-time analysis for short-term prognosis in intensive care," IBM J. Res. Develop., vol. 56, no. 5, pp. 3:1-3:10, 2012.
162. S. Nemati and M. M. Ghassemi, "A fast and memory-efficient algorithm for learning and retrieval of phenotypic dynamics in multivariate cohort time series," in Proc. IEEE Int. Conf. Big Data, 2014, pp. 41-44.
163. R. K. Avent and J. D. Charlton, "A critical review of trend-detection methodologies for biomedical monitoring systems," Crit. Rev. Biomed. Eng., vol. 17, no. 6, pp. 621-659, 1990.
164. M. Saeed and R. Mark, "A novel method for the efficient retrieval of similar multiparameter physiologic time series using wavelet-based symbolic representations," in Proc. AMIA Annu. Symp., 2006, pp. 679-683.
165. M. Hauskrecht et al., "Outlier detection for patient monitoring and alerting," J. Biomed. Inf., vol. 46, no. 1, pp. 47-55, 2013.
166. S. Saria et al., "Learning individual and population level traits from clinical temporal data," in Proc. NIPS, Predictive Models in Personalized Medicine Workshop, 2010, DOI: 10.1.1.232.390.
167. P. Schulam, F. Wigley, and S. Saria, "Clustering longitudinal clinical marker trajectories from electronic health data: Applications to phenotyping and endotype discovery," 2015.
168. J. Ross and J. Dy, "Nonparametric mixture of Gaussian processes with constraints," in Proc. 30th Int. Conf. Mach. Learn., 2013, pp. 1346-1354.
169. L. H. Lehman et al., "A physiological time series dynamics-based approach to patient monitoring and outcome prediction," IEEE J. Biomed. Health Inf., vol. 19, no. 3, pp. 1068-1076, 2015.
170. B. M. Marlin, D. C. Kale, R. G. Khemani, and R. C. Wetzel, "Unsupervised pattern discovery in electronic health care data using probabilistic clustering models," in Proc. 2nd ACM SIGHIT Int. Health Inf. Symp., 2012, pp. 389-398.
171. S. Nemati and R. Adams, "Supervised learning in dynamic Bayesian networks," Tech. Rep., 2014.
172. Y. Bengio, A. Courville, and P. Vincent, "Representation learning: A review and new perspectives," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 8, pp. 1798-1828, Aug. 2013.
173. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning representations by back-propagating errors," Cogn. Model., vol. 323, pp. 533-536, 1988, DOI: 10.1038/323533a0.
174. E. Phadia, Prior Processes and Their Applications: Nonparametric Bayesian Estimation. New York, NY, USA: Springer-Verlag, 2013.
175. J. Shi and T. Choi, Gaussian Process Regression Analysis for Functional Data. London, U.K.: Chapman & Hall, 2011.
176. L. Clifton, D. A. Clifton, M. A. Pimentel, P. J. Watkinson, and L. Tarassenko, "Gaussian processes for personalized e-health monitoring with wearable sensors," IEEE Trans. Biomed. Eng., vol. 60, no. 1, pp. 193-197, 2013.
177. C. Orphanidou et al., "Signal quality indices for the electrocardiogram and photoplethysmogram: Derivation and applications to wireless monitoring," IEEE J. Biomed. Health Inf., vol. 19, no. 3, pp. 832-838, 2015.
178. D. A. Clifton, L. Clifton, S. Hugueny, D. Wong, and L. Tarassenko, "An extreme function theory for novelty detection," IEEE J. Sel. Top. Signal Process., vol. 7, no. 1, pp. 28-37, 2013.
179. L. Clifton, D. Clifton, and M. Pimentel, "Gaussian processes for personalised e-health monitoring with wearable sensors," IEEE Trans. Biomed. Eng., vol. 60, no. 1, pp. 193-197, 2013.
180. D. Clifton et al., "Pinning the tail on the distribution: A multivariate extension to the generalised Pareto distribution," in Proc. IEEE Int. Workshop Mach. Learn. Signal Process., 2011, pp. 1-6.
181. L. Clifton et al., "Probabilistic novelty detection with support vector machines," IEEE Trans. Reliab., vol. 63, no. 2, pp. 455-467, 2014.
182. D. Clifton, D. Wong, L. Clifton, R. Pullinger, and L. Tarassenko, "A large-scale clinical validation of an integrated monitoring system in the emergency department," IEEE Trans. Inf. Technol. Biomed., vol. 17, no. 4, pp. 835-877, 2013.
183. R. Duerichen et al., "Multitask Gaussian processes for multivariate physiological time-series analysis," IEEE Trans. Biomed. Eng., vol. 62, no. 1, pp. 314-322, 2015.
184. M. A. Pimentel, D. A. Clifton, and L. Tarassenko, "Gaussian process clustering for the functional characterisation of vital-sign trajectories," in Proc. IEEE Int. Workshop Mach. Learn. Signal Process., 2013, pp. 1-6.
185. J. Behar, A. E. Johnson, J. Oster, and G. Clifford, "An echo state neural network for foetal ECG extraction optimised by random search," in Proc. Adv. Neural Inf. Process. Syst., 2013.
186. M. Ghassemi, L. H. Lehman, J. Snoek, and S. Nemati, "Global optimization approaches for parameter tuning in biomedical signal processing: A focus of multi-scale entropy," Tech. Rep., 2014.
187. J. Snoek, H. Larochelle, and R. P. Adams, "Practical Bayesian optimization of machine learning algorithms," in Proc. Adv. Neural Inf. Process. Syst., 2012, pp. 2951-2959.
188. J. Fan, K. Mei, J. Peng, N. Zheng, and L. Gao, "Hierarchical classification of large-scale patient records for automatic treatment stratification," 2015.
189. D. C. Kale et al., "An examination of multivariate time series hashing with applications to health care," in Proc. IEEE Int. Conf. Data Mining, 2014, pp. 260-269.
190. T. A. Lasko, J. C. Denny, and M. A. Levy, "Computational phenotype discovery using unsupervised feature learning over noisy, sparse, irregular clinical data," PloS One, vol. 8, no. 6, 2013, Art. ID e66341.
191. J. Wiens, J. Guttag, and E. Horvitz, "Patient risk stratification for hospital-associated c. diff as a time-series classification task," in Proc. Adv. Neural Inf. Process. Syst., 2012.
192. T. F. Liu and R. W. Shafer, "Web resources for HIV type 1 genotypic-resistance test interpretation," Clin. Infectious Diseases, vol. 42, no. 11, pp. 1608-1618, 2006.
193. N. Beerenwinkel et al., "Geno2pheno: Estimating phenotypic drug resistance from HIV-1 genotypes," Nucleic Acids Res., vol. 31, no. 13, pp. 3850-3855, 2003.
194. M. C. Prosperi et al., "Investigation of expert rule bases, logistic regression, non-linear machine learning techniques for predicting response to antiretroviral treatment," Antivir. Ther., vol. 14, no. 3, pp. 433-442, 2009.
195. M. Laabei et al., "Predicting the virulence of MRSA from its genome sequence," Genome Res., vol. 24, no. 5, pp. 839-849, 2014, DOI: 10.1101/gr.165415.113.
196. L. Rishishwar, R. A. Petit, C. S. Kraft, and I. K. Jordan, "Genome sequence-based discriminator for vancomycin-intermediate staphylococcus aureus," J. Bacteriol., vol. 196, no. 5, pp. 940-948, 2014.
197. A. Drouin et al., "Learning interpretable models of phenotypes from whole genome sequences with the set covering machine," in Proc. Neural Inf. Process. Syst. Comput. Biol. Workshop, 2014.
198. K. E. Niehaus et al., "Machine learning for the prediction of antibacterial susceptibility in mycobacterium tuberculosis," in Proc. IEEE-EMBS Int. Conf. Biomed. Health Inf., 2014, pp. 618-621.
199. L.-W. Lehman, M. Saeed, W. Long, J. Lee, and R. Mark, "Risk stratification of ICU patients using topic models inferred from unstructured progress notes," in Proc. AMIA Annu. Symp. Proc., 2012, vol. 2012, p. 505.
200. M. Ghassemi et al., "A multivariate timeseries modeling approach to severity of illness assessment and forecasting in ICU with sparse, heterogeneous clinical data," 2015.
201. M. Ghassemi, R. Mark, and S. Nemati, "A visualization of evolving clinical sentiment using vector representations of clinical notes," Tech. Rep., 2015.
202. S. A. Collins et al., "Relationship between nursing documentation and patients' mortality," Amer. J. Crit. Care, vol. 22, no. 4, pp. 306-313, 2013.
203. K. D. Mandl and I. S. Kohane, "Escaping the EHR trap-The future of health IT," New England J. Med., vol. 366, no. 24, pp. 2240-2242, 2012.
204. L. A. Celi, R. G. Mark, D. J. Stone, and R. A. Montgomery, "'Big data' in the intensive care unit. Closing the data loop," Amer. J. Respirat. Crit. Care Med., vol. 187, no. 11, pp. 1157-1160, 2013.
205. A. Holzinger and I. Jurisica, "Knowledge discovery and data mining in biomedical informatics: The future is in integrative, interactive machine learning solutions," in Proc. Interactive Knowl. Disc. Data Mining Biomed. Inf., 2014, pp. 1-18.
206. J. Deng et al., "Imagenet: A large-scale hierarchical image database," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2009, pp. 248-255.
207. G. B. Moody and R. G. Mark, "The impact of the MIT-BIH arrhythmia database," IEEE Eng. Med. Biol. Mag., vol. 20, no. 3, pp. 45-50, 2001.
208. N. Black, "High-quality clinical databases: Breaking down barriers," Lancet, vol. 353, no. 9160, pp. 1205-1206, 1999.