Recurrent Neural Networks for Multivariate Time Series with Missing Values

Scientific Reports

Volumn 8, Issue 1, 2018, Pages

  Che, Zhengping ^a   Purushotham, Sanjay ^a   Cho, Kyunghyun ^b   Sontag, David ^c   Liu, Yan ^a  

^a Department of Computer Science ^*  (United States)

^b POLYTECHNIC UNIVERSITY   (United States)

^c USA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CLASSIFICATION; LEARNING; MASKING; NERVOUS SYSTEM; PREDICTION; TIME SERIES ANALYSIS; DEEP LEARNING; GATED RECURRENT UNIT NETWORK;

EID: 85045746406     PISSN: None     EISSN: 20452322     Source Type: Journal    
DOI: 10.1038/s41598-018-24271-9     Document Type: Article

References (46)

1. Rubin, D. B. Inference and missing data. Biom. 63, 581-592 (1976).
2. Johnson, A. et al. Mimic-iii, a freely accessible critical care database. Sci. Data (2016).
3. Schafer, J. L. & Graham, J. W. Missing data: our view of the state of the art. Psychol. methods (2002).
4. Kreindler, D. M. & Lumsden, C. J. The effects of the irregular sample and missing data in time series analysis. Nonlinear Dyn. Syst. Analysis for Behav. Sci. Using Real Data (2012).
5. De Boor, C., De Boor, C., Mathématicien, E.-U., De Boor, C. & De Boor, C. A practical guide to splines 27 (Springer-Verlag, New York, 1978).
6. Mondal, D. & Percival, D. B. Wavelet variance analysis for gappy time series. Annals Inst. Stat. Math. 62, 943-966 (2010).
7. Rehfeld, K., Marwan, N., Heitzig, J. & Kurths, J. Comparison of correlation analysis techniques for irregularly sampled time series. Nonlinear Process. Geophys. 18 (2011).
8. Garca-Laencina, P. J., Sancho-Gómez, J.-L. & Figueiras-Vidal, A. R. Pattern classification with missing data: a review. Neural Comput. Appl. 19 (2010).
9. Mazumder, R., Hastie, T. & Tibshirani, R. Spectral regularization algorithms for learning large incomplete matrices. J. machine learning research 11, 2287-2322 (2010).
10. Koren, Y., Bell, R. & Volinsky, C. Matrix factorization techniques for recommender systems. Comput. 42 (2009).
11. White, I. R., Royston, P. & Wood, A. M. Multiple imputation using chained equations: issues and guidance for practice. Stat. medicine 30, 377-399 (2011).
12. Azur, M. J., Stuart, E. A., Frangakis, C. & Leaf, P. J. Multiple imputation by chained equations: what is it and how does it work? Int. journal methods psychiatric research 20, 40-49 (2011).
13. Wells, B. J., Chagin, K. M., Nowacki, A. S. & Kattan, M. W. Strategies for handling missing data in electronic health record derived data. EGEMS 1 (2013).
14. Hochreiter, S. & Schmidhuber, J. Long short-term memory. Neural computation 9, 1735-1780 (1997).
15. Cho, K. et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP, 1724-1734 (2014).
16. Bahdanau, D., Cho, K. & Bengio, Y. Neural machine translation by jointly learning to align and translate. ICLR (2015).
17. Sutskever, I., Vinyals, O. & Le, Q. V. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, 3104-3112 (2014).
18. Hinton, G. et al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. Signal Process. Mag. IEEE 29, 82-97 (2012).
19. Bengio, Y. & Gingras, F. Recurrent neural networks for missing or asynchronous data. Adv. neural information processing systems 395-401 (1996).
20. Tresp, V. & Briegel, T. A solution for missing data in recurrent neural networks with an application to blood glucose prediction. NIPS 971-977 (1998).
21. Parveen, S. & Green, P. Speech recognition with missing data using recurrent neural nets. In Advances in Neural Information Processing Systems, 1189-1195 (2001).
22. Lipton, Z. C., Kale, D. & Wetzel, R. Directly modeling missing data in sequences with rnns: Improved classification of clinical time series. In Machine Learning for Healthcare Conference, 253-270 (2016).
23. Choi, E., Bahadori, M. T., Schuetz, A., Stewart, W. F. & Sun, J. Doctor ai: Predicting clinical events via recurrent neural networks. In Machine Learning for Healthcare Conference, 301-318 (2016).
24. Pham, T., Tran, T., Phung, D. & Venkatesh, S. Deepcare: A deep dynamic memory model for predictive medicine. In Advances in Knowledge Discovery and Data Mining, 30-41 (2016).
25. Che, Z., Purushotham, S., Cho, K., Sontag, D. & Liu, Y. Recurrent neural networks for multivariate time series with missing values. arXiv preprint arXiv:1606.01865 (2016).
26. Vodovotz, Y., An, G. & Androulakis, I. P. A systems engineering perspective on homeostasis and disease. Front. bioengineering biotechnology 1 (2013).
27. Zhou, L. & Hripcsak, G. Temporal reasoning with medical data-a review with emphasis on medical natural language processing. J. biomedical informatics 40, 183-202 (2007).
28. Batista, G. E. & Monard, M. C. et al. A study of k-nearest neighbour as an imputation method. HIS 87, 48 (2002).
29. Josse, J. & Husson, F. Handling missing values in exploratory multivariate data analysis methods. J. de la Société Française de Stat. 153, 79-99 (2012).
30. Stekhoven, D. J. & Bühlmann, P. Missforest-non-parametric missing value imputation for mixed-type data. Bioinforma. 28, 112-118 (2011).
31. Alex Rubinsteyn, S. F. fancyimpute. https://github.com/hammerlab/fancyimpute (2015).
32. English, P. predictive-imputer. https://github.com/log0ymxm/predictive-imputer (2016).
33. Jones, E., Oliphant, T. & Peterson, P. Scipy: Open source scientific tools for python. http://www.scipy.org/(2001).
34. Pedregosa, F. et al. Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 12, 2825-2830 (2011).
35. Ioffe, S. & Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on Machine Learning, 448-456 (2015).
36. Srivastava, N., Hinton, G. E., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R. Dropout: a simple way to prevent neural networks from overfitting. JMLR 15 (2014).
37. Kingma, D. & Ba, J. Adam: A method for stochastic optimization. ICLR (2015).
38. Chollet, F. et al. Keras. https://github.com/keras-team/keras (2015).
39. Bergstra, J. et al. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy) (2010).
40. Madeo, R. C., Lima, C. A. & Peres, S. M. Gesture unit segmentation using support vector machines: segmenting gestures from rest positions. In SAC (2013).
41. Silva, I., Moody, G., Scott, D. J., Celi, L. A. & Mark, R. G. Predicting in-hospital mortality of icu patients: The physionet/computing in cardiology challenge 2012. In CinC (2012).
42. Gal, Y. & Ghahramani, Z. A theoretically grounded application of dropout in recurrent neural networks. In Advances in Neural Information Processing Systems, 1019-1027 (2016).
43. Che, Z., Kale, D., Li, W., Bahadori, M. T. & Liu, Y. Deep computational phenotyping. In SIGKDD (2015).
44. Purushotham, S., Meng, C., Che, Z. & Liu, Y. Benchmark of deep learning models on large healthcare mimic datasets. arXiv preprint arXiv:1710.08531 (2017).
45. Johnson, A. E., Pollard, T. J. & Mark, R. G. Reproducibility in critical care: a mortality prediction case study. In Machine Learning for Healthcare Conference, 361-376 (2017).
46. Luo, Y.-F. & Rumshisky, A. Interpretable topic features for post-icu mortality prediction. In AMIA Annual Symposium Proceedings, 827 (2016).