Predicting mental conditions based on “history of present illness” in psychiatric notes with deep neural networks

Journal of Biomedical Informatics

Volumn 75, Issue , 2017, Pages S138-S148

  Tran, Tung ^a   Kavuluru, Ramakanth ^a,b  

^a UNIVERSITY OF KENTUCKY   (United States)

^b UNIVERSITY OF KENTUCKY   (United States)

Author keywords

Convolutional and recurrent neural networks; Hierarchical attention networks; Multi label text classification; Psychiatric condition prediction

Indexed keywords

CLASSIFICATION (OF INFORMATION); CONVOLUTION; DISEASES; FORECASTING; NATURAL LANGUAGE PROCESSING SYSTEMS; RECURRENT NEURAL NETWORKS; TEXT PROCESSING;

ATTENTION DEFICIT HYPERACTIVITY DISORDER; ATTENTION MECHANISMS; CONDITION PREDICTION; CONVOLUTIONAL NEURAL NETWORKS (CNN); INTERPRETATION OF MODELS; MULTI-LABEL TEXT CLASSIFICATION; NAMED ENTITY RECOGNITION; PROBABILITY ESTIMATE;

DEEP NEURAL NETWORKS;

ANXIETY; ARTICLE; ARTIFICIAL NEURAL NETWORK; ATTENTION DEFICIT DISORDER; BIPOLAR DISORDER; CONVOLUTIONAL NEURAL NETWORK; DEEP NEURAL NETWORK; DEPRESSION; DIAGNOSTIC TEST; EATING DISORDER; HUMAN; INTERMETHOD COMPARISON; MACHINE LEARNING; MEDICAL HISTORY; MEDICAL RECORD; MENTAL HEALTH; NATURAL LANGUAGE PROCESSING; NEUROPSYCHIATRIC GENOME SCALE AND RDOC INDIVIDUALIZED DOMAIN; PANIC; POSTTRAUMATIC STRESS DISORDER; PREDICTION; PRIORITY JOURNAL; PSYCHOSIS; RECURRENT NEURAL NETWORK; MENTAL DISEASE;

HUMANS; MACHINE LEARNING; MENTAL DISORDERS; NATURAL LANGUAGE PROCESSING; NEURAL NETWORKS (COMPUTER);

EID: 85021835221     PISSN: 15320464     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jbi.2017.06.010     Document Type: Article

References (43)

1. Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, Xiaoqiang Zheng, TensorFlow: Large-scale Machine Learning on Heterogeneous Systems, 2015. Software available from < http://tensorflow.org/>.
2. Abbe, A., Grouin, C., Zweigenbaum, P., Falissard, B., Text mining applications in psychiatry: a systematic literature review. Int. J. Methods Psychiatr. Res. 25:2 (2016), 86–100.
3. Aronson, Alan R., Lang, François-Michel, An overview of MetaMap: historical perspective and recent advances. J. Am. Med. Inform. Assoc. 17:3 (2010), 229–236.
4. Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio, Neural Machine Translation by Jointly Learning to Align and Translate, 2014. arXiv preprint Available from: < arXiv:1409.0473>.
5. Bengio, Yoshua, Ducharme, Réjean, Vincent, Pascal, Janvin, Christian, A neural probabilistic language model. J. Mach. Learn. Res. 3 (2003), 1137–1155.
6. Adrian Benton, Margaret Mitchell, Dirk Hovy, Multitask learning for mental health conditions with limited social media data, in: Proceedings of the 15th Conference of the EACL, Long Papers, vol. 1, 2017, pp. 152–162.
7. James Bergstra, Olivier Breuleux, Frédéric Bastien, Pascal Lamblin, Razvan Pascanu, Guillaume Desjardins, Joseph Turian, David Warde-Farley, Yoshua Bengio, Theano: a CPU and GPU math expression compiler, in: Proceedings of the Python for Scientific Computing Conference (SciPy), Oral Presentation, June 2010.
8. Caruana, Rich, Multitask learning. Learning to Learn, 1998, Springer, 95–133.
9. Collobert, Ronan, Weston, Jason, A unified architecture for natural language processing: deep neural networks with multitask learning. Proceedings of the 25th International Conference on Machine Learning, 2008, ACM, 160–167.
10. Collobert, Ronan, Weston, Jason, Bottou, Léon, Karlen, Michael, Kavukcuoglu, Koray, Kuksa, Pavel, Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12 (2011), 2493–2537.
11. Benjamin Druss, Elizabeth Walker, Mental Disorders and Medical Comorbidity. < http://www.rwjf.org/content/dam/farm/reports/issue_briefs/2011/rwjf69438>.
12. Filannino, Michele, Stubbs, Amber, Uzuner, Özlem, Symptom severity prediction from neuropsychiatric clinical records: overview of 2016 CEGS N-GRID Shared Tasks Track 2. J. Biomed. Inform. 75 (2017), S62–S70.
13. Gers, Felix A., Schmidhuber, Jürgen, Cummins, Fred, Learning to forget: continual prediction with LSTM. Neural Comput. 12:10 (2000), 2451–2471.
14. Goldberg, Yoav, A primer on neural network models for natural language processing. J. Artif. Intell. Res. 57 (2016), 345–420.
15. Graves, Alex, Supervised sequence labelling with recurrent neural networks. Studies in Computational Intelligence, vol. 385, 2012, Springer, 10.1007/978-3-642-24797-2 ISBN 978-3-642-24796-5.
16. Hochreiter, Sepp, Schmidhuber, Jürgen, Long short-term memory. Neural Comput. 9:8 (1997), 1735–1780.
17. Jackson, Richard G., Patel, Rashmi, Jayatilleke, Nishamali, Kolliakou, Anna, Ball, Michael, Gorrell, Genevieve, Roberts, Angus, Dobson, Richard J., Stewart, Robert, Natural language processing to extract symptoms of severe mental illness from clinical text: the clinical record interactive search comprehensive data extraction (cris-code) project. BMJ Open, 7(1), 2017, e012012.
18. Kalchbrenner, Nal, Blunsom, Phil, Recurrent continuous translation models. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2013, 1700–1709.
19. Andrej Karpathy, Neural Networks Part 2: Setting up the Data and the Loss, 2016. < http://cs231n.github.io/neural-networks-2/>.
20. Kavuluru, Ramakanth, Lu, Yuan, Leveraging output term co-occurrence frequencies and latent associations in predicting medical subject headings. Data Knowl. Eng. 94:Part B (2014), 189–201.
21. Kavuluru, Ramakanth, Rios, Anthony, Lu, Yuan, An empirical evaluation of supervised learning approaches in assigning diagnosis codes to electronic medical records. Artif. Intell. Med. 65:2 (2015), 155–166.
22. Yoon Kim, Convolutional neural networks for sentence classification, in: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), October 2014, pp. 1746–1751.
23. Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton, Imagenet classification with deep convolutional neural networks, in: Advances in Neural Information Processing Systems, 2012, pp. 1097–1105.
24. Li, Jiwei, Chen, Xinlei, Hovy, Eduard, Jurafsky, Dan, Visualizing and understanding neural models in NLP. Proceedings of NAACL-HLT, 2016, 681–691.
25. Mikolov, Tomas, Sutskever, Ilya, Chen, Kai, Corrado, Greg S., Dean, Jeff, Distributed representations of words and phrases and their compositionality. Advances in Neural Information Processing Systems, 2013, 3111–3119.
26. Nair, Vinod, Hinton, Geoffrey E., Rectified linear units improve restricted Boltzmann machines. Proceedings of the 27th International Conference on Machine Learning (ICML-10), 2010, 807–814.
27. Nam, Jinseok, Kim, Jungi, Mencía, Eneldo Loza, Gurevych, Iryna, Fürnkranz, Johannes, Large-scale multi-label text classification – revisiting neural networks. Machine Learning and Knowledge Discovery in Databases, 2014, Springer, 437–452.
28. National Institute of Mental Health, Any Mental Illness (AMI) among U.S. Adults. < https://www.nimh.nih.gov/health/statistics/prevalence/any-mental-illness-ami-among-us-adults.shtml>.
29. National Library of Medicine, Unified Medical Language System Reference Manual, 2009. < http://www.ncbi.nlm.nih.gov/books/NBK9676/>.
30. Ngiam, Jiquan, Coates, Adam, Lahiri, Ahbik, Prochnow, Bobby, Le, Quoc V., Ng, Andrew Y., On optimization methods for deep learning. Proceedings of the 28th International Conference on Machine Learning (ICML-11), 2011, 265–272.
31. Pascanu, Razvan, Mikolov, Tomas, Bengio, Yoshua, On the difficulty of training recurrent neural networks. Proceedings of the 30th International Conference on Machine Learning 28 (2013), 1310–1318.
32. Perlis, R.H., Iosifescu, D.V., Castro, V.M., Murphy, S.N., Gainer, V.S., Minnier, Jessica, Cai, T., Goryachev, S., Zeng, Q., Gallagher, P.J., Using electronic medical records to enable large-scale studies in psychiatry: treatment resistant depression as a model. Psychol. Med. 42:01 (2012), 41–50.
33. Pestian, John P., Matykiewicz, Pawel, Linn-Gust, Michelle, South, Brett, Uzuner, Ozlem, Wiebe, Jan, Bretonnel Cohen, K., Hurdle, John, Brew, Christopher, Sentiment analysis of suicide notes: a shared task. Biomed. Inform. Insights, 5(Suppl. 1), 2012, 3.
34. Pyysalo, Sampo, Ginter, Filip, Moen, Hans, Salakoski, Tapio, Ananiadou, Sophia, Distributional semantics resources for biomedical text processing. Proceedings of 5th International Symposium on Languages in Biology and Medicine, 2013, 39–44.
35. Rios, Anthony, Kavuluru, Ramakanth, Supervised extraction of diagnosis codes from EMRs: role of feature selection, data selection, and probabilistic thresholding. 2013 IEEE International Conference on Healthcare Informatics (ICHI), 2013, IEEE, 66–73.
36. Rios, Anthony, Kavuluru, Ramakanth, Analyzing the moving parts of a large-scale multi-label text classification pipeline: experiences in indexing biomedical articles. 2015 International Conference on Healthcare Informatics (ICHI), 2015, IEEE, 1–7.
37. Rios, Anthony, Kavuluru, Ramakanth, Convolutional neural networks for biomedical text classification: application in indexing biomedical articles. Proceedings of the 6th ACM Conference on Bioinformatics, Computational Biology and Health Informatics, 2015, ACM, 258–267.
38. Rumshisky, A., Ghassemi, M., Naumann, T., Szolovits, P., Castro, V.M., McCoy, T.H., Perlis, R.H., Predicting early psychiatric readmission with natural language processing of narrative discharge summaries. Transl. Psychiat., 6(10), 2016, e921.
39. Srivastava, Nitish, Hinton, Geoffrey, Krizhevsky, Alex, Sutskever, Ilya, Salakhutdinov, Ruslan, Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15:1 (2014), 1929–1958.
40. Stubbs, Amber, Filannino, Michele, Uzuner, Özlem, De-identification of psychiatric intake records: overview of 2016 CEGS N-GRID Shared Tasks Track 1. J. Biomed. Inform. 75 (2017), S4–S18.
41. Tieleman, Tijmen, Hinton, Geoffrey, Lecture 6.5-rmsprop: divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks Mach. Learn., 4(2), 2012.
42. Tsoumakas, Grigorios, Katakis, Ioannis, Vlahavas, Ioannis P., Mining multi-label data. Data Mining and Knowledge Discovery Handbook, 2010, 667–685.
43. Yang, Zichao, Yang, Diyi, Dyer, Chris, He, Xiaodong, Smola, Alex, Hovy, Eduard, Hierarchical attention networks for document classification. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016, Association for Computational Linguistics, 1480–1489.