Document-level attention-based BiLSTM-CRF incorporating disease dictionary for disease named entity recognition

Computers in Biology and Medicine

Volumn 108, Issue , 2019, Pages 122-132

  Xu, Kai ^a   Yang, Zhenguo ^a,b   Kang, Peipei ^a   Wang, Qi ^a   Liu, Wenyin ^a  

^a GUANGDONG UNIVERSITY OF TECHNOLOGY   (China)

^b CITY UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Biomedical informatics; Machine learning; Named entity recognition; Neural network; String matching

Indexed keywords

CLOCK AND DATA RECOVERY CIRCUITS (CDR CIRCUITS); COMPLEX NETWORKS; DEEP LEARNING; LEARNING SYSTEMS; NEURAL NETWORKS; RANDOM PROCESSES;

ATTENTION MECHANISMS; BIOMEDICAL INFORMATICS; BIOMEDICAL RESEARCH; CONDITIONAL RANDOM FIELD; NAMED ENTITY RECOGNITION; NEURAL NETWORK METHOD; STATE-OF-THE-ART METHODS; STRING MATCHING;

DISEASES;

ARTICLE; BIDIRECTIONAL LONG SHORT TERM MEMORY NETWORK; BOOK; COMPARATIVE STUDY; CONDITIONAL RANDOM FIELD; DEEP LEARNING; DISEASE ONTOLOGY; HUMAN; KNOWLEDGE BASE; LONG TERM MEMORY; MACHINE LEARNING; MEDICAL RESEARCH; NAMED ENTITY RECOGNITION; NATIONAL HEALTH ORGANIZATION; PREDICTION; PRIORITY JOURNAL; RECOGNITION; SHORT TERM MEMORY; SYSTEMATIZED NOMENCLATURE OF MEDICINE; DATA MINING; DISEASES; LANGUAGE; NOMENCLATURE;

DATA MINING; DEEP LEARNING; DISEASE; HUMANS; LANGUAGE; TERMINOLOGY AS TOPIC;

EID: 85064312088     PISSN: 00104825     EISSN: 18790534     Source Type: Journal    
DOI: 10.1016/j.compbiomed.2019.04.002     Document Type: Article

References (60)

1. Dogan, R.I., Leaman, R., Lu, Z., NCBI disease corpus: a resource for disease name recognition and concept normalization. J. Biomed. Inform. 47 (2014), 1–10, 10.1016/j.jbi.2013.12.006.
2. Leaman, R., Islamaj Doğan, R., Lu, Z., DNorm: disease name normalization with pairwise learning to rank. Bioinformatics 29:22 (2013), 2909–2917, 10.1093/bioinformatics/btt474 https://academic.oup.com/bioinformatics/article/29/22/2909/312804.
3. Lin, Y.-F., Tsai, T.-H., Chou, W.-C., Wu, K.-P., Sung, T.-Y., Hsu, W.-L., A maximum entropy approach to biomedical named entity recognition. Proceedings of the 4th International Conference on Data Mining in Bioinformatics, BIOKDD'04, Springer-Verlag, Berlin, Heidelberg, 2004, 56–61 http://dl.acm.org/citation.cfm?id=3000580.3000588.
4. Jimeno, A., Jimenez-Ruiz, E., Lee, V., Gaudan, S., Berlanga, R., Rebholz-Schuhmann, D., Assessment of disease named entity recognition on a corpus of annotated sentences. BMC Bioinf., 9(3), 2008, 10.1186/1471-2105-9-S3-S3 S3 https://doi.org/10.1186/1471-2105-9-S3-S3.
5. Aronson, A.R., Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program. Proceedings of the AMIA Symposium, 2001, American Medical Informatics Association, 17 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2243666/.
6. Lowe, D.M., Sayle, R.A., LeadMine: a grammar and dictionary driven approach to entity recognition. Suppl 1 Text mining for chemistry and the CHEMDNER track J. Cheminf., 7, 2015, 10.1186/1758-2946-7-S1-S5 S5.
7. Lafferty, J., McCallum, A., Pereira, F.C., Conditional random fields: probabilistic models for segmenting and labeling sequence data. http://dl.acm.org/citation.cfm?id=645530.655813, 2001 282-289.
8. Sun, C., Guan, Y., Wang, X., Lin, L., Rich features based Conditional Random Fields for biological named entities recognition. Comput. Biol. Med. 37:9 (2007), 1327–1333, 10.1016/j.compbiomed.2006.12.002 http://www.sciencedirect.com/science/article/pii/S0010482506002332.
9. Lee, W., Kim, K., Lee, E.Y., Choi, J., Conditional random fields for clinical named entity recognition: a comparative study using Korean clinical texts. Comput. Biol. Med. 101 (2018), 7–14, 10.1016/j.compbiomed.2018.07.019 http://www.sciencedirect.com/science/article/pii/S0010482518302105.
10. Leaman, R., Gonzalez, G., BANNER: an executable survey of advances in biomedical named entity recognition. http://www.scopus.com/inward/citedby.url?scp=40549140499&partnerID=8YFLogxK, 2008.
11. Lee, H.-C., Hsu, Y.-Y., Kao, H.-Y., AuDis: an automatic CRF-enhanced disease normalization in biomedical text. Database, 2016, 10.1093/database/baw091 https://academic.oup.com/database/article/doi/10.1093/database/baw091/2630473.
12. Campos, D., Matos, S., Oliveira, J.L., A modular framework for biomedical concept recognition. BMC Bioinf., 14(1), 2013, 281, 10.1186/1471-2105-14-281 https://doi.org/10.1186/1471-2105-14-281.
13. Davis, A.P., Wiegers, T.C., Rosenstein, M.C., Mattingly, C.J., MEDIC: a practical disease vocabulary used at the Comparative Toxicogenomics Database. Database, 2012, 10.1093/database/bar065 https://academic.oup.com/database/article/doi/10.1093/database/bar065/430135.
14. Leaman, R., Lu, Z., TaggerOne: joint named entity recognition and normalization with semi-Markov Models. Bioinformatics 32:18 (2016), 2839–2846, 10.1093/bioinformatics/btw343.
15. LeCun, Y., Bengio, Y., Hinton, G., Deep learning. Nature 521:7553 (2015), 436–444, 10.1038/nature14539 https://www.nature.com/articles/nature14539.
16. Santiso, S., Perez, A., Casillas, A., Exploring joint AB-LSTM with embedded lemmas for adverse drug reaction discovery. IEEE Journal of Biomedical and Health Informatics, 2018, 1, 10.1109/JBHI.2018.2879744.
17. Luo, L., Yang, Z., Yang, P., Zhang, Y., Wang, L., Lin, H., Wang, J., An attention-based BiLSTM-CRF approach to document-level chemical named entity. Bioinformatics, 2017, 10.1093/bioinformatics/btx761 https://doi.org/10.1093/bioinformatics/btx761 btx761-btx761.
18. Wei, Q., Chen, T., Xu, R., He, Y., Gui, L., Disease named entity recognition by combining conditional random fields and bidirectional recurrent neural networks. Database. J. Biol. Databases. Curation, 2016, 10.1093/database/baw140.
19. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J., Distributed representations of words and phrases and their compositionality. Advances in Neural Information Processing Systems, 2013, 3111–3119 http://dl.acm.org/citation.cfm?id=2999792.2999959.
20. Pyysalo, S., Ginter, F., Moen, H., Salakoski, T., Ananiadou, S., Distributional Semantics Resources for Biomedical Text Processing, Tokyo, Japan. 2013, 39–43 http://bio.nlplab.org/pdf/pyysalo13literature.pdf.
21. Gridach, M., Character-level neural network for biomedical named entity recognition. J. Biomed. Inform. 70 (2016), 85–91 http://www.sciencedirect.com/science/article/pii/S1532046417300977.
22. Xu, K., Zhou, Z., Hao, T., Liu, W., A bidirectional LSTM and conditional random fields approach to medical named entity recognition. Advances in Intelligent Systems and Computing, vol. 639, 2017, t, 355–365 Cairo, Egyp https://doi.org/10.1007/978-3-319-64861-3_33.
23. Ratinov, L., Roth, D., Design challenges and misconceptions in named entity recognition. Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL-2009), 2009, Association for Computational Linguistics, Boulder, Colorado, 147–155 http://www.aclweb.org/anthology/W09-1119.
24. Xu, K., Zhou, Z., Gong, T., Hao, T., Liu, W., SBLC: a hybrid model for disease named entity recognition based on semantic bidirectional LSTMs and conditional random fields. BMC Med. Inf. Decis. Mak., 18(5), 2018, 114, 10.1186/s12911-018-0690-y https://doi.org/10.1186/s12911-018-0690-y.
25. Lou, Y., Zhang, Y., Qian, T., Li, F., Xiong, S., Ji, D., A transition-based joint model for disease named entity recognition and normalization. Bioinformatics 33:15 (2017), 2363–2371, 10.1093/bioinformatics/btx172 https://doi.org/10.1093/bioinformatics/btx172.
26. Li, J., Sun, Y., Johnson, R.J., Sciaky, D., Wei, C.-H., Leaman, R., Davis, A.P., Mattingly, C.J., Wiegers, T.C., Lu, Z., BioCreative V CDR task corpus: a resource for chemical disease relation extraction. Database. J. Biol. Databases. Curation, 2016, 10.1093/database/baw068.
27. Savova, G.K., Masanz, J.J., Ogren, P.V., Zheng, J., Sohn, S., Kipper-Schuler, K.C., Chute, C.G., Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications. J. Am. Med. Inform. Assoc. JAMIA 17:5 (2010), 507–513, 10.1136/jamia.2009.001560 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2995668/.
28. Bird, S., NLTK: the natural language toolkit. Proceedings of the COLING/ACL on Interactive Presentation Sessions, 2006, Association for Computational Linguistics, 69–72 http://dl.acm.org/citation.cfm?id=1225421.
29. Bengio, Y., Courville, A., Vincent, P., Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35:8 (2013), 1798–1828 http://ieeexplore.ieee.org/abstract/document/6472238/.
30. Mikolov, T., Chen, K., Corrado, G., Dean, J., Efficient Estimation of Word Representations in Vector Space. arXiv preprint arXiv:1301.3781 https://arxiv.org/abs/1301.3781.
31. Habibi, M., Weber, L., Neves, M., Wiegandt, D.L., Leser, U., Deep learning with word embeddings improves biomedical named entity recognition. Bioinformatics 33:14 (2017), i37–i48, 10.1093/bioinformatics/btx228 https://academic.oup.com/bioinformatics/article/33/14/i37/3953940.
32. Chiu, B., Crichton, G., Korhonen, A., Pyysalo, S., How to train good word embeddings for biomedical NLP. Proceedings of the 15th Workshop on Biomedical Natural Language Processing, 2016, Association for Computational Linguistics, Berlin, Germany, 166–174 http://anthology.aclweb.org/W16-2922.
33. Schriml, L.M., Arze, C., Nadendla, S., Chang, Y.-W.W., Mazaitis, M., Felix, V., Feng, G., Kibbe, W.A., Disease Ontology: a backbone for disease semantic integration. Nucleic Acids Res. 40:D1 (2012), D940–D946, 10.1093/nar/gkr972 https://academic.oup.com/nar/article/40/D1/D940/2903651.
34. Lipscomb, C.E., Medical Subject Headings (MeSH). Bull. Med. Libr. Assoc. 88:3 (2000), 265–266 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC35238/.
35. Hamosh, A., Scott, A.F., Amberger, J.S., Bocchini, C.A., McKusick, V.A., Online Mendelian Inheritance in Man (OMIM), a knowledgebase of human genes and genetic disorders. Nucleic Acids Res. 33:suppl_1 (2005), D514–D517 https://academic.oup.com/nar/article-abstract/33/suppl_1/D514/2505259.
36. Elkin, P.L., Brown, S.H., Husser, C.S., Bauer, B.A., Wahner-Roedler, D., Rosenbloom, S.T., Speroff, T., Evaluation of the content coverage of SNOMED CT: ability of SNOMED clinical terms to represent clinical problem lists. Mayo Clin. Proc. 81:6 (2006), 741–748, 10.4065/81.6.741 http://www.sciencedirect.com/science/article/pii/S002561961161728X.
37. Aymé, S., Rath, A., Bellet, B., WHO International Classification of Diseases (ICD) Revision Process: incorporating rare diseases into the classification scheme: state of art. Orphanet J. Rare Dis., 5(1), 2010, P1, 10.1186/1750-1172-5-S1-P1 https://doi.org/10.1186/1750-1172-5-S1-P1.
38. Sioutos, N., Coronado, S. d., Haber, M.W., Hartel, F.W., Shaiu, W.-L., Wright, L.W., NCI Thesaurus: a semantic model integrating cancer-related clinical and molecular information. J. Biomed. Inform. 40:1 (2007), 30–43, 10.1016/j.jbi.2006.02.013 http://www.sciencedirect.com/science/article/pii/S1532046406000311.
39. Repository for the human disease ontology, https://github.com/DiseaseOntology/HumanDiseaseOntology, Jan. 2019.
40. Korkontzelos, I., Piliouras, D., Dowsey, A.W., Ananiadou, S., Boosting drug named entity recognition using an aggregate classifier. Artif. Intell. Med. 65:2 (2015), 145–153 http://www.sciencedirect.com/science/article/pii/S0933365715000780.
41. Pons, E., Becker, B.F.H., Akhondi, S.A., Afzal, Z., van Mulligen, E.M., Kors, J.A., Extraction of chemical-induced diseases using prior knowledge and textual information. Database. J. Biol. Databases. Curation, 2016, 10.1093/database/baw046.
42. Dogan, R.I., Lu, Z., An inference method for disease name normalization. 2012 AAAI Fall Symposium Series, 2012 http://www.aaai.org/ocs/index.php/FSS/FSS12/paper/viewPaper/5604.
43. Demner-Fushman, D., Rogers, W.J., Aronson, A.R., MetaMap lite: an evaluation of a new java implementation of MetaMap. J. Am. Med. Inform. Assoc. 24:4 (2017), 841–844, 10.1093/jamia/ocw177 https://academic.oup.com/jamia/article/24/4/841/2961848.
44. Fredkin, E., Trie memory, commun. ACM 3:9 (1960), 490–499, 10.1145/367390.367400 http://doi.acm.org/10.1145/367390.367400.
45. Aho, A.V., Corasick, M.J., Efficient string matching: an aid to bibliographic search, commun. ACM 18:6 (1975), 333–340, 10.1145/360825.360855 http://doi.acm.org/10.1145/360825.360855.
46. Knuth, D., Morris, J. Jr., Pratt, V., Fast pattern matching in strings. SIAM J. Comput. 6:2 (1977), 323–350, 10.1137/0206024 https://doi.org/10.1137/0206024.
47. Hochreiter, S., Schmidhuber, J., Long short-term memory. Neural Comput. 9:8 (1997), 1735–1780 http://www.mitpressjournals.org/doi/abs/10.1162/neco.1997.9.8.1735.
48. Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C., Neural architectures for named entity recognition. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016, Association for Computational Linguistics, San Diego, California, 260–270 http://www.aclweb.org/anthology/N16-1030.
49. Bahdanau, D., Cho, K., Bengio, Y., Neural Machine Translation by Jointly Learning to Align and Translate. 2015 arXiv: 1409.0473 http://arxiv.org/abs/1409.0473.
50. Luong, T., Pham, H., Manning, C.D., Effective approaches to attention-based neural machine translation. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015, Association for Computational Linguistics, Lisbon, Portugal, 1412–1421 http://aclweb.org/anthology/D15-1166.
51. Raffel, C., Ellis, D.P.W., Feed-forward networks with attention can solve some long-term memory problems. ICLR 2016 Workshop, 2016 https://openreview.net/forum?id=81DD7ZNyxI6O2Pl0Ul5j.
52. Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E., Hierarchical attention networks for document classification. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016, Association for Computational Linguistics, San Diego, California, 1480–1489 http://www.aclweb.org/anthology/N16-1174.
53. Bharadwaj, A., Mortensen, D., Dyer, C., Carbonell, J., Phonologically aware neural model for named entity recognition in low resource transfer settings. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016, Association for Computational Linguistics, Austin, Texas, 1462–1472, 10.18653/v1/D16-1153 http://www.aclweb.org/anthology/D16-1153.
54. Rei, M., Crichton, G., Pyysalo, S., Attending to characters in neural sequence labeling models. Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, the COLING 2016 Organizing Committee, Osaka, Japan, 2016, 309–318 http://www.aclweb.org/anthology/C16-1030.
55. Pandey, C., Ibrahim, Z., Wu, H., Iqbal, E., Dobson, R., Improving RNN with Attention and Embedding for Adverse Drug Reactions, DH ’17. 2017, ACM, New York, NY, USA, 67–71, 10.1145/3079452.3079501 http://doi.acm.org/10.1145/3079452.3079501.
56. Xu, G., Wang, C., He, X., Improving clinical named entity recognition with global neural attention. Web and Big Data, Lecture Notes in Computer Science, 2018, Springer, Cham, 264–279, 10.1007/978-3-319-96893-3_20 https://link.springer.com/chapter/10.1007/978-3-319-96893-3_20.
57. Hirschman, L., Yeh, A., Blaschke, C., Valencia, A., Overview of BioCreAtIvE: critical assessment of information extraction for biology. BMC Bioinf., 6(1), 2005, S1, 10.1186/1471-2105-6-S1-S1 https://doi.org/10.1186/1471-2105-6-S1-S1.
58. Wei, C.-H., Peng, Y., Leaman, R., Davis, A.P., Mattingly, C.J., Li, J., Wiegers, T.C., Lu, Z., Assessing the state of the art in biomedical relation extraction: overview of the BioCreative V chemical-disease relation (CDR) task. Database. J. Biol. Databases. Curation, 2016, 10.1093/database/baw032.
59. Leaman, R., Miller, C., Gonzalez, G., Enabling recognition of diseases in biomedical text with machine learning: corpus and benchmark. Proceedings of the 2009 Symposium on Languages in Biology and Medicine, vol. 82, 2009, 9 https://books.google.com.sg/books/about/Enabling_Recognition_of_Disease_Mentions.html?id=Mf5AywAACAAJ&redir_esc=y.
60. Kingma, D.P., Ba, J., Adam: A Method for Stochastic Optimization. 2014 arXiv: 1412.6980 http://arxiv.org/abs/1412.6980.