Empower sequence labeling with task-aware neural language model

32nd AAAI Conference on Artificial Intelligence, AAAI 2018

Volumn , Issue , 2018, Pages 5253-5260

  Liu, Liyuan ^a   Shang, Jingbo ^a   Ren, Xiang ^b   Xu, Frank F ^c   Gui, Huan ^b   Peng, Jian ^a   Han, Jiawei ^a  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^b UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^c SHANGHAI JIAO TONG UNIVERSITY   (China)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; COMPUTATIONAL LINGUISTICS; KNOWLEDGE MANAGEMENT; LEARNING SYSTEMS; NATURAL LANGUAGE PROCESSING SYSTEMS; SPEECH RECOGNITION;

BENCHMARK DATASETS; NAMED ENTITY RECOGNITION; NEURAL NETWORKS (NNS); PART OF SPEECH TAGGING; SPECIFIC KNOWLEDGE; TRAINING SEQUENCES; TRANSFER LEARNING; TRANSFER LEARNING METHODS;

DATA MINING;

EID: 85058796861     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (34)

1. Chieu, H. L., and Ng, H. T. 2002. Named entity recognition: A maximum entropy approach using global information. In COLING.
2. Chiu, J. P. C., and Nichols, E. 2016. Named entity recognition with bidirectional lstm-cnns. TACL.
3. Collobert, R.; Weston, J.; Bottou, L.; Karlen, M.; Kavukcuoglu, K.; and Kuksa, P. P. 2011. Natural language processing (almost) from scratch. JMLR.
4. Fernandez, J.; Yu, Z.; and Downey, D. 2017. Vecshare: A framework for sharing word representation vectors.
5. Florian, R.; Ittycheriah, A.; Jing, H.; and Zhang, T. 2003. Named entity recognition through classifier combination. In CoNLL.
6. Glorot, X., and Bengio, Y. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics.
7. Hashimoto, K.; Xiong, C.; Tsuruoka, Y.; and Socher, R. 2016. A joint many-task model: Growing a neural network for multiple nlp tasks. arXiv:1611.01587.
8. Jozefowicz, R.; Vinyals, O.; Schuster, M.; Shazeer, N.; and Wu, Y. 2016. Exploring the limits of language modeling. arXiv:1602.02410.
9. Jozefowicz, R.; Zaremba, W.; and Sutskever, I. 2015. An empirical exploration of recurrent network architectures. In ICML.
10. Karpathy, A. 2015. The unreasonable effectiveness of recurrent neural networks. http://karpathy.github.io/2015/05/21/rnn-effectiveness/. Accessed: 2017-08-22.
11. Lafferty, J. D.; McCallum, A.; and Pereira, F. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In ICML.
12. Lample, G.; Ballesteros, M.; Kawakami, K.; Subramanian, S.; and Dyer, C. 2016. Neural architectures for named entity recognition. In NAACL-HLT.
13. Liu, L.; Ren, X.; Zhu, Q.; Zhi, S.; Gui, H.; Ji, H.; and Han, J. 2017. Heterogeneous Supervision for Relation Extraction: A Representation Learning Approach. Proc. EMNLP.
14. Luo, G.; Huang, X.; Lin, C.-Y.; and Nie, Z. 2015. Joint named entity recognition and disambiguation. In EMNLP.
15. Ma, X., and Hovy, E. 2016. End-to-end sequence labeling via bi-directional lstm-cnns-crf. In ACL.
16. Manning, C. D. 2011. Part-of-speech tagging from 97% to 100%: is it time for some linguistics? In International Conference on Intelligent Text Processing and Computational Linguistics. Springer.
17. Marcus, M. P.; Marcinkiewicz, M. A.; and Santorini, B. 1993. Building a large annotated corpus of english: The penn treebank. Computational linguistics.
18. McCallum, A., and Li, W. 2003. Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons. In CoNLL.
19. Mikolov, T.; Sutskever, I.; Chen, K.; Corrado, G. S.; and Dean, J. 2013. Distributed representations of words and phrases and their compositionality. In NIPS.
20. Peng, N., and Dredze, M. 2016. Improving named entity recognition for chinese social media with word segmentation representation learning. In ACL.
21. Pennington, J.; Socher, R.; and Manning, C. D. 2014. Glove: Global vectors for word representation. In EMNLP.
22. Peters, M. E.; Ammar, W.; Bhagavatula, C.; and Power, R. 2017. Semi-supervised sequence tagging with bidirectional language models. arXiv:1705.00108.
23. Ratinov, L., and Roth, D. 2009. Design challenges and misconceptions in named entity recognition. In CoNLL.
24. Rei, M. 2017. Semi-supervised multitask learning for sequence labeling. In ACL.
25. Reimers, N., and Gurevych, I. 2017. Reporting score distributions makes a difference: Performance study of lstm-networks for sequence tagging. arXiv preprint arXiv:1707.09861.
26. Sha, F., and Pereira, F. 2003. Shallow parsing with conditional random fields. In NAACL-HLT.
27. Shang, J.; Liu, J.; Jiang, M.; Ren, X.; Voss, C. R.; and Han, J. 2017. Automated phrase mining from massive text corpora. arXiv:1702.04457.
28. Søgaard, A., and Goldberg, Y. 2016. Deep multi-task learning with low level tasks supervised at lower layers. In ACL.
29. Søgaard, A. 2011. Semisupervised condensed nearest neighbor for part-of-speech tagging. In NAACL-HLT.
30. Srivastava, R. K.; Greff, K.; and Schmidhuber, J. 2015. Highway networks. arXiv:1505.00387.
31. Sun, X. 2014. Structure regularization for structured prediction. In NIPS.
32. Tjong Kim Sang, E. F., and Buchholz, S. 2000. Introduction to the conll-2000 shared task: Chunking. In Learning language in logic and CoNLL.
33. Tjong Kim Sang, E. F., and De Meulder, F. 2003. Introduction to the conll-2003 shared task: Language-independent named entity recognition. In Natural language learning at NAACL-HLT.
34. Yang, Z.; Salakhutdinov, R.; and Cohen, W. W. 2017. Transfer learning for sequence tagging with hierarchical recurrent networks. arXiv:1703.06345.