Semi-supervised multitask learning for sequence labeling

ACL 2017 - 55th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers)

Volumn 1, Issue , 2017, Pages 2121-2130

  Rei, Marek ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

BENCHMARKING; COMPUTATIONAL LINGUISTICS; LINGUISTICS; NATURAL LANGUAGE PROCESSING SYSTEMS; SEMANTICS;

CONSISTENT PERFORMANCE; LANGUAGE MODEL; MULTITASK LEARNING; NAMED ENTITY RECOGNITION; POS TAGGING; SEMI-SUPERVISED; SEQUENCE LABELING;

MODELING LANGUAGES;

EID: 85040601183     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.18653/v1/P17-1194     Document Type: Conference Paper

References (34)

1. Rami Al-Rfou, Guillaume Alain, Amjad Almahairi, Christof Angermueller, Dzmitry Bahdanau, Nicolas Ballas, Frédéric Bastien, Justin Bayer, Ana-toly Belikov, Alexander Belopolsky, Yoshua Bengio, Arnaud Bergeron, James Bergstra, Valentin Bisson, Josh Bleecher Snyder, Nicolas Bouchard, Nicolas Boulanger-Lewandowski, and Others. 2016. Theano: A Python framework for fast computation of mathematical expressions. arXiv e-prints abs/1605.0:19. http://arxiv.org/abs/1605.02688.
2. Isabelle Augenstein and Anders Søgaard. 2017. MultiTask Learning of Keyphrase Boundary Classification. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. http://arxiv.org/abs/1704.00514.
3. Joachim Bingel and Anders Søgaard. 2017. Identifying beneficial task relations for multi-task learning in deep neural networks. In arXiv preprint. http://arxiv.org/abs/1702.08303.
4. Rich Caruana. 1998. Multitask Learning. Ph.D. thesis.
5. Ciprian Chelba, Tomáš Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, Phillipp Koehn, and Tony Robinson. 2013. One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling. In arXiv preprint. http://arxiv.org/abs/1312.3005.
6. Hao Cheng, Hao Fang, and Mari Ostendorf. 2015. Open-Domain Name Error Detection using a MultiTask RNN. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.
7. Ronan Collobert and Jason Weston. 2008. A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning. Proceedings of the 25th international conference on Machine learning (ICML'08) https://doi.org/10.1145/1390156.1390177.
8. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural Language Processing (Almost) from Scratch. Journal of Machine Learning Research 12. https://doi.org/10.1145/2347736.2347755.
9. Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton. 2013. Speech recognition with deep recurrent neural networks. International Conference on Acoustics, Speech and Signal Processing (ICASSP) https://doi.org/10.1109/ICASSP.2013.6638947.
10. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-term Memory. Neural Computation 9. https://doi.org/10.1.1.56.7752.
11. Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional LSTM-CRF Models for Sequence Tagging. arXiv:1508.01991 http://arxiv.org/pdf/1508.01991v1.pdf.
12. Ozan Irsoy and Claire Cardie. 2014. Opinion Mining with Deep Recurrent Neural Networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).
13. Jin-Dong Kim, Tomoko Ohta, Yoshimasa Tsuruoka, Yuka Tateisi, and Nigel Collier. 2004. Introduction to the Bio-entity Recognition Task at JNLPBA. Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and Its Applications https://doi.org/10.3115/1567594.1567610.
14. Martin Krallinger, Florian Leitner, Obdulia Rabal, Miguel Vazquez, Julen Oyarzabal, and Alfonso Valencia. 2015. CHEMDNER: The drugs and chemical names extraction challenge. Journal of Chemin-formatics 7(Suppl 1). https://doi.org/10.1186/17582946-7-S1-S1.
15. John Lafferty, Andrew McCallum, and Fernando Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning.
16. Guillaume Lample, Miguel Ballesteros, Sandeep Sub-ramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural Architectures for Named Entity Recognition. In Proceedings of NAACL-HLT 2016.
17. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics 19. https://doi.org/10.1162/coli.2010.36.1.36100.
18. Tomáš Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. In Proceedings of the International Conference on Learning Representations (ICLR 2013). https://doi.org/10.1162/153244303322533223.
19. Tomáš Mikolov, Martin Karafiát, Lukáš Burget, Jan Cernocký, and Sanjeev Khudanpur. 2010. Recurrent Neural Network based Language Model. Inter-speech (September):1045-1048.
20. Andriy Mnih and Yee Whye Teh. 2012. A fast and simple algorithm for training neural probabilistic language models. In Neural Information Processing Systems (NIPS).
21. Frederic Morin and Yoshua Bengio. 2005. Hierarchical probabilistic neural network language model. Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics https://doi.org/10.1109/JCDL.2003.1204852.
22. Hwee Tou Ng, Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, and Christopher Bryant. 2014. The CoNLL-2014 Shared Task on Grammatical Error Correction. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task. http://www.aclweb.org/anthology/W/W14/W141701.
23. Joakim Nivre, Željko Agic, Maria Jesus Aranzabe, Masayuki Asahara, Aitziber Atutxa, Miguel Ballesteros, John Bauer, Kepa Bengoetxea, Riyaz Ahmad Bhat, Cristina Bosco, Sam Bowman, et al. 2015. Universal dependencies 1.2. LIN-DAT/CLARIN digital library at the Institute of Formal and Applied Linguistics, Charles University. http://hdl.handle.net/11234/1-1548.
24. Tomoko Ohta, Yuka Tateisi, and Jin-Dong Kim. 2002. The GENIA corpus: An annotated research abstract corpus in molecular biology domain. Proceedings of the second international conference on Human Language Technology Research http://portal.acm.org/citation.cfm?id=1289260.
25. Barbara Plank, Anders Søgaard, and Yoav Goldberg. 2016. Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Loss. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL-16). pages 412-418. http://arxiv.org/abs/1604.05529.
26. Marek Rei, Gamal K. O. Crichton, and Sampo Pyysalo. 2016. Attending to Characters in Neural Sequence Labeling Models. In Proceedings of the 26th International Conference on Computational Linguistics (COLING-2016). http://arxiv.org/abs/1611.04361.
27. Marek Rei and Helen Yannakoudakis. 2016. Compositional Sequence Labeling Models for Error Detection in Learner Writing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. https://aclweb.org/anthology/P/P16/P16-1112.pdf.
28. Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhut-dinov. 2014. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research (JMLR) 15. https://doi.org/10.1214/12-AOS1000.
29. Erik F. Tjong Kim Sang and Sabine Buchholz. 2000. Introduction to the CoNLL-2000 shared task: Chunking. Proceedings of the 2nd Workshop on Learning Language in Logic and the 4th Conference on Computational Natural Language Learning 7. https://doi.org/10.3115/1117601.1117631.
30. Erik F. Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition. In Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003. http://arxiv.org/abs/cs/0306050.
31. Yoshimasa Tsuruoka, Yuka Tateishi, Jin Dong Kim, Tomoko Ohta, John McNaught, Sophia Ananiadou, and Jun'ichi Tsujii. 2005. Developing a robust part-of-speech tagger for biomedical text. In Proceedings of Panhellenic Conference on Informatics.
32. Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011. A New Dataset and Method for Automatically Grading ESOL Texts. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. http://www.aclweb.org/anthology/P11-1019.
33. Matthew D. Zeiler. 2012. ADADELTA: An Adaptive Learning Rate Method. arXiv preprint arXiv:1212.5701 http://arxiv.org/abs/1212.5701.
34. GuoDong Zhou and Jian Su. 2004. Exploring Deep Knowledge Resources in Biomedical Name Recognition. Workshop on Natural Language Processing in Biomedicine and Its Applications at COLING.