When is multitask learning effective? Semantic sequence prediction under varying data conditions

15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017 - Proceedings of Conference

Volumn 1, Issue , 2017, Pages 44-53

  Alonso, Héctor Martínez ^b   Plank, Barbara ^a  

^a UNIVERSITY OF GRONINGEN   (Netherlands)

^b UNIVERSITÉ PARIS DESCARTES   (France)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; LEARNING SYSTEMS; MULTI-TASK LEARNING; SEMANTICS;

DATA CHARACTERISTICS; LABEL DISTRIBUTION; SEQUENCE LABELING; SEQUENCE PREDICTION;

LINEARIZATION;

EID: 85021626507     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.18653/v1/e17-1005     Document Type: Conference Paper

References (39)

1. Rie Kubota Ando and Tong Zhang. 2005. A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research, 6(Nov):1817-1853.
2. Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 1998. The Berkeley FrameNet project. In ACL.
3. Miguel Ballesteros, Chris Dyer, and Noah A. Smith. 2015. Improved Transition-based Parsing by Modeling Characters instead of Words with LSTMs. In EMNLP.
4. Johannes Bjerva, Barbara Plank, and Johan Bos. 2016. Semantic tagging with deep residual networks. In COLING.
5. Chloé Braud, Barbara Plank, and Anders Søgaard. 2016. Multi-view and multi-task training of rst discourse parsers. In COLING.
6. Rich Caruana. 1997. Multitask learning. In Learning to learn, pages 95-133. Springer.
7. Hao Cheng, Hao Fang, and Mari Ostendorf. 2015. Open-Domain Name Error Detection using a Multi- Task RNN. In EMNLP.
8. Kyunghyun Cho. 2015. Natural Language Understanding with Distributed Representation. ArXiv, abs/1511.07916.
9. Massimiliano Ciaramita and Yasemin Altun. 2006. Broad-coverage sense disambiguation and information extraction with a supersense sequence tagger. In EMNLP.
10. Trevor Cohn and Lucia Specia. 2013. Modelling annotator bias with multi-task gaussian processes: An application to machine translation quality estimation. In ACL.
11. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12(Aug):2493-2537.
12. D. Alan Cruse. 1986. Lexical semantics. Cambridge University Press.
13. Dipanjan Das, Desai Chen, André FT Martins, Nathan Schneider, and Noah A. Smith. 2014. Frame-semantic parsing. Computational linguistics, 40(1):9-56.
14. Lingjia Deng and Janyce Wiebe. 2015. Mpqa 3.0: An entity/event-level sentiment corpus. In NAACL.
15. Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, and Noah A. Smith. 2015. Transition- Based Dependency Parsing with Stack Long Short- Term Memory. In ACL.
16. Jeffrey L. Elman. 1990. Finding structure in time. Cognitive science, 14(2):179-211.
17. Yoav Goldberg. 2015. A Primer on Neural Network Models for Natural Language Processing. ArXiv, abs/1510.00726.
18. Alex Graves and Jürgen Schmidhuber. 2005. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks, 18(5):602-610.
19. Pankaj Gupta, Hinrich Schütze, and Bernt Andrassy. 2016. Table Filling Multi-Task Recurrent Neural Network for Joint Entity and Relation Extraction. In COLING.
20. Karl Moritz Hermann, Dipanjan Das, Jason Weston, and Kuzman Ganchev. 2014. Semantic frame identification with distributed word representations. In ACL.
21. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735-1780.
22. Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991.
23. Eliyahu Kiperwasser and Yoav Goldberg. 2016. Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations. TACL.
24. Meghana Kshirsagar, Sam Thomson, Nathan Schneider, Jaime Carbonell, Noah A. Smith, and Chris Dyer. 2015. Frame-semantic role labeling with heterogeneous annotations. In ACL-IJCNLP.
25. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In NAACL-HLT.
26. Pengfei Liu, Shafiq Joty, and Helen Meng. 2015. Finegrained opinion mining with recurrent neural networks and word embeddings. In EMNLP.
27. Minh-Thang Luong, Quoc V. Le, Ilya Sutskever, Oriol Vinyals, and Lukasz Kaiser. 2016. Multi-task sequence to sequence learning. In ICLR.
28. Xuezhe Ma and Eduard Hovy. 2016. End-to-end Sequence Labeling via Bi-directional LSTM-CNNs- CRF. arXiv preprint arXiv:1603.01354.
29. Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a Large Annotated Corpus of English: The Penn Treebank. Comput. Linguist., 19(2):x.
30. George A. Miller, Claudia Leacock, Randee Tengi, and Ross T. Bunker. 1993. A semantic concordance. In Proceedings of the workshop on Human Language Technology.
31. Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D. Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, et al. 2016. Universal dependencies v1: A multilingual treebank collection. In LREC.
32. Barbara Plank, Anders Søgaard, and Yoav Goldberg. 2016. Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Loss. In ACL.
33. Barbara Plank. 2016. Keystroke dynamics as signal for shallow syntactic parsing. In COLING.
34. Robert Rosenthal. 1979. The file drawer problem and tolerance for null results. Psychological bulletin, 86(3):638.
35. Kashif Shah and Lucia Specia. 2016. Large-scale multitask learning for machine translation quality estimation. In NAACL.
36. Anders Søgaard and Yoav Goldberg. 2016. Deep multi-task learning with low level tasks supervised at lower layers. In ACL.
37. Charles Sutton, Andrew McCallum, and Khashayar Rohanimanesh. 2007. Dynamic conditional random fields: Factorized probabilistic models for labeling and segmenting sequence data. Journal of Machine Learning Research, 8(Mar):693-723.
38. Erik F. Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. In HLT-NAACL, pages 142-147. Association for Computational Linguistics.
39. Piek Vossen, Laura Bloksma, Horacio Rodriguez, Salvador Climent, Nicoletta Calzolari, Adriana Roventini, Francesca Bertagna, Antonietta Alonge, and Wim Peters. 1998. The eurowordnet base concepts and top ontology. Deliverable D017 D, 34:D036.