Semantic tagging with deep residual networks

COLING 2016 - 26th International Conference on Computational Linguistics, Proceedings of COLING 2016: Technical Papers

Volumn , Issue , 2016, Pages 3531-3541

  Bjerva, Johannes ^a   Plank, Barbara ^a   Bos, Johan ^a  

^a UNIVERSITY OF GRONINGEN   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

SEMANTIC WEB; SEMANTICS;

LOSS FUNCTIONS; PART OF SPEECH TAGGING; POS TAGGING; SEMANTIC PARSING; SEMANTIC TAGGING; SEMANTIC TAGS; WORD AND CHARACTERS;

COMPUTATIONAL LINGUISTICS;

EID: 85021654356     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (43)

1. Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Gregory S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian J. Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Józefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mane, Rajat Monga, Sherry Moore, Derek Gordon Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul A. Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda B. Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. Tensorflow: Farge-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv: 1603.04467.
2. Rami Al-Rfou, Bryan Perozzi, and Steven Skiena. 2013. Polyglot: Distributed word representations for multilingual nlp. CoNLL-2013.
3. Jonathan Berant and Percy Fiang. 2014. Semantic parsing via paraphrasing. In ACL, pages 1415-1425.
4. Johannes Bjerva. 2016. Byte-based language identification with deep convolutional networks. arXiv preprint arXiv: 1609.09004.
5. Johan Bos, Valerio Basile, Kilian Evang, Noortje Venhuizen, and Johannes Bjerva. Forthcoming. The Groningen Meaning Bank. In Nancy Ide and James Pustejovsky, editors, The Handbook of Linguistic Annotation. Springer, Berlin.
6. Johan Bos. 2008. Wide-Coverage Semantic Analysis with Boxer. In J. Bos and R. Delmonte, editors, Semantics in Text Processing. STEP 2008 Conference Proceedings, volume 1 of Research in Computational Semantics, pages 277-286. College Publications.
7. Thorsten Brants. 2000. Tnt: a statistical part-of-speech tagger. In Proceedings of the sixth conference on Applied natural language processing, pages 224-231. Association for Computational Finguistics.
8. Alastair Butler. 2010. The Semantics of Grammatical Dependencies, Volume 23. Emerald Group Publishing Fimited.
9. Hao Cheng, Hao Fang, and Mari Ostendorf. 2015. Open-domain name error detection using a multi-task rnn. In EMNLP.
10. Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Fearning phrase representations using rnn encoder-decoder for statistical machine translation. In EMNLP.
11. François Chollet. 2015. Keras. https://github.com/fchollet/keras.
12. Grzegorz Chrupala. 2013. Text segmentation with character-level text embeddings. In Workshop on Deep Learning for Audio, Speech and Language Processing, ICML.
13. Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv: 1412.3555.
14. Junyoung Chung, Kyunghyun Cho, and Yoshua Bengio. 2016. A character-level decoder without explicit segmentation for neural machine translation. Procedings of ACL 2016, arXiv preprint arXiv: 1603.06147.
15. Alexis Conneau, Holger Schwenk, Loïc Barrault, and Yann Fecun. 2016. Very deep convolutional networks for natural language processing. arXiv preprint arXiv: 1606.01781.
16. Ann Copestake, Dan Flickinger, Ivan Sag, and Carl Pollard. 2005. Minimal recursion semantics: An introduction. Journal of Research on Language and Computation, 3(2-3): 281-332.
17. James Curran, Stephen Clark, and Johan Bos. 2007. Linguistically Motivated Large-Scale NLP with C&C and Boxer. In Procedings of ACL 2007, pages 33-36, Prague, Czech Republic.
18. Cicero Nogueira dos Santos and Bianca Zadrozny. 2014. Learning character-level representations for part-of-speech tagging. In ICML, pages 1818-1826.
19. Bradley Efron and Robert J Tibshirani. 1994. An introduction to the bootstrap. CRC press.
20. Kilian Evang, Valerio Basile, Grzegorz Chrupala, and Johan Bos. 2013. Elephant: Sequence labeling for word and sentence segmentation. In EMNLP, pages 1422-1426.
21. Dan Gillick, Cliff Brunk, Oriol Vinyals, and Amarnag Subramanya. 2015. Multilingual language processing from bytes. arXiv preprint arXiv: 1512.00103.
22. Yoav Goldberg. 2015. A primer on neural network models for natural language processing. arXiv preprint arXiv: 1510.00726.
23. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep learning. http://www.deeplearningbook.org. Book in preparation for MIT Press.
24. Alex Graves and Jürgen Schmidhuber. 2005. Framewise phoneme classification with bidirectional 1stm and other neural network architectures. Neural Networks, 18(5): 602-610.
25. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Deep residual learning for image recognition. arXiv preprint arXiv: 1512.03385.
26. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Identity mappings in deep residual networks. arXiv preprint arXiv: 1603.05027.
27. Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv: 1502.03167.
28. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6980.
29. Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11): 2278-2324.
30. M.P. Marcus, B. Santorini, and M.A. Marcinkiewicz. 1993. Building a Large Annotated Corpus of English: The Penn Treebank. Computational Linguistics, 19(2): 313-330.
31. Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning (ICML-10), pages 807-814.
32. Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, et al. 2016. Universal dependencies vl: A multilingual treebank collection. In Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC 2016).
33. Robert Östling. 2016. Morphological reinflection with convolutional neural networks. In Proceedings of the 2016 Meeting of SIGMORPHON, Berlin, Germany. Association for Computational Linguistics.
34. Barbara Plank, Anders Søgaard, and Yoav Goldberg. 2016. Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss. In Proceedings of ACL 2016, arXiv preprint arXiv: 1604.05529.
35. Sameer S Pradhan, Wayne Ward, Kadri Hacioglu, James H Martin, and Daniel Jurafsky. 2004. Shallow semantic parsing using support vector machines. In HLT-NAACL, pages 233-240.
36. Anders Søgaard, Anders Johannsen, Barbara Plank, Dirk Hovy, and Hector Martinez. 2014. Whats in a p-value in nlp? In CoNLL-2014.
37. Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1): 1929-1958.
38. Ilya Sutskever, James Martens, and Geoffrey E Hinton. 2011. Generating text with recurrent neural networks. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pages 1017-1024.
39. Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1-9.
40. Peilu Wang, Yao Qian, Frank K Soong, Lei He, and Hai Zhao. 2015. A unified tagging solution: Bidirectional 1stm recurrent neural network with word embedding. arXiv preprint arXiv: 1511.00215.
41. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google's neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv: 1609.08144.
42. Zhilin Yang, Ruslan Salakhutdinov, and William Cohen. 2016. Multi-task cross-lingual sequence tagging from scratch. arXiv preprint arXiv: 1603.06270.
43. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Advances in Neural Information Processing Systems, pages 649-657.