Distilling an ensemble of greedy dependency parsers into one MST parser

EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings

Volumn , Issue , 2016, Pages 1744-1753

  Kuncoro, Adhiguna ^a   Ballesteros, Miguel ^b   Kong, Lingpeng ^a   Dyer, Chris ^c   Smith, Noah A ^d  

^a Carnegie Mellon University ^*  (United States)

^b UNIVERSITAT POMPEU FABRA   (Spain)

^c DEEPMIND   (United Kingdom)

^d UNIVERSITY OF WASHINGTON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COST BENEFIT ANALYSIS; DISTILLATION; GRAPHIC METHODS; LONG SHORT-TERM MEMORY; NATURAL LANGUAGE PROCESSING SYSTEMS; UNCERTAINTY ANALYSIS;

CROSS ENTROPY; DEPENDENCY PARSER; FIRST ORDER; FIRST-ORDER GRAPHS; MINIMUM BAYES RISK; SINGLE MODELS; STATE OF THE ART; UNCERTAINTY ESTIMATES;

COMPUTATIONAL LINGUISTICS;

EID: 85021649927     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.18653/v1/d16-1180     Document Type: Conference Paper

References (44)

1. Daniel Andor, Chris Alberti, David Weiss, Aliaksei Severyn, Alessandro Presta, Kuzman Ganchev, Slav Petrov, and Michael Collins. 2016. Globally normalized transition-based neural networks. In Proc. of ACL.
2. Jimmy Ba and Rich Caruana. 2014. Do deep nets really need to be deep? In Proc. of NIPS.
3. Miguel Ballesteros, Chris Dyer, and Noah A. Smith. 2015. Improved transition-based parsing by modeling characters instead of words with LSTMs. In Proc. of EMNLP.
4. Miguel Ballesteros, Yoav Goldberg, Chris Dyer, and Noah A. Smith. 2016. Training with exploration improves a greedy stack-LSTM parser. In Proc. of EMNLP.
5. Bernd Bohnet and Joakim Nivre. 2012. A transition-based system for joint part-of-speech tagging and labeled non-projective dependency parsing. In Proc. of EMNLP-CoNLL.
6. Cristian Bucila, Rich Caruana, and Alexandru Niculescu-Mizil. 2006. Model compression. In Proc. of KDD.
7. Danqi Chen and Christopher Manning. 2014. A fast and accurate dependency parser using neural networks. In Proc. of EMNLP.
8. Marie-Catherine De Marneffe and Christopher D. Manning. 2008. Stanford typed dependencies manual.
9. Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, and Noah A. Smith. 2015. Transition-based dependency parsing with stack long short-term memory. In Proc. of ACL.
10. Jason M. Eisner. 1996. Three new probabilistic models for dependency parsing: An exploration. In Proc. of COLING.
11. Kevin Gimpel and Noah A Smith. 2010. Softmaxmargin CRFs: Training log-linear models with cost functions. In Proc. of NAACL.
12. Jan Hajic, Massimiliano Ciaramita, Richard Johansson, Daisuke Kawahara, Maria Antònia Martí, Lluís Màrquez, Adam Meyers, Joakim Nivre, Sebastian Padó, Jan Štepánek, Pavel Stranák, Mihai Surdeanu, Nianwen Xue, and Yi Zhang. 2009. The CoNLL-2009 Shared Task: Syntactic and semantic dependencies in multiple languages. In Proc. of CONLL 2009 Shared Task.
13. Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. 2015. Distilling the knowledge in a neural network. CoRR, abs/1503.02531.
14. Yoon Kim and Alexander M. Rush. 2016. Sequence-level knowledge distillation. In Proc. of EMNLP.
15. Eliyahu Kiperwasser and Yoav Goldberg. 2016. Simple and accurate dependency parsing using bidirectional LSTM feature representations. TACL.
16. Terry Koo and Michael Collins. 2010. Efficient third-order dependency parsers. In Proc. of ACL.
17. Phong Le and Willem Zuidema. 2014. The inside-outside recursive neural network model for dependency parsing. In Proc. of EMNLP.
18. Wang Ling, Chris Dyer, Alan W Black, and Isabel Trancoso. 2015. Two/too simple adaptations of word2vec for syntax problems. In Proc. of NAACL.
19. Xuezhe Ma and Hai Zhao. 2012. Fourth-order dependency parsing. In Proc. of COLING.
20. André F. T. Martins, Dipanjan Das, Noah A. Smith, and Eric P. Xing. 2008. Stacking dependency parsers. In Proc. of EMNLP.
21. André F. T. Martins, Noah A. Smith, and Eric P. Xing. 2009. Polyhedral outer approximations with application to natural language parsing. In Proc. of ICML.
22. André F. T. Martins, Mariana S.C. Almeida, and Noah A. Smith. 2013. Turning on the turbo: Fast third-order non-projective turbo parsers. In Proc. of ACL.
23. Ryan McDonald, Koby Crammer, and Fernando Pereira. 2005a. Online large-margin training of dependency parsers. In Proc. of ACL.
24. Ryan McDonald, Fernando Pereira, Kiril Ribarov, and Jan Hajic. 2005b. Non-projective dependency parsing using spanning tree algorithms. In Proc. of EMNLP.
25. Joakim Nivre and Ryan McDonald. 2008. Integrating graph-based and transition-based dependency parsers. In Proc. of ACL.
26. Joakim Nivre. 2003. An efficient algorithm for projective dependency parsing. In Proc. of IWPT.
27. Slav Petrov. 2010. Products of random latent variable grammars. In Proc.of NAACL.
28. Kenji Sagae and Alon Lavie. 2006. Parser combination by reparsing. In Proc. of NAACL.
29. David A. Smith and Jason Eisner. 2006. Minimum risk annealing for training log-linear models. In Proc. of ACL.
30. Mihai Surdeanu and Christopher D. Manning. 2010. Ensemble models for dependency parsing: Cheap and good? In Proc. of NAACL.
31. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proc. of NIPS.
32. Oscar Täckström, Ryan T. McDonald, and Joakim Nivre. 2013. Target language adaptation of discriminative transfer parsers. In Proc. of NAACL.
33. Ben Taskar, Vassil Chatalbashev, Daphne Koller, and Carlos Guestrin. 2005. Learning structured prediction models: A large margin approach. In Proc. of ICML.
34. Ivan Titov and James Henderson. 2006. Bayes risk minimization in natural language parsing. Technical report, University of Geneva.
35. Kristina Toutanova, Dan Klein, Christopher D. Manning, and Yoram Singer. 2003. Feature-rich part-of-speech tagging with a cyclic dependency network. In Proc. of NAACL.
36. Ioannis Tsochantaridis, Thorsten Joachims, Thomas Hofmann, and Yasemin Altun. 2005. Large margin methods for structured and interdependent output variables. JMLR.
37. Oriol Vinyals, Lukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey Hinton. 2015. Grammar as a foreign language. In Proc. of NIPS.
38. Wenhui Wang and Baobao Chang. 2016. Graph-based dependency parsing with bidirectional LSTM. In Proc. of ACL.
39. David Weiss, Chris Alberti, Michael Collins, and Slav Petrov. 2015. Structured training for neural network transition-based parsing. In Proc. of ACL.
40. Nianwen Xue, Fu-Dong Chiou, and Martha Palmer. 2002. Building a large-scale annotated chinese corpus. In Proc. of COLING.
41. Majid Yazdani and James Henderson. 2015. Incremental recurrent neural network dependency parser with search-based discriminative training. In Proc. of CoNLL.
42. Yue Zhang and Stephen Clark. 2008. A tale of two parsers: Investigating and combining graph-based and transition-based dependency parsing using beam-search. In Proc. of EMNLP.
43. Yue Zhang and Joakim Nivre. 2011. Transition-based dependency parsing with rich non-local features. In Proc. of ACL.
44. Chenxi Zhu, Xipeng Qiu, Xinchi Chen, and Xuanjing Huang. 2015. A re-ranking model for dependency parser with recursive convolutional neural network. In Proc. of ACL-IJCNLP.