Structured training for neural network transition-based parsing

ACL-IJCNLP 2015 - 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, Proceedings of the Conference

Volumn 1, Issue , 2015, Pages 323-333

  Weiss, David ^a   Alberti, Chris ^a   Collins, Michael ^a   Petrov, Slav ^a  

^a GOOGLE INC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; NATURAL LANGUAGE PROCESSING SYSTEMS; SYNTACTICS;

BEAM SEARCH; DEPENDENCY PARSING; FIXED NETWORKS; NETWORK REPRESENTATION; NETWORK TRANSITION; STANFORD; TREEBANKS;

NEURAL NETWORKS;

EID: 84943750795     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/v1/p15-1032     Document Type: Conference Paper

References (45)

1. Bernd Bohnet and Jonas Kuhn. 2012. The best of both worlds: A graph-based completion model for transition-based parsers. In Proc. EACL, pages 77-87.
2. Bernd Bohnet. 2010. Top accuracy and fast dependency parsing is not a contradiction. In Proc. 23rd International Conference on Computational Linguistics (Coling 2010), pages 89-97.
3. Ĺeon Bottou. 2010. Large-scale machine learning with stochastic gradient descent. In Proc. COMPSTAT, pages 177-186.
4. Sabine Buchholz and Erwin Marsi. 2006. Conll-x shared task on multilingual dependency parsing. In Proc. CoNLL, pages 149-164.
5. Ciprian Chelba, Tomas Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, and Phillipp Koehn. 2013. One billion word benchmark for measuring progress in statistical language modeling. CoRR.
6. Danqi Chen and Christopher D. Manning. 2014. A fast and accurate dependency parser using neural networks. In Proc. EMNLP, pages 740-750.
7. Wenliang Chen, Min Zhang, and Yue Zhang. 2013. Semi-supervised feature transformation for dependency parsing. In Proc. 2013 EMNLP, pages 1303-1313.
8. Michael Collins and Brian Roark. 2004. Incremental parsing with the perceptron algorithm. In Proc. ACL, Main Volume, pages 111-118, Barcelona, Spain.
9. Marie-Catherine De Marneffe, Bill MacCartney, and Christopher D. Manning. 2006. Generating typed dependency parses from phrase structure parses. In Proc. LREC, pages 449-454.
10. Trinh Minh Tri Do and Thierry Artires. 2010. Neural conditional random fields. In AISTATS, volume 9, pages 177-184.
11. Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, and Noah A. Smith. 2015. Transitionbased dependency parsing with stack long shortterm memory. In Proc. ACL.
12. James Henderson. 2004. Discriminative training of a neural network statistical parser. In Proc. ACL, Main Volume, pages 95-102.
13. Geoffrey E. Hinton. 2012. A practical guide to training restricted boltzmann machines. In Neural Networks: Tricks of the Trade (2nd ed.), Lecture Notes in Computer Science, pages 599-619. Springer.
14. Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and RalphWeischedel. 2006. Ontonotes: The 90% solution. In Proc. HLT-NAACL, pages 57-60.
15. Zhongqiang Huang and Mary Harper. 2009. Selftraining PCFG grammars with latent annotations across languages. In Proc. 2009 EMNLP, pages 832-841, Singapore.
16. John Judge, Aoife Cahill, and Josef van Genabith. 2006. Questionbank: Creating a corpus of parseannotated questions. In Proc. ACL, pages 497-504.
17. Terry Koo, Xavier Carreras, and Michael Collins. 2008. Simple semi-supervised dependency parsing. In Proc. ACL-HLT, pages 595-603.
18. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Proc. NIPS, pages 1097-1105.
19. Zhenghua Li, Min Zhang, and Wenliang Chen. 2014. Ambiguity-Aware ensemble training for semisupervised dependency parsing. In Proc. ACL, pages 457-467.
20. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313-330.
21. Andre Martins, Miguel Almeida, and Noah A. Smith. 2013. Turning on the turbo: Fast third-order nonprojective turbo parsers. In Proc. ACL, pages 617-622.
22. David McClosky, Eugene Charniak, and Mark Johnson. 2006. Effective self-Training for parsing. In Proc. HLT-NAACL, pages 152-159.
23. Ryan McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav Goldberg, Dipanjan Das, Kuzman Ganchev, Keith Hall, Slav Petrov, Hao Zhang, Oscar Täckström, Claudia Bedini, Nuria Bertomeu Castelĺo, and Jungmee Lee. 2013. Universal dependency annotation for multilingual parsing. In Proc. ACL, pages 92-97.
24. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. CoRR, abs/1301.3781.
25. Vinod Nair and Geoffrey E. Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In Proc. 27th ICML, pages 807-814.
26. Joakim Nivre, Johan Hall, Sandra Kübler, Ryan Mc-Donald, Jens Nilsson, Sebastian Riedel, and Deniz Yuret. 2007. The CoNLL 2007 shared task on dependency parsing. In Proc. CoNLL, pages 915-932.
27. Joakim Nivre. 2004. Incrementality in deterministic dependency parsing. In Proc. ACL Workshop on Incremental Parsing, pages 50-57.
28. Joakim Nivre. 2008. Algorithms for deterministic incremental dependency parsing. Computational Linguistics, 34(4):513-553.
29. Jian Peng, Liefeng Bo, and Jinbo Xu. 2009. Conditional neural fields. In Proc. NIPS, pages 1419-1427.
30. Slav Petrov and Ryan McDonald. 2012. Overview of the 2012 shared task on parsing the web. Notes of the First Workshop on Syntactic Analysis of Non-Canonical Language (SANCL).
31. Slav Petrov, Leon Barrett, Romain Thibaux, and Dan Klein. 2006. Learning accurate, compact, and interpretable tree annotation. In Proc. ACL, pages 433-440.
32. Slav Petrov, Pi-Chuan Chang, Michael Ringgaard, and Hiyan Alshawi. 2010. Uptraining for accurate deterministic question parsing. In Proc. EMNLP, pages 705-713.
33. Adwait Ratnaparkhi. 1997. A linear observed time statistical parser based on maximum entropy models. In Proc. EMNLP, pages 1-10.
34. Jun Suzuki, Hideki Isozaki, Xavier Carreras, and Michael Collins. 2009. An empirical study of semisupervised structured conditional models for dependency parsing. In Proc. 2009 EMNLP, pages 551-560.
35. Jun Suzuki, Hideki Isozaki, and Masaaki Nagata. 2011. Learning condensed feature representations from large unsupervised data sets for supervised learning. In Proc. ACL-HLT, pages 636-641.
36. Ivan Titov and James Henderson. 2007. Fast and robust multilingual dependency parsing with a generative latent variable model. In Proc. EMNLP, pages 947-951.
37. Ivan Titov and James Henderson. 2010. A latent variable model for generative dependency parsing. In Trends in Parsing Technology, pages 35-55. Springer.
38. Kristina Toutanova, Dan Klein, Christopher D. Manning, and Yoram Singer. 2003. Feature-rich part-ofspeech tagging with a cyclic dependency network. In NAACL.
39. Oriol Vinyals, Lukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey E. Hinton. 2015. Grammar as a foreign language. arXiv:1412.7449.
40. Hiroyasu Yamada and Yuji Matsumoto. 2003. Statistical dependency analysis with support vector machines. In Proc. IWPT, pages 195-206.
41. Matthew D. Zeiler, MarcAurelio Ranzato, Rajat Monga, Mark Z. Mao, K. Yang, Quoc Viet Le, Patrick Nguyen, Andrew W. Senior, Vincent Vanhoucke, Jeffrey Dean, and Geoffrey E. Hinton. 2013. On rectified linear units for speech processing. In IEEE International Conference on Acoustics, Speech and Signal Processing, pages 3517-3521.
42. Yue Zhang and Stephen Clark. 2008. A tale of two parsers: investigating and combining graphbased and transition-based dependency parsing using beam-search. In Proc. EMNLP, pages 562-571.
43. Hao Zhang and Ryan McDonald. 2014. Enforcing structural diversity in cube-pruned dependency parsing. In Proc. ACL, pages 656-661.
44. Yue Zhang and Joakim Nivre. 2011. Transition-based dependency parsing with rich non-local features. In Proc. ACL-HLT, pages 188-193.
45. Hao Zhou, Yue Zhang, and Jiajun Chen. 2015. A neural probabilistic structured-prediction model for transition-based dependency parsing. In Proc. ACL