Parser showdown at the wall street corral: An empirical investigation of error types in parser output

EMNLP-CoNLL 2012 - 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Proceedings of the Conference

Volumn , Issue , 2012, Pages 1048-1059

  Kummerfeld, Jonathan K ^a   Hall, David ^a   Curran, James R ^b   Klein, Dan ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF SYDNEY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

EMPIRICAL INVESTIGATION; ERROR TYPES; F-SCORE; LINGUISTIC INFORMATION; TREE TRANSFORMATION; WALL STREETS;

ERRORS; LINGUISTICS; REPAIR;

NATURAL LANGUAGE PROCESSING SYSTEMS;

EID: 84883363441     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (42)

1. S. Abney, S. Flickenger, C. Gdaniec, C. Grishman, P. Harrison, D. Hindle, R. Ingria, F. Jelinek, J. Klavans, M. Liberman, M. Marcus, S. Roukos, B. Santorini, and T. Strzalkowski. 1991. Procedure for quantitatively comparing the syntactic coverage of english grammars. In Proceedings of the workshop on Speech and Natural Language, pages 306-311, Pacific Grove, California, USA, February.
2. Michael Auli and Adam Lopez. 2011. A comparison of loopy belief propagation and dual decomposition for integrated ccg supertagging and parsing. In Proceedings of ACL, pages 470-480, Portland, Oregon, USA, June.
3. Emily M. Bender, Dan Flickinger, Stephan Oepen, and Yi Zhang. 2011. Parser evaluation over local and nonlocal deep dependencies in a large corpus. In Proceedings of EMNLP, pages 397-408, Edinburgh, United Kingdom, July.
4. Daniel M. Bikel. 2004. Intricacies of collins' parsing model. Computational Linguistics, 30(4):479-511.
5. Nathan Bodenstab, Aaron Dunlop, Keith Hall, and Brian Roark. 2011. Beam-width prediction for efficient context-free parsing. In Proceedings of ACL, pages 440-449, Portland, Oregon, USA, June.
6. Ted Briscoe and John Carroll. 2006. Evaluating the accuracy of an unlexicalized statistical parser on the PARC DepBank. In Proceedings of ACL, pages 41-48, Sydney, Australia, July.
7. Ted Briscoe, John Carroll, Jonathan Graham, and Ann Copestake, 2002. Relational Evaluation Schemes, pages 4-8. Las Palmas, Canary Islands, Spain, May.
8. John Carroll, Ted Briscoe, and Antonio Sanfilippo. 1998. Parser evaluation: a survey and a new proposal. In Proceedings of LREC, pages 447-454, Granada, Spain, May.
9. Daniel Cer, Marie-Catherine de Marneffe, Daniel Jurafsky, and Christopher D. Manning. 2010. Parsing to stanford dependencies: Trade-offs between speed and accuracy. In Proceedings of LREC, Valletta, Malta, May.
10. Eugene Charniak and Mark Johnson. 2005. Coarse-to-fine n-best parsing and maxent discriminative reranking. In Proceedings of ACL, pages 173-180, Ann Arbor, Michigan, USA, June.
11. Eugene Charniak. 2000. A maximum-entropy-inspired parser. In Proceedings of NAACL, pages 132-139, Seattle, Washington, USA, April.
12. Stephen Clark and Julia Hockenmaier. 2002. Evaluating a wide-coverage ccg parser. In Proceedings of the LREC Beyond Parseval Workshop, Las Palmas, Canary Islands, Spain, May.
13. Andrew B. Clegg and Adrian J. Shepherd. 2005. Evaluating and integrating treebank parsers on a biomedical corpus. In Proceedings of the ACL Workshop on Software, pages 14-33, Ann Arbor, Michigan, USA, June.
14. Michael Collins. 1997. Three generative, lexicalised models for statistical parsing. In Proceedings of ACL, pages 16-23, Madrid, Spain, July.
15. Michael Collins. 2000. Discriminative reranking for natural language parsing. In Proceedings of ICML, pages 175-182, Palo Alto, California, USA, June.
16. Michael Collins. 2003. Head-driven statistical models for natural language parsing. Computational Linguistics, 29(4):589-637.
17. Rebecca Dridan and Stephan Oepen. 2011. Parser evaluation using elementary dependency matching. In Proceedings of IWPT, pages 225-230, Dublin, Ireland, October.
18. Aaron Dunlop, Nathan Bodenstab, and Brian Roark. 2011. Efficient matrix-encoded grammars and low latency parallelization strategies for cyk. In Proceedings of IWPT, pages 163-174, Dublin, Ireland, October.
19. Jennifer Foster and Josef van Genabith. 2008. Parser evaluation and the bnc: Evaluating 4 constituency parsers with 3 metrics. In Proceedings of LREC, Mar-rakech, Morocco, May.
20. Daniel Gildea. 2001. Corpus variation and parser performance. In Proceedings of EMNLP, pages 167-202, Pittsburgh, Pennsylvania, USA, June.
21. Tadayoshi Hara, Yusuke Miyao, and Jun'ichi Tsujii. 2007. Evaluating impact of re-training a lexical disambiguation model on domain adaptation of an hpsg parser. In Proceedings of IWPT, pages 11-22, Prague, Czech Republic, June.
22. Tadayoshi Hara, Yusuke Miyao, and Jun'ichi Tsujii. 2009. Descriptive and empirical approaches to capturing underlying dependencies among parsing errors. In Proceedings of EMNLP, pages 1162-1171, Singapore, August.
23. James Henderson. 2003. Inducing history representations for broad coverage statistical parsing. In Proceedings of NAACL, pages 24-31, Edmonton, Canada, May.
24. James Henderson. 2004. Discriminative training of a neural network statistical parser. In Proceedings of ACL, pages 95-102, Barcelona, Spain, July.
25. Liang Huang. 2008. Forest reranking: Discriminative parsing with non-local features. In Proceedings of ACL, pages 586-594, Columbus, Ohio, USA, June.
26. Tracy H. King, Richard Crouch, Stefan Riezler, Mary Dalrymple, and Ronald M. Kaplan. 2003. The PARC 700 dependency bank. In Proceedings of the 4th International Workshop on Linguistically Interpreted Corpora at EACL, Budapest, Hungary, April.
27. Dan Klein and Christopher D. Manning. 2003a. Accurate unlexicalized parsing. In Proceedings of ACL, pages 423-430, Sapporo, Japan, July.
28. Dan Klein and Christopher D. Manning. 2003b. Fast exact inference with a factored model for natural language parsing. In Proceedings of NIPS, pages 3-10, Vancouver, British Columbia, Canada, December.
29. Dekang Lin. 1998. A dependency-based method for evaluating broad-coverage parsers. Natural Language Engineering, 4(2):97-114.
30. Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of english: the penn treebank. Computational Linguistics, 19(2):313-330.
31. David McClosky, Eugene Charniak, and Mark Johnson. 2006. Effective self-training for parsing. In Proceedings of NAACL, pages 152-159, New York, New York, USA, June.
32. Ryan McDonald and Joakim Nivre. 2007. Characterizing the errors of data-driven dependency parsing models. In Proceedings of EMNLP, pages 122-131, Prague, Czech Republic, June.
33. Yusuke Miyao, Rune Sætre, Kenji Sagae, Takuya Matsuzaki, and Jun'ichi Tsujii. 2008. Task-oriented evaluation of syntactic parsers and their representations. In Proceedings of ACL, pages 46-54, Columbus, Ohio, USA, June.
34. Dominick Ng and James R. Curran. 2012. N-best CCG parsing and reranking. In Proceedings of ACL, Jeju, South Korea, July.
35. Dominick Ng, Matthew Honnibal, and James R. Curran. 2010. Reranking a wide-coverage ccg parser. In Proceedings of ALTA, pages 90-98, Melbourne, Australia, December.
36. Joakim Nivre, Laura Rimell, Ryan McDonald, and Carlos Gómez-Rodríguez. 2010. Evaluation of dependency parsers on unbounded dependencies. In Proceedings of Coling, pages 833-841, Beijing, China, August.
37. Slav Petrov and Dan Klein. 2007. Improved inference for unlexicalized parsing. In Proceedings of NAACL, pages 404-411, Rochester, New York, USA, April.
38. Slav Petrov and Ryan McDonald. 2012. SANCL Shared Task. LDC2012E43. Linguistic Data Consortium. Philadelphia, Philadelphia, USA.
39. Slav Petrov, Leon Barrett, Romain Thibaux, and Dan Klein. 2006. Learning accurate, compact, and inter-pretable tree annotation. In Proceedings of ACL, pages 433-440, Sydney, Australia, July.
40. Chris Quirk and Simon Corston-Oliver. 2006. The impact of parse quality on syntactically-informed statistical machine translation. In Proceedings of EMNLP, pages 62-69, Sydney, Australia, July.
41. Kun Yu, Yusuke Miyao, Takuya Matsuzaki, Xiangli Wang, and Junichi Tsujii. 2011. Analysis of the difficulties in chinese deep parsing. In Proceedings of IWPT, pages 48-57, Dublin, Ireland, October.
42. Deniz Yuret, Aydin Han, and Zehra Turgut. 2010. Semeval-2010 task 12: Parser evaluation using textual entailments. In Proceedings of the 5th International Workshop on Semantic Evaluation, pages 51-56, Uppsala, Sweden, July.