Parser evaluation and the BNC: Evaluating 4 constituency parsers with 3 metrics

Proceedings of the 6th International Conference on Language Resources and Evaluation, LREC 2008

Volumn , Issue , 2008, Pages 3047-3051

  Foster, Jennifer ^a   Van Genabith, Josef ^a  

^a DUBLIN CITY UNIVERSITY   (Ireland)

Author keywords

[No Author keywords available]

Indexed keywords

FORESTRY;

BRITISH NATIONAL CORPORA; DEPENDENCY STRUCTURES; DEPENDENCY TREES; EVALUATION METRICS; NORTH AMERICAN; PHRASE STRUCTURE TREES; STATE OF THE ART; TRAINING MATERIAL;

COMPUTATIONAL LINGUISTICS;

EID: 84943783874     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (20)

1. Ann Bies, Mark Ferguson, Karen Katz, and Robert MacIntyre. 1995. Bracketing guidelines for Treebank II style, Penn Treebank project. Technical Report Tech Report MS-CIS-95-06, University of Pennsylvania, Philadelphia, PA.
2. Ezra Black, Steve Abney, Dan Flickinger, Claudia Gdaniec, Robert Grishman, Philip Harrison, Donald Hindle, Robert Ingria, Fred Jelinek, Judith Klavans, Mark Liberman, Mitchell Marcus, Salim Roukos, Beatrice Santorini, and Tomek Strzalkowski. 1991. A procedure for quantitatively comparing the syntactic coverage of english grammars. In Proceedings of the 1991 DARPA Speech and Natural Language Workshop, pages 306-311.
3. Lou Burnard. 2000. User reference guide for the British National Corpus. Technical report, Oxford University Computing Services.
4. John Carroll, Anette Frank, Dekang Lin, Detlef Prescher, and Hans Uszkoreit, editors. 2002. Proceedings of the "Beyond Parseval - Towards Improved Evaluation Measures for Parsing Systems" Workshop at the 3rd International Conference on Linguistic Resources and Evaluation (LREC-02), Las Palmas, Gran Canaria.
5. Eugene Charniak and Mark Johnson. 2005. Course-tofine n-best-parsing and maxent discriminative reranking. In Proceedings of the 43rd Annual Meeting of the ACL (ACL-05), pages 173-180, Ann Arbor, Michigan, June.
6. Eugene Charniak. 2000. A maximum-entropy-inspired parser. In Proceedings of the Annual Meeting of the North American Association for Computational Linguistics (NAACL-00), pages 132-139, Seattle, Washington.
7. Grzegorz Chrupala, Nicolas Stroppa, Josef van Genabith, and Georgiana Dinu. 2007. Better training for function labeling. In Proceedings of the Recent Advances in Natural Language Processing Conference (RANLP-07), pages 133-138, Borovets, Bulgaria.
8. Jennifer Foster, Joachim Wagner, Djamé Seddah, and Josef van Genabith. 2007. Adapting WSJ-trained parsers to the British National Corpus using in-domain self-training. In Proceedings of the Tenth International Workshop on Parsing Technologies (IWPT-07), pages 33-35, Prague, Czech Republic.
9. Richard Johansson and Pierre Nugues. 2007. Extended constituent-to-dependency conversion for English. In Joakim Nivre, Heiki-Jaan Kaalep, Kadri Muischnek, and Mare Koit, editors, Proceedings of NODALIDA 2007, pages 105-112, Tartu, Estonia.
10. Sandra Kübler and Heike Telljohann. 2002. Towards a dependency-oriented evaluation for partial parsing. In Carroll et al. (Carroll et al., 2002), pages 9-16.
11. Dekang Lin. 1998. Dependency-based evaluation of MINIPAR. In John Carroll, editor, Proceedings of The Evaluation of Parsing Systems Workshop at the 3rd International Conference on Linguistic Resources and Evaluation (LREC), Cognitive Science Research Papers 489, pages 48-56. University of Sussex, Brighton, England.
12. Mitchell Marcus, Grace Kim, Mary Ann Marcinkiewicz, Robert MacIntyre, Ann Bies, Mark Ferguson, Karen Katz, and Britta Schasberger. 1994. The Penn Treebank: Annotating Predicate Argument Structure. In Proceedings of the ARPA Workshop on Human Language Technology, pages 110-115, Princeton, NJ.
13. David McClosky, Eugene Charniak, and Mark Johnson. 2006a. Effective self-training for parsing. In Proceedings of the Human Language Technology Conference and North American chapter of the ACL annual meeting (HLT-NAACL-06), pages 152-159, New York, June.
14. David McClosky, Eugene Charniak, and Mark Johnson. 2006b. Reranking and self-training for parser adaptation. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL (COLING-ACL-06), pages 337-344, Sydney, Australia, July.
15. Joakim Nivre, Johan Hall, Sandra Kübler, Ryan MacDonald, Jens Nilsson, Sebastian Riedel, and Deniz Yuret. 2007. The CoNLL 2007 shared task on dependency parsing. In Proceedings of the CoNLL Shared Task Session of EMNLP-CoNLL 2007, pages 915-932, Prague, Czech Republic, June.
16. Slav Petrov, Leon Barrett, Romain Thibaux, and Dan Klein. 2006. Learning accurate, compact, and interpretable tree annotation. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 433-440, Sydney, Australia.
17. Ines Rehbein and Josef van Genabith. 2007. Treebank annotation schemes and parser evaluation for German. In Proceedings of the Joint EMNLP-CoNLL 2007, pages 630-639, Prague, Czech Republic.
18. Geoffrey Sampson and Anna Babarczy. 2002. A test of the leaf-ancestor metric for parse accuracy. In Carroll et al. (Carroll et al., 2002).
19. Joachim Wagner, Djamé Seddah, Jennifer Foster, and Josef van Genabith. 2007. C-structures and f-structures for the British National Corpus. In Proceedings of the 12th International Workshop on Lexical Functional Grammar, Stanford, CA.
20. Hiroyasu Yamada and Yuji Matsumoto. 2003. Statistical dependency analysis with support vector machines. In Proceedings of the 8th International Workshop on Parsing Technologies (IWPT-03), pages 195-206, Nancy, France.