VERTa: Linguistic features in MT evaluation

Proceedings of the 8th International Conference on Language Resources and Evaluation, LREC 2012

Volumn , Issue , 2012, Pages 3944-3950

  Comelles, Elisabet ^a   Atserias, Jordi ^b   Arranz, Victoria ^c   Castellón, Irene ^a  

^a UNIVERSITY OF BARCELONA   (Spain)

^b Fundació Barcelona Media   (Spain)

^c ELDA/ELRA   (France)

Author keywords

Automatic metric; Linguistically based metric; MT evaluation

Indexed keywords

LEARNING SYSTEMS; SEMANTICS; SYNTACTICS;

AUTOMATIC METRIC; LEXICAL INFORMATION; LINGUISTIC INFORMATION; LINGUISTIC KNOWLEDGE; LINGUISTICALLY-BASED METRIC; MACHINE LEARNING APPROACHES; MT EVALUATIONS; SEMANTIC INFORMATION;

LINGUISTICS;

EID: 85037100945     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (19)

1. Albrecth, J. S., Hwa R. (2007). A Re-examination of Machine Learning Approaches for Sentence-Level MT Evaluation. In The Proceedings of the 45th Annual Meeting of the ACL, Prague, Czech Republic.
2. Albrecth, J. S., Hwa R. (2007). Regression for Sentence-Level MT Evaluation with Pseudo References. In The Proceedings of the 45th Annual Meeting of the ACL, Prague, Czech Republic.
3. Callison-Burch, C., Osborne, M. And Koehn. P. (2006). Re-evaluating the role of BLEU in machine translation research. In Proceedings of the EACL 2006, pages 249-256.
4. Comelles, E., Arranz, V. And Castellon, I. (2010). Constituency and Dependency Parsers Evaluation. SEPLN (ed.), Revista de la Sociedad Española para el Procesamiento del Lenguaje Natural. Valencia: SEPLN, v. 45, p. 59-66. Valencia, Spain. ISSN: 1135-5948
5. Denkowski M. J., Lavie, A. (2011). METEOR 1. 3: Automatic Metric for Reliable Optimization and Evaluation of Machine Translation Systems. In Proceedings of the 6th Workshop on Statistical Machine Translation (ACL-2011), pages 85-91, Edinburgh, Scotland, UK.
6. Giménez, J., Màrquez, L. (2007). Linguistic features for automatic evaluation of heterogeneous MT systems. In Proceedings of the 2nd Workshop on Statistical Machine Translation (ACL), pages 256-264, Prague, Czech Republic.
7. Giménez J., Màrquez, L. (2008). A smorgasbord of features for automatic MT evaluation in Proceedings of the 3rd Workshop on Statistical Machine Translation (ACL), pages 195-198, Columbus, OH.
8. Gimenez, J., (2008). Empirical Machine Translation and its Evaluation. Doctoral Dissertation. UPC, Barcelona, Spain.
9. Giménez, J., Márquez, L. (2007). Linguistic Features for Automatic Evaluation of Heterogeneous MT Systems. In Proceedings of the ACL 2nd SMT Workshop, Prague, Czech Republic pages 256-264.
10. He, Y., Du J., Way, A. And van Genabith, J. (2010). The DCU Dependency-based Metric in WMT-Metrics MATR 2010. In Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and Metrics MATR (WMT 2010), Uppsala, Sweden, pages 349-353.
11. Lavie, A., Denkowski, M. J. (2009). The METEOR Metric for Automatic Evaluation of Machine Translation. Machine Translation, 23, Springer.
12. Leusch, G., Ney, H. (2008). BLEUSP, INVWER, CDER: Three improved MT evaluation measures. In NIST Metrics for Machine Translation 2008 Evaluation (MetricsMATR08), Waikiki, Honolulu, Hawaii, October 2008.
13. Liu, D., Hildea, D. (2005). Syntactic Features for Evaluation of Machine Translation. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, Ann Arbor, pages 25-32.
14. De Marneffe, M. C., MacCartney, B. And Manning, C. D. (2006). Generating Typed Dependency Parses from Phrase Structure Parses. In Proceedings of the 5th Edition of the International Conference on Language Resources and Evaluation (LREC-2006). Genoa, Italy.
15. De Marneffe, M. C., Manning, C. D. (2008). The Stanford typed dependencies representation. In COLING Workshop on Cross-framework and Cross-domain Parser Evaluation, Manchester, UK.
16. Owczarzak, K., van Genabith, J. And Way, A. (2007). Dependency-Based Automatic Evaluation for Machine Translation. In Proceedings of SSST, NAACL-HLT/AMTA Workshop on Syntax and Structure in Statistical Translation, Rochester, New York, pages 80-87.
17. Owczarzak, K., van Genabith, J. And Way, A. (2007). Labelled Dependencies in Machine Translation Evaluation in Proceedings of the ACL Workshop on Statistical Machine Translation, Prague, Czech Republic, pages 104-111,.
18. Papineni, K., Roukos, S., Ward. T. And Zhu, W. (2002). BLEU: A Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL02), Philadelphia. PA, pages 311-318.
19. Specia, L., Giménez, J. (2010). Combining Confidence Estimation and Reference-based Metrics for Segment-level MT Evaluation. In the 9th Conference of the Association for Machine Translation in the Americas, Denver, Colorado.