Ipa and stout: Leveraging linguistic and source-based features for machine translation evaluation

Proceedings of the Annual Meeting of the Association for Computational Linguistics

Volumn , Issue , 2014, Pages 394-401

  Gonzalez, Meritxell ^a   Cedeno, Alberto Barron ^a   Marquez, Lluis ^b  

^a UNIVERSITAT POLITÈCNICA DE CATALUNYA   (Spain)

^b QATAR COMPUTING RESEARCH INSTITUTE   (Qatar)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; COMPUTER AIDED LANGUAGE TRANSLATION; MACHINE TRANSLATION;

EVALUATION MEASURES; LANGUAGE PAIRS; LINGUISTIC STRUCTURE; MACHINE TRANSLATION EVALUATIONS; STANDARD METRICS; SYSTEM LEVELS; TARGET LANGUAGE;

SEMANTICS;

EID: 85122612495     PISSN: 0736587X     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (31)

1. Alberto Barron-Cedeno, Monica Lestari Paramita, Paul Clough, and Paolo Rosso. 2014. A Comparison of Approaches for Measuring Cross-Lingual Similarity of Wikipedia Articles. Advances in Information Retrieval. Proceedings of the 36th European Conference on IR Research, LNCS (8416):424?429. Springer-Verlag.
2. Ondrej Bojar, Christian Buck, Chris Callison-Burch, Christian Federmann, Barry Haddow, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Specia. 2013. Findings of the 2013 Workshop on Statistical Machine Translation. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 1?44, Sofia, Bulgaria, August. Association for Computational Linguistics.
3. Peter F. Brown, Jennifer C. Lai, and Robert L. Mercer. 1991. Aligning Sentences in Parallel Corpora. In Douglas E. Appelt, editor, Proceedings of the 29th Annual Meeting of the Association for Computational Linguistics (ACL 1991), pages 169?176, Berkeley, CA, USA. Association for Computational Linguistics.
4. Marie Candito, Benot Crabb, and Pascal Denis. 2010a. Statistical French dependency parsing: Treebank conversion and first results. In The seventh international conference on Language Resources and Evaluation (LREC), Valletta, Malta.
5. Marie Candito, Joakim Nivre, Pascal Denis, and Enrique Henestroza Anguiano. 2010b. Benchmarking of Statistical Dependency Parsers for French. In Proc. 23rd Intl. COLING Conference on Computational Linguistics: Poster Volume, pages 108?116, Beijing, China.
6. Eugene Charniak and Mark Johnson. 2005. Coarseto-Fine N-best Parsing and MaxEnt Discriminative Reranking. In Proc. 43rd Annual Meeting of the Association for Computational Linguistics (ACL), pages 173?180, Stroudsburg, PA, USA. Association for Computational Linguistics.
7. Michael Denkowski and Alon Lavie. 2011. Meteor 1.3: Automatic Metric for Reliable Optimization and Evaluation of Machine Translation Systems. In Proceedings of the 6th Workshop on Statistical Machine Translation, pages 85?91, Stroudsburg, PA, USA. Association for Computational Linguistics.
8. George Doddington. 2002. Automatic Evaluation of Machine Translation Quality Using N-gram Co-Occurrence Statistics. In Proceedings of the 2nd International Conference on Human Language Technology, pages 138?145, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.
9. E. Gabrilovich and S. Markovitch. 2007. Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis. In 20th International Joint Conference on Artificial Intelligence, pages 1606? 1611, San Francisco, CA, USA.
10. William A. Gale and Kenneth, W. Church. 1993. A Program for Aligning Sentences in Bilingual Corpora. Computational Linguistics, 19:75?102.
11. Jesus Gimenez and Lluis Marquez. 2007. Linguistic Features for Automatic Evaluation of Heterogeneous MT Systems. In Proc. of 2nd Workshop on statistical Machine Translation (WMT07), ACL07, Prague, Czech Republic.
12. Jesus Gimenez and Lluis Marquez. 2010a. Asiya: An Open Toolkit for Automatic Machine Translation (Meta-)Evaluation. The Prague Bulletin of Mathematical Linguistics, 94:77?86.
13. Jesus Gimenez and Lluis Marquez. 2010b. Linguistic Measures for Automatic Machine Translation Evaluation. Machine Translation, 24(3?4):77?86.
14. Jesus Gimenez. 2008. Empirical Machine Translation and its Evaluation. Ph.D.Thesis, Universitat Politcnica de Catalunya, July.
15. Meritxell Gonzalez and Jesus Gimenez. 2014. Asiya: An Open Toolkit for Automatic Machine Translation (Meta-)Evaluation, v3.0, February. http://asiya.lsi.upc.edu.
16. Meritxell Gonzalez, Jesus Gimenez, and Lluis Marquez. 2012. A Graphical Interface for MT Evaluation and Error Analysis. In Proc. Annual Meeting of the Association for Computational Linguistics (ACL). System Demonstration, pages 139?144, Jeju, South Korea, July. Association for Computational Linguistics.
17. Meritxell Gonzalez, Laura Mascarell, and Lluis Marquez. 2013.TSearch: Flexible and Fast Search over Automatic translation for Improved Quality/Error Analysis. In Proc. 51st Annual Meeting of the Association for Computational Linguistics (ACL). System Demonstration, pages 181?186, Sofia, Bulgaria, August.
18. Paul Jaccard. 1901. Etude comparative de la distribution florale dans une portion des Alpes et des Jura. Bulletin del la Societe Vaudoise des Sciences Naturelles, 37:547?579.
19. Ding Liu and Daniel Gildea. 2005. Syntactic Features for Evaluation of Machine Translation. In Proceedings of ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization, pages 25?32, Stroudsburg, PA, USA. Association for Computational Linguistics.
20. Matous Machacek and Ondrej Bojar. 2013. Results of the WMT13 metrics shared task. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 45?51, Sofia, Bulgaria, August. Association for Computational Linguistics.
21. Matous Machacek and Ondrej Bojar. 2014 (to appear). Results of the WMT14 metrics shared task. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimare, US, June. Association for Computational Linguistics.
22. Paul McNamee and James Mayfield. 2004. Character N-Gram Tokenization for European Language Text Retrieval. Information Retrieval, 7(1-2):73?97.
23. I. Dan Melamed, Ryan Green, and Joseph P. Turian. 2003. Precision and Recall of Machine Translation. In Proceedings of the Joint Conference on Human Language Technology and the North American Chapter of the Association for Computational Linguistics (HLT-NAACL), pages 61?63, Stroudsburg, PA, USA. Association for Computational Linguistics.
24. Sonja Nieben, Franz Josef Och, Gregor Leusch, and Hermann Ney. 2000. An Evaluation Tool for Machine Translation: Fast Evaluation for MT Research. In Proceedings of the 2nd International Conference on Language Resources and Evaluation (LREC).
25. Joakim Nivre, Johan Hall, Jens Nilsson, Atanas Chanev, Gulsen Eryigit, Sandra Kubler, Svetoslav Marinov, and Erwin Marsi. 2007. MaltParser: A language-independent system for data-driven dependency parsing. Natural Language Engineering, 13(2):95?135.
26. Slav Petrov and Dan Klein. 2007. Improved Inference for Unlexicalized Parsing. In Proc. Human Language Technologies (HLT), pages 404?411. Association for Computational Linguistics, April.
27. Slav Petrov, Leon Barrett, Romain Thibaux, and Dan Klein. 2006. Learning accurate, compact, and interpretable tree annotation. In 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, ACL-44, pages 433?440, Stroudsburg, PA, USA. Association for Computational Linguistics.
28. Bruno Pouliquen, Ralf Steinberger, and Camelia Ignat. 2003. Automatic Identification of Document Translations in Large Multilingual Document Collections. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP-2003), pages 401?408, Borovets, Bulgaria.
29. Michel Simard, George F. Foster, and Pierre Isabelle. 1992. Using Cognates to Align Sentences in Bilingual Corpora. In Proceedings of the Fourth International Conference on Theoretical and Methodological Issues in Machine Translation.
30. Matthew Snover, Nitin Madnani, Bonnie Dorr, and Richard Schwartz. 2009. Fluency, adequacy, or HTER? Exploring different human judgments with a tunable MT metric. In Proceedings of the 4th Workshop on Statistical Machine Translation, pages 259? 268, Stroudsburg, PA, USA. Association for Computational Linguistics.
31. Christoph Tillmann, Stefan Vogel, Hermann Ney, A. Zubiaga, and H. Sawaf. 1997. Accelerated DP based Search for Statistical Translation. In Proceedings of European Conference on Speech Communication and Technology, pages 2667?2670.