Using discourse structure improves machine translation evaluation

52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014 - Proceedings of the Conference

Volumn 1, Issue , 2014, Pages 687-698

  Guzmán, Francisco ^a   Joty, Shafiq ^a   Màrquez, Lluís ^a   Nakov, Preslav ^a  

^a QATAR COMPUTING RESEARCH INSTITUTE   (Qatar)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER AIDED LANGUAGE TRANSLATION; TEXT PROCESSING;

DISCOURSE STRUCTURE; EVALUATION METRICS; FIRST DESIGNS; MACHINE TRANSLATION EVALUATIONS; RHETORICAL STRUCTURE THEORY; SIMILARITY MEASURE; STATE OF THE ART; SYSTEM LEVELS;

COMPUTATIONAL LINGUISTICS;

EID: 84906925055     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/v1/p14-1065     Document Type: Conference Paper

References (43)

1. Nicholas Asher and Alex Lascarides, 2003. Logics of Conversation. Cambridge University Press.
2. Chris Callison-Burch, Philipp Koehn, Christof Monz, and Omar Zaidan. 2011. Findings of the 2011 workshop on statistical machine translation. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 22-64, Edinburgh, Scotland, July. ACL.
3. Chris Callison-Burch, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Specia. 2012. Findings of the 2012 workshop on statistical machine translation. In Proceedings of the Seventh Workshop on Statistical Machine Translation, pages 10-51, Montréal, Canada, June. ACL.
4. Lynn Carlson and Daniel Marcu. 2001. Discourse Tagging Reference Manual. Technical Report ISI-TR- 545, University of Southern California Information Sciences Institute.
5. Bruno Cartoni, Sandrine Zufferey, Thomas Meyer, and Andrei Popescu-Belis. 2011. How comparable are parallel corpora? measuring the distribution of general vocabulary and connectives. In Proceedings of the 4th Workshop on Building and Using Comparable Corpora: Comparable Corpora and the Web, pages 78-86, Portland, Oregon, June. ACL.
6. David Chiang, Yuval Marton, and Philip Resnik. 2008. Online large-margin training of syntactic and structural translation features. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing (EMNLP'08), Honolulu, Hawaii, USA.
7. David Chiang. 2005. A hierarchical phrase-based model for statistical machine translation. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, ACL '05, pages 263-270, Ann Arbor, Michigan.
8. Michael Collins and Nigel Duffy. 2001. Convolution Kernels for Natural Language. In Neural Information Processing Systems, NIPS'01, pages 625-632, Vancouver, Canada.
9. Elisabet Comelles, Jesús Giménez, Lluís Màrquez, Irene Castellón, and Victoria Arranz. 2010. Document-level automatic mt evaluation based on discourse representations. In Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR, pages 333-338, Uppsala, Sweden, July. ACL.
10. Michael Denkowski and Alon Lavie. 2011. Meteor 1.3: Automatic metric for reliable optimization and evaluation of machine translation systems. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 85-91, Edinburgh, Scotland, July. ACL.
11. George Doddington. 2002. Automatic evaluation of machine translation quality using n-gram cooccurrence statistics. In Proceedings of the Second International Conference on Human Language Technology Research, HLT '02, pages 138-145, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.
12. Michel Galley, Mark Hopkins, Kevin Knight, and Daniel Marcu. 2004. What's in a translation rule? In Proceedings of the 2004 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technology, HLT-NAACL, pages 273-280.
13. Jesús Giménez and Lluís Màrquez. 2007. Linguistic features for automatic evaluation of heterogenous MT systems. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 256-264, Prague, Czech Republic, June. ACL.
14. Jesús Giménez and Lluís Màrquez. 2009. On the robustness of syntactic and semantic features for automatic MT evaluation. In Proceedings of the Fourth Workshop on Statistical Machine Translation, pages 250-258, Athens, Greece, March. ACL.
15. Jesús Giménez and Lluís Màrquez. 2010a. Asiya: an Open Toolkit for Automatic Machine Translation (Meta-)Evaluation. The Prague Bulletin of Mathematical Linguistics, 94:77-86.
16. Jesús Giménez and Lluís Màrquez. 2010b. Linguistic Measures for Automatic Machine Translation Evaluation. Machine Translation, 24(3-4):77-86.
17. Michael Halliday and Ruqaiya Hasan, 1976. Cohesion in English. Longman, London.
18. Christian Hardmeier and Marcello Federico. 2010. Modelling pronominal anaphora in statistical machine translation. In Proceedings of the International Workshop on Spoken Language Translation, pages 283-289.
19. Christian Hardmeier, Joakim Nivre, and Jörg Tiedemann. 2012. Document-wide decoding for phrasebased statistical machine translation. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLPCoNLL '12, pages 1179-1190, Jeju Island, Korea. ACL.
20. Christian Hardmeier. 2012. Discourse in statistical machine translation. A survey and a case study. Discours. Revue de linguistique, psycholinguistique et informatique, 11(8726).
21. Mark Hopkins and Jonathan May. 2011. Tuning as ranking. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, EMNLP '11.
22. Shafiq Joty, Giuseppe Carenini, and Raymond T. Ng. 2012. A Novel Discriminative Framework for Sentence-Level Discourse Analysis. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL '12, pages 904-915, Jeju Island, Korea. ACL.
23. Shafiq Joty, Giuseppe Carenini, Raymond T. Ng, and Yashar Mehdad. 2013. Combining Intra- and Multi-sentential Rhetorical Parsing for Documentlevel Discourse Analysis. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, ACL '13, pages 486-496, Sofia, Bulgaria. ACL.
24. Hans Kamp and Uwe Reyle. 1993. From Discourse to Logic: Introduction to Model theoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory. Number 42 in Studies in Linguistics and Philosophy. Kluwer Academic Publishers.
25. Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Proceedings of Workshop on Text Summarization Branches Out, pages 74-81, Barcelona.
26. Ding Liu and Daniel Gildea. 2005. Syntactic features for evaluation of machine translation. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 25-32, Ann Arbor, Michigan, June. ACL.
27. Chi-kiu Lo, Anand Karthik Tumuluru, and Dekai Wu. 2012. Fully automatic semantic mt evaluation. In Proceedings of the Seventh Workshop on Statistical Machine Translation, pages 243-252, Montréal, Canada, June. ACL.
28. William Mann and Sandra Thompson. 1988. Rhetorical Structure Theory: Toward a Functional Theory of Text Organization. Text, 8(3):243-281.
29. Thomas Meyer, Andrei Popescu-Belis, Najeh Hajlaoui, and Andrea Gesmundo. 2012. Machine translation of labeled discourse connectives. In Proceedings of the Tenth Biennial Conference of the Association for Machine Translation in the Americas (AMTA).
30. Thomas Meyer. 2011. Disambiguating temporalcontrastive connectives for machine translation. In Proceedings of the ACL 2011 Student Session, pages 46-51, Portland, OR, USA, June. ACL.
31. Alessandro Moschitti and Roberto Basili. 2006. A Tree Kernel approach to Question and Answer Classification in Question Answering Systems. In Proceedings of the 5th international conference on Language Resources and Evaluation, Genoa, Italy.
32. Alessandro Moschitti, Silvia Quarteroni, Roberto Basili, and Suresh Manandhar. 2007. Exploiting Syntactic and Shallow Semantic Kernels for Question/ Answer Classification. In Proceedings of the ACL-2007, pages 776-783, Prague, Czech Republic.
33. Kishore Papineni, Salim Roukos, ToddWard, andWei- Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the Association for Computational Linguistics (ACL'02), Philadelphia, PA, USA.
34. Maja Popovic and Hermann Ney. 2007. Word error rates: Decomposition over POS classes and applications for error analysis. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 48-55, Prague, Czech Republic, June. ACL.
35. Chris Quirk, Arul Menezes, and Colin Cherry. 2005. Dependency treelet translation: Syntactically informed phrasal smt. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, ACL '05, pages 271-279, Ann Arbor, Michigan.
36. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of the 7th Biennial Conference of the Association for Machine Translation in the Americas, AMTA '06, Cambridge, MA, USA.
37. Charles Sutton, Andrew McCallum, and Khashayar Rohanimanesh. 2007. Dynamic Conditional Random Fields: Factorized Probabilistic Models for Labeling and Segmenting Sequence Data. Journal of Machine Learning Research (JMLR), 8:693-723.
38. Kuo-Chung Tai. 1979. The tree-to-tree correction problem. Journal of the ACM, 26(3):422-433, July.
39. Taro Watanabe, Jun Suzuki, Hajime Tsukada, and Hideki Isozaki. 2007. Online large-margin training for statistical machine translation. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL '07, Prague, Czech Republic.
40. Bonnie Webber, Andrei Popescu-Belis, Katja Markert, and Jörg Tiedemann, editors. 2013. Proceedings of the Workshop on Discourse in Machine Translation. ACL, Sofia, Bulgaria, August.
41. Bonnie Webber. 2004. D-LTAG: Extending Lexicalized TAG to Discourse. Cognitive Science, 28(5):751-779.
42. Billy T. M. Wong and Chunyu Kit. 2012. Extending machine translation evaluation metrics with lexical cohesion to document level. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLPCoNLL, pages 1060-1068, Jeju Island, Korea, July. ACL.
43. Dekai Wu and Pascale Fung. 2009. Semantic roles for smt: A hybrid two-pass model. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Short Papers, pages 13-16, Boulder, Colorado, June.