Odds of successful transfer of low-level concepts: A key metric for bidirectional speech-to-speech machine translation in DARPA's TRANSTAC program

Proceedings of the 6th International Conference on Language Resources and Evaluation, LREC 2008

Volumn , Issue , 2008, Pages 1565-1571

  Sanders, Gregory A ^a   Bronsart, Sébastien ^a   Condon, Sherri ^b   Schlenoff, Craig ^a  

^a NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY   (United States)

^b MITRE CORPORATION   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; COMPUTER AIDED LANGUAGE TRANSLATION; MARINE COMMUNICATION; SEMANTICS; SPEECH COMMUNICATION; SPEECH TRANSMISSION; TRANSLATION (LANGUAGES);

ADVANCED RESEARCHES; AUTOMATED MT METRICS; CIVILIAN POPULATIONS; MACHINE TRANSLATIONS; NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY; SOURCE LANGUAGE; SPOKEN LANGUAGES; TRANSLATION SYSTEMS;

PROGRAM TRANSLATORS;

EID: 77954761840     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (16)

1. Banerjee, S., and A. Lavie (2005). METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proceedings of the Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization at the 43rd Annual Meeting of the Association for Computational Linguistics (ACL). Ann Arbor, MI.
2. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, (10), pp. 37-46.
3. Condon, S., J. Phillips, C. Doran, J. Aberdeen, D. Parvaz, B. Oshika, G. Sanders, and C. Schlenoff (2008). Applying Automated Metrics to Speech Translation Dialogs. In Proceedings of LREC-2008.
4. Frederking, R. and S. Nirenburg (1994). Three Heads are Better Than One. In Proceedings of the Fourth Conference on Applied Natural Language Processing (ANLP), pp. 95-100.
5. Frederking, R., A. W. Black, R. D. Brown, J. Moody, and E. Steinbrecher (2002). Field Testing the Tongues Speech-to-Speech Machine Translation System. In Proceedings of LREC-2002 (Las Palmas, Gran Canaria, Canary Islands, Spain), pp. 160-164.
6. Hogan, C., and R. E. Frederking (1998). An Evaluation of the Multi-Engine MT Architecture. In Proceedings of AMTA-1998, pp. 113-124.
7. King, M. (1996). Evaluating Natural Language Processing Systems. In Communications of the ACM (39) 1, pp. 73-79.
8. Knight, K. and I. Chander (1994). Automated Postediting of Documents. In Proceedings of the National Conference on Artificial Intelligence (AAAI).
9. Lavie, A., K. Sagae, and S. Jayaraman (2004). The Significance of Recall in Automatic Metrics for MT Evaluation. In Proceedings of the 6th Conference of the Association for Machine Translation in the Americas (AMTA). Washington, DC.
10. Lita, L. V., M. Rogati, and A. Lavie (2005). BLANC: Learning Evaluation Metrics for MT. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP). (Vancouver, Canada, October 2005). pp. 740-747.
11. Melamed, I. D., R. Green, and J. P. Turian (2003). Precision and Recall of Machine Translation. Proceedings of HLT/NAACL 2003 (Edmonton, Alberta, Canada), pp. 61-63.
12. Papineni, K., S. Roukos, T. Ward, and W. Zhu (2001). BLEU: A Method for Automatic Evaluation of Machine Translation. (IBM research report available at http://domino. watson. ibm.com/library/cybergig.nsf/).
13. Polzar, M., and E. Charniak (2006). Bllip: An Improved Evaluation Metric for Machine Translation. Brown University, Providence, Rhode Island. (Available as: http://www.cs.brown. edu/research/pubs/theses/masters/2006/mpozar.pdf)
14. Russo-Lassner, G., J. Lin, and P. Resnik (2005). A Paraphrase-based Approach to Machine Translation Evaluation. LAMP-TR-125 CS-TR-4754 UMIACS-TR-2005-57, University of Maryland, College Park, MD, August, 2005.
15. Snover M., B. Dorr, R. Schwartz, J. Makhoul, L. Micciulla, and R. Weischedel (2005). A Study of Translation Error Rate with Targeted Human Annotation. LAMP-TR-126, CS-TR-4755, UMIACS-TR-2005-58, University of Maryland, College Park, MD, July, 2005
16. Weiss, B., C. Schlenoff, G. Sanders, M. P. Steves, S. Condon, J. Phillips, and D. Parvaz (2008). Performance Evaluation of Speech Translation Systems. In Proceedings of LREC-2008.