Performance evaluation of speech translation systems

Proceedings of the 6th International Conference on Language Resources and Evaluation, LREC 2008

Volumn , Issue , 2008, Pages 379-386

  Weiss, Brian A ^a   Schlenoff, Craig ^a   Sanders, Greg ^a   Steves, Michelle P ^a   Condon, Sherri ^b   Phillips, Jon ^b   Parvaz, Dan ^b  

^a NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY   (United States)

^b MITRE CORPORATION   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

HUMAN COMPUTER INTERACTION; PROGRAM TRANSLATORS; SPEECH RECOGNITION; TESTING;

ACADEMIC INSTITUTIONS; EVALUATION OF SPEECH TRANSLATION SYSTEMS; LOCAL POPULATIONS; MACHINE TRANSLATION SYSTEMS; NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY; SERVICE PERSONNEL; TRANSLATION SYSTEMS; VERBAL COMMUNICATIONS;

TRANSLATION (LANGUAGES);

EID: 84987911489     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (9)

1. Bannerjee, S. and A. Lavie (2005). METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proceedings of Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization, 43rd Annual Meeting of the Association of Computational Linguistics (ACL-2005), pp. 65-73.
2. Belvin, R., Riehemann, S., and K. Precoda. (2004). A Fine-Grained Evaluation Method for Speech-to-Speech Machine Translation Using Concept Annotations. In Proceedings of LREC 2004, pp. 1427-1430.
3. Condon, S., Phillips, J., Doran, C., Aberdeen, J., Parvaz, D., Oshika, B., Sanders G. and C. Schlenoff. (2008) Applying automated metrics to speech translation dialogs. In Proceedings of LREC 2008.
4. Information Access Division (2007). Tools - Evaluation Tools. http://www.nist.gov/speech/tools/.
5. Levin, L. Gates, D., Lavie, A., Pianesi, F., Wallace, D., Watanabe, T., and M. Woszczyna. (2000). Evaluation of a practical interlingua for task-oriented dialogue. In Proceedings of the NAACL-ANLP 2000 Workshop on Applied interlinguas: practical applications of interlingual approaches to NLP, Volume 2, pp. 18-23.
6. Papineni, K., Roukos, S., Ward, T., and W-J. Zhu. (2002). Bleu: A method for automatic evaluation of machine translation. In Proceedings of ACL 2002, pp. 311-318.
7. Sanders, G., Bronsart, S., Condon, S., and C. Schlenoff. (2008). Odds of successful transfer of low-level concepts: A key metric for bidirectional speech-to-speech machine translation in DARPA's TRANSTAC program. In Proceedings of LREC 2008.
8. Schlenoff, C., Steves, M., Weiss, B., Shneier, M., and A. Virts. (2007). Applying SCORE to field-based performance evaluations of soldier worn sensor technolo gies. Journal of Field Robotics, Volume 24, (8-9), pp. 671-698.
9. Snover, M., Dorr, B., Schwartz, R., Makhoul, J., Micciulla, L., Weischedel, R. (2005). A Study of Translation Error Rate with Targeted Human Annotation. LAMP-TR-126, CS-TR-4755, UMIACS-TR-2008-58, University of M aryland, Colle ge Park.