Goodness: A method for measuring machine translation confidence

ACL-HLT 2011 - Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies

Volumn 1, Issue , 2011, Pages 211-219

  Bach, Nguyen ^a   Huang, Fei ^b   Al Onaizan, Yaser ^b  

^a Carnegie Mellon University   (United States)

^b IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONFIDENCE MEASURE; ERROR PREDICTION; F-SCORE; MEASURING MACHINES; N-BEST LIST; PEARSON CORRELATION; RE-RANKING; SENTENCE LEVEL; STATISTICAL MACHINE TRANSLATION;

COMPUTATIONAL LINGUISTICS; CORRELATION METHODS; ERRORS; INFORMATION THEORY; SPEECH TRANSMISSION; VISUALIZATION;

TRANSLATION (LANGUAGES);

EID: 84859097957     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. Nguyen Bach, Matthias Eck, Paisarn Charoenpornsawat, Thilo Khler, Sebastian Stker, ThuyLinh Nguyen, Roger Hsiao, Alex Waibel, Stephan Vogel, Tanja Schultz, and Alan Black. 2007. The CMU TransTac 2007 Eyes-free and Hands-free Two-way Speech-to-Speech Translation System. In Proceedings of the IWSLT'07, Trento, Italy.
2. Nguyen Bach, Qin Gao, and Stephan Vogel. 2009. Sourceside dependency tree reordering models with subtree movements and constraints. In Proceedings of the MTSummit-XII, Ottawa, Canada, August. International Association for Machine Translation.
3. John Blatz, Erin Fitzgerald, George Foster, Simona Gandrabur, Cyril Goutte, Alex Kulesza, Alberto Sanchis, and Nicola Ueffing. 2004. Confidence estimation for machine translation. In The JHU Workshop Final Report, Baltimore, Maryland, USA, April.
4. David Chiang, Kevin Knight, and Wei Wang. 2009. 11,001 new features for statistical machine translation. In Proceedings of HLT-ACL, pages 218-226, Boulder, Colorado, June. Association for Computational Linguistics.
5. Koby Crammer and Yoram Singer. 2003. Ultraconservative online algorithms for multiclass problems. Journal of Machine Learning Research, 3:951-991.
6. Niyu Ge. 2004. Max-posterior HMM alignment for machine translation. In Presentation given at DARPA/TIDES NIST MT Evaluation workshop.
7. Nizar Habash and Jun Hu. 2009. Improving arabic-chinese statistical machine translation using english as pivot language. In Proceedings of the 4th Workshop on Statistical Machine Translation, pages 173-181, Morristown, NJ, USA. Association for Computational Linguistics.
8. Almut Silja Hildebrand and Stephan Vogel. 2008. Combination of machine translation systems via hypothesis selection from combined n-best lists. In Proceedings of the 8th Conference of the AMTA, pages 254-261, Waikiki, Hawaii, October.
9. Fei Huang. 2009. Confidence measure for word alignment. In Proceedings of the ACL-IJCNLP '09, pages 932-940, Morristown, NJ, USA. Association for Computational Linguistics.
10. Abraham Ittycheriah and Salim Roukos. 2005. A maximum entropy word aligner for arabic-english machine translation. In Proceedings of the HTL-EMNLP'05, pages 89-96, Morristown, NJ, USA. Association for Computational Linguistics.
11. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of ACL'07, pages 177-180, Prague, Czech Republic, June.
12. Yanjun Ma, Sylwia Ozdowska, Yanli Sun, and Andy Way. 2008. Improving word alignment using syntactic dependencies. In Proceedings of the ACL-08: HLT SSST-2, pages 69-77, Columbus, OH.
13. Marie-Catherine Marneffe, Bill MacCartney, and Christopher Manning. 2006. Generating typed dependency parses from phrase structure parses. In Proceedings of LREC'06, Genoa, Italy.
14. Ryan McDonald, Koby Crammer, and Fernando Pereira. 2005. Flexible text segmentation with structured multilabel classification. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 987-994, Vancouver, British Columbia, Canada, October. Association for Computational Linguistics.
15. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of ACL'02, pages 311-318, Philadelphia, PA, July.
16. Chris Quirk. 2004. Training a sentence-level machine translation confidence measure. In Proceedings of the 4th LREC.
17. Sylvain Raybaud, Caroline Lavecchia, David Langlois, and Kamel Smaili. 2009. Error detection for statistical machine translation using linguistic features. In Proceedings of the 13th EAMT, Barcelona, Spain, May.
18. Binyamin Rozenfeld, Ronen Feldman, and Moshe Fresko. 2006. A systematic cross-comparison of sequence classifiers. In Proceedings of the SDM, pages 563-567, Bethesda, MD, USA, April.
19. Alberto Sanchis, Alfons Juan, and Enrique Vidal. 2007. Estimation of confidence measures for machine translation. In Proceedings of the MT Summit XI, Copenhagen, Denmark.
20. Libin Shen, Jinxi Xu, and Ralph Weischedel. 2008. A new string-to-dependency machine translation algorithm with a target dependency language model. In Proceedings of ACL-08: HLT, pages 577-585, Columbus, Ohio, June. Association for Computational Linguistics.
21. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of AMTA'06, pages 223-231, August.
22. Radu Soricut and Abdessamad Echihabi. 2010. Trustrank: Inducing trust in automatic translations via ranking. In Proceedings of the 48th ACL, pages 612-621, Uppsala, Sweden, July. Association for Computational Linguistics.
23. Lucia Specia, Zhuoran Wang, Marco Turchi, John Shawe- Taylor, and Craig Saunders. 2009. Improving the confidence of machine translation quality estimates. In Proceedings of the MT Summit XII, Ottawa, Canada.
24. Christoph Tillmann. 2006. Efficient dynamic programming search algorithms for phrase-based SMT. In Proceedings of the Workshop on Computationally Hard Problems and Joint Inference in Speech and Language Processing, pages 9-16, Morristown, NJ, USA. Association for Computational Linguistics.
25. Nicola Ueffing and Hermann Ney. 2007. Word-level confidence estimation for machine translation. Computational Linguistics, 33(1):9-40.
26. Taro Watanabe, Jun Suzuki, Hajime Tsukada, and Hideki Isozaki. 2007. Online large-margin training for statistical machine translation. In Proceedings of the EMNLPCoNLL, pages 764-773, Prague, Czech Republic, June. Association for Computational Linguistics.
27. Deyi Xiong, Min Zhang, and Haizhou Li. 2010. Error detection for statistical machine translation using linguistic features. In Proceedings of the 48th ACL, pages 604-611, Uppsala, Sweden, July. Association for Computational Linguistics.