Quality estimation for automatic speech recognition

COLING 2014 - 25th International Conference on Computational Linguistics, Proceedings of COLING 2014: Technical Papers

Volumn , Issue , 2014, Pages 1813-1823

  Negri, Matteo ^a   Turchi, Marco ^a   De Souza, Jos̈e G C ^a,b   Falavigna, Daniele ^a  

^a FONDAZIONE BRUNO KESSLER   (Italy)

^b UNIVERSITY OF TRENTO   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; LINGUISTICS; TRANSCRIPTION;

AUTOMATIC SPEECH RECOGNITION; AUTOMATIC TRANSCRIPTION; FEATURE SETS; QUALITY ESTIMATION; SOURCE SIGNALS; TESTING CONDITIONS; WORD ERROR RATE;

SPEECH RECOGNITION;

EID: 84943792040     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (42)

1. Nguyen Bach, Fei Huang, and Yaser Al-Onaizan. 2011. Goodness: A Method for Measuring Machine Translation Confidence. In The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference, 19-24 June, 2011, Portland, Oregon, USA, pages 211-219. The Association for Computer Linguistics.
2. James Bergstra and Yoshua Bengio. 2012. Random Search for Hyper-Parameter Optimization. Journal of Machine Learning Research, 13:281-305.
3. John Blatz, Erin Fitzgerald, George Foster, Simona Gandrabur, Cyril Goutte, Alex Kulesza, Alberto Sanchis, and Nicola Ueffing. 2003. Confidence Estimation for Machine Translation. Summer workshop final report, JHU/CLSP.
4. Paul Boersma and David Weenink. 2005. Praat: Doing Phonetics by Computer (Version 4.3.01). Retrieved from http://www.praat.org/.
5. Ondrej Bojar, Christian Buck, Chris Callison-Burch, Christian Federmann, Barry Haddow, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Specia. 2013. Findings of the 2013Workshop on Statistical Machine Translation. In Eighth Workshop on Statistical Machine Translation, WMT-2013, pages 1-44, Sofia, Bulgaria.
6. Chris Callison-Burch, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Specia. 2012. Findings of the 2012Workshop on Statistical Machine Translation. In Proceedings of the SeventhWorkshop on Statistical Machine Translation (WMT'12), pages 10-51, Montŕeal, Canada.
7. Mauro Cettolo, Jan Niehues, Sebastian Stüker, Luisa Bentivogli, and Marcello Federico. 2013. Report on the 10th IWSLT Evaluation Campaign. Proceedings of the International Workshop on Spoken Language Translation, Heidelberg, Germany.
8. José G. C. de Souza, Christian Buck, Marco Turchi, and Matteo Negri. 2013. FBK-UEdin participation to the WMT13 quality estimation shared task. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 352-358, Sofia, Bulgaria, August. Association for Computational Linguistics.
9. José G. C. de Souza, Jes̈us González-Rubio, Christian Buck, Marco Turchi, and Matteo Negri. 2014a. FBK-UPVUEdin participation in the WMT14 Quality Estimation shared-task. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, MD, USA, June.
10. José G. C. de Souza, Marco Turchi, and Matteo Negri. 2014b. Predicting Machine Translation Quality Estimation Across Domains. In Proceedings of the 25th International Conference on Computational Linguistics, COLING'14, Dublin, Ireland.
11. Gunnar Evermann and Philip C. Woodland. 2000. Large Vocabulary Decoding and Confidence Estimation Using Word Posterior Probabilities. In Proc. of ICASSP, pages 2366-2369, Istanbul, Turkey, June.
12. Daniele Falavigna, Roberto Gretter, Fabio Brugnara, Diego Giuliani, and Romain Serizel. 2013. FBK@IWSLT 2013 - ASR Tracks. In Proceedings of the IWSLT 2013 workshop, Heidelberg, Germany.
13. Marcello Federico, Nicola Bertoldi, and Mauro Cettolo. 2008. IRSTLM: An Open Source Toolkit for Handling Large Scale Language Models. pages 1618-1621, Brisbane, Australia, September.
14. Marcello Federico, Luisa Bentivogli, Michael Paul, and Sebastian Stuker. 2011. Overview of the IWSLT 2011 Evaluation Campaign. In International Workshop on Spoken Language Translation, pages 11-27.
15. Marcello Federico, Mauro Cettolo, Luisa Bentivogli, Michael Paul, and Sebastian Stüker. 2012. Overview of the IWSLT 2012 Evaluation Campaign. In Proc. of the International Workshop on Spoken Language Translation, Hong Kong, HK, December.
16. Mariano Felice. 2012. Linguistic Indicators for Quality Estimation of Machine Translations. Master's thesis, University of Wolverhampton, UK.
17. Malte Gabsdil and Oliver Lemon. 2004. Combining acoustic and pragmatic features to predict recognition performance in spoken dialogue systems. pages 344-351.
18. Pierre Geurts, Damien Ernst, and LouisWehenkel. 2006. Extremely randomized trees. Mach. Learn., 63(1):3-42, April.
19. Sharon Goldwater, Dan Jurafsky, and Christopher Manning. 2010. Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase speech recognition error rates. 52(3):181-200.
20. Trevor Hastie, Robert Tibshirani, Jerome Friedman, T Hastie, J Friedman, and R Tibshirani. 2009. The elements of statistical learning, volume 2. Springer.
21. Derrick Higgins, Xiaoming Xi, Klaus Zechner, and David Williamson. 2011. A three-stage approach for automated scoring of spontaneous responses. (25):282-306.
22. Diane J. Litman, Julia B. Hirschberg, and Marc Swerts. 2000. Predicting Automatic Speech Recognition Performance Using Prosodic Cues. In Proceedings of NAACL, pages 218-225.
23. Annie Louis and Ani Nenkova. 2013. Automatically assessing machine summary content without a gold standard. Computational Linguistics, 39(2):267-300.
24. Yashar Mehdad, Matteo Negri, and Marcello Federico. 2012. Match without a Referee: Evaluating MT Adequacy without Reference Translations. In Proceedings of the Machine Translation Workshop (WMT2012).
25. Nicolai Meinshausen and Peter Bühlmann. 2010. Stability selection. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 72(4):417-473.
26. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikitlearn : Machine Learning in Python. Journal of Machine Learning Research, 12:2825-2830.
27. Christopher B. Quirk. 2004. Training a Sentence-Level Machine Translation Confidence Measure. In Proceedings of LREC 2004.
28. Raphael Rubino, Jos̈e GC de Souza, Jennifer Foster, and Lucia Specia. 2013a. Topic Models for Translation Quality Estimation for Gisting Purposes. In Proceedings of the Machine Translation Summit XIV.
29. Raphael Rubino, Joachim Wagner, Jennifer Foster, Johann Roturier, Rasoul Samad Zadeh Kaljahi, and Fred Hollowood. 2013b. DCU-Symantec at the WMT 2013 Quality Estimation Shared Task. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 392-397.
30. Alberto Sanchis, Alfons Juan, and Enrique Vidal. 2012. A Word-Based Naive Bayes Classifier for Confidence Estimation in Speech Recognition. 20(12):565-574.
31. Helmut Schmid. 1995. Improvements in Part-of-Speech Tagging with an Application to German. In Proceedings of the ACL SIGDAT-Workshop, pages 47-50, Dublin, Ireland.
32. Matthew Stephen Seigel. 2013. Condence Estimation for Automatic Speech Recognition Hypotheses. University of Cambridge. PhD Thesis.
33. John Shawe-Taylor and Nello Cristianini. 2004. Kernel methods for pattern analysis. Cambridge university press.
34. Radu Soricut and Abdessamad Echihabi. 2010. TrustRank: Inducing Trust in Automatic Translations via Ranking. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, ACL '10, pages 612-621, Stroudsburg, PA, USA. Association for Computational Linguistics.
35. Lucia Specia, Nicola Cancedda, Marc Dymetman, Marco Turchi, and Nello Cristianini. 2009. Estimating the Sentence-Level Quality of Machine Translation Systems. In Proceedings of the 13th Annual Conference of the European Association for Machine Translation (EAMT'09), pages 28-35, Barcelona, Spain.
36. Lucia Specia, Dhwaj Raj, and Marco Turchi. 2010. Machine Translation Evaluation versus Quality Estimation. Machine translation, 24(1):39-50.
37. Lucia Specia. 2011. Exploiting Objective Annotations for Measuring Translation Post-editing Effort. Proceedings of the 15th Conference of the European Association for Machine Translation, pages 73-80.
38. Rafic Antoon Sukkar and Chin-Hui Lee. 1996. Vocabulary Independent Discriminative Utterance Verification for Nonkeyword Rejection in Subword Based Speech Recognition. 6(6):420-429.
39. Marco Turchi, Antonios Anastasopoulos, Jos̈e G. C. de Souza, and Matteo Negri. 2014. Adaptive Quality Estimation for Machine Translation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, ACL'14, Baltimore, MD, USA. Association for Computational Linguistics.
40. Nicola Ueffing and Hermann Ney. 2007. Word-Level Confidence Estimation for Machine Translation. Comput. Linguist., 33(1):9-40, March.
41. Frank Wessel, Ralf Schlüter, Klaus Macherey, and Hermann Ney. 2001. Confidence Measures for Large Vocabulary Continuous Speech Recognition. 9(3):288-298.
42. Su-Youn Yoon, Lei Chen, and Klaus Zechner. 2010. Predicting word accuracy for the automatic speech recognition of non-native speech. In Proc. of INTERSPEECH, pages 773-776, Makuhari, Chiba, Japan. 1823.