Accurate transcription of broadcast news speech using multiple noisy transcribers and unsupervised reliability metrics

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2011, Pages 4980-4983

  Audhkhasi, Kartik ^a   Georgiou, Panayiotis ^a   Narayanan, Shrikanth S ^a  

^a University of Southern California   (United States)

Author keywords

crowd sourcing; evaluator reliability; Speech transcription

Indexed keywords

AUDIO CLIPS; BROADCAST NEWS; CROWD SOURCING; EVALUATOR RELIABILITY; GOLD STANDARDS; MECHANICAL TURKS; QUALITY OF WORK; RELIABILITY INFORMATION; RELIABILITY METRICS; SENTENCE ERRORS; SIMPLE METRICS; SPEECH TRANSCRIPTION; TIME-CONSUMING PROCESS; WORD ERROR RATE;

GOLD; SIGNAL PROCESSING; SPEECH COMMUNICATION; SPEECH TRANSMISSION; STANDARDS; TRANSCRIPTION;

RELIABILITY;

EID: 80051628698     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2011.5947474     Document Type: Conference Paper

References (12)

1. R. Snow, B. O' Connor, D. Jurafsky, and A. Y. Ng, "Cheap and fast - but is it good? Evaluating non-expert annotations for natural language tasks," in Proceedings of EMNLP. ACM, 2008, vol. 1, pp. 254-263.
2. C. Callison-Burch, "Fast, cheap and creative: Evaluating translation quality using Amazon's Mechanical Turk," in Proceedings of EMNLP. ACM, 2009, vol. 1, pp. 286-295.
3. B. Lambert, R. Singh, and B. Raj, "Creating linguistic plausibility dataset with non-expert annotators," in Proceedings of InterSpeech. ISCA, 2010, pp. 1906-1909.
4. M. Denkowski, H. AI-Haj, and A. Lavie, "Turker-assisted paraphrasing for English-Arabic machine translation," in Proceedings of HLT. NAACL, 2010, pp. 66-70.
5. V. Ambati and S. Vogel, "Can crowds build parallel corpora for machine translation systems?," in Proceedings of HLT. NAACL, 2010, pp. 62-65.
6. M. Marge, Banerjee S., and A. I. Rudnicky, "Using the Amazon Mechanical Turk for transcription of spoken language," in Proceedings of ICASSP. IEEE, 2010.
7. J. Fiscus, "A post processing system to yield reduced word error rates: Recognizer Output Voting Error Reduction (ROVER)," in Proceedings of ASRU. IEEE, 1997, pp. 347-354.
8. B. C. Roy, S. Vasoughi, and D. Roy, "Automatic estimation of transcription accuracy and difficulty," in Proceedings of Inter-Speech. ISCA, 2010, pp. 1902-1905.
9. HUB4-NE, "1997 Spanish Broadcast News Speech," Linguistic Data Consortium, 1998.
10. Carnegie Mellon University, "Sphinx-3," Pittsburgh, Pennysylvania.
11. A. Stolcke, "SRILM - an extensible language modeling toolkit," in Proceedings of ICSLP. ISCA, 2002, pp. 901-904.
12. SCTK Version 1.2c, "NIST Speech Recognition Scoring Toolkit," NIST, 2008.