Speaking to the Crowd: Looking at past achievements in using crowdsourcing for speech and predicting future challenges

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn , Issue , 2011, Pages 3037-3040

  Parent, Gabriel ^a   Eskenazi, Maxine ^a  

^a Carnegie Mellon University ^*  (United States)

Author keywords

Crowdsourcing; Human computation; Speech acquisition; Transcription

Indexed keywords

CROWDSOURCING; FUTURE CHALLENGES; HUMAN COMPUTATION; SPEECH ACQUISITION; SPEECH TECHNOLOGY; TECHNIQUES USED;

SPEECH PROCESSING;

TRANSCRIPTION;

EID: 84865734254     PISSN: None     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. Surowiecki, J. "The Wisdom of Crowds", Doubleday, 2004
2. Ledie, J., Odero, B., Minkov, E., Kiss, I., Polifroni, J., "Crowd translator: on building localized speech recognizers through micropayments", in ACM SIGOPS Operating Systems Review, vol 43, 4, New York, NY, 2010.
3. Lane, I., Waibel, A., Eck, M. and Rottmann, K. "Tools for Collecting Speech Corpora via Mechanical-Turk", Proc. NAACL Workshop: Creating Speech and Language Data with Amazon's Mechanical Turk, Los Angeles, 2010.
4. Polifroni, J. Kiss, I., Seneff, S., "Speech for Content Creation," Proc. SiMPE, Lisbon, Portugal, 2010.
5. McGraw, I., Lee, C., Hetherington, L., Seneff, S., Glass, J., "Collecting Voices from the Cloud," Proceedings of LREC, Malta, 2010.
6. McGraw, I., Gruenstein, A., Sutherland, A., "A Self-Labeling Speech Corpus: Collecting Spoken Words with an Online Educational Game," Proceedings Interspeech, Brighton, UK, 2009.
7. Gruenstein, A., McGraw, I., Sutherland, A., "A Self-Transcribing Speech Corpus: Collecting Continuous Speech with an Online Educational Game", Proceedings of SLaTE 2009, Brighton, UK, 2009.
8. Chernova, S., Orkin, J., Breazeal, C., "Crowdsourcing HRI through online multiplayer games", Proc. Dialog with Robots: AAAI fall symposium, 2010.
9. Freitas, J., Calado, A., Braga, D., Silva, P., Dias, M., "Crowdsourcing platform for large-scale speech data collection", Proc. FALA, Vigo, 2010.
10. Suendermann, D., Liscombe, J., Pieraccini, R., "How to Drink from a Fire Hose: One Person Can Annoscribe 693 Thousand Utterances in One Month". Proc. SIGDIAL 2010, Tokyo, Japan, 2010.
11. Parent, G. & Eskenazi, M., "Toward better crowdsourced transcription: transcription of a year of the Let's Go Bus information System data". Proc. IEEE SLT, Berkeley, 2010.
12. Marge, M., Banerjee, S., Rudnicky, A., "Using the Amazon Mechanical Turk to Transcribe and Annotate Meeting Speech for Extractive Summarization". Proc. NAACL Workshop: Creating Speech and Language Data with Amazon's Mechanical Turk, Los Angeles, 2010.
13. Marge, M., Banerjee, S., Rudnicky, A., "Using the Amazon Mechanical Turk for Transcription of Spoken Language". Proc. ICASSP, Dallas, 2010.
14. Akasaka, R., "Foreign Accented Speech Transcription and Accent Recognition Using a Game-based Approach", Masters Thesis, Swarthmore Department of Linguistics, 2009.
15. Novotney, S., Callison-Burch, C., "Cheap, Fast and Good Enough: Automatic Speech Recognition with Non-Expert Transcription". Proc. NAACL, 2010.
16. Audhkhasi, K., Georgiou, P., Narayanan, S. "Accurate Transcription of Broadcast News Speech Using Multiple Noisy Transcribers and Unsupervised Reliability Metrics", Proc. ICASSP, Prague, 2011.
17. Evanini, K., Higgins, D., Zechner, K., "Using Amazon Mechanical Turk for transcription of non-native speech", Proc. NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, 2010.
18. Schlaikjer, A., "A dual-use speech CAPTCHA: Aiding visually impaired web users while providing transcriptions of Audio Streams", LTI-CMU Technical Report 07-014, 2007.
19. Kunath, S.A., and Weinberger, S.H., "The wisdom of the crowd's ear: speech accent rating and annotation with Amazon Mechanical Turk", Proc. NAACL HLT Workshop: Creating Speech and Language Data with Amazon's Mechanical Turk, Los Angeles, 2010.
20. Yang, Z., Li, B., Zhu,Y., King, I., Levow, G. and Meng, H., "Collection of User Judgments on Spoken Dialog System with Crowdsourcing", Proc. SLT 2010.
21. Wolters, M., Isaac, K., Renals, S. "Evaluating Speech Synthesis Intelligibility using Amazon Mechanical Turk", In Proc. 7th Speech Synthesis Workshop (SSW7), 2010.
22. Adda, G., Mariani, J., "Language resources & Amazon Mechanical Turk ethical, legal and other issues", In Proc. LISLR2010, LREC2010.
23. CrowdFlower, http://www.crowdflower.com
24. Goto, M., Ogata, J., "PodCastle: Recent Advances of a Spoken Document Retrieval Service Improved by Anonymous User Contributions", Proc Interspeech2011, Florence, 2011.
25. Evanini, K., Zechner, K., "Using crowdsourcing to provide prosodic annotations for non-native speech", Proc. Interspeech2011, Florence, 2011.
26. Audhkasi, K., Georgiou, P., Narayanan, S., "Reliability-weighted acoustic model adaptation using crowd sourced transcriptions", Proc. Interspeech2011, Florence, 2011.
27. Cooke, M., Barker, J., Lecumberri, ML, Wasilewski, K., "Crowdsourcing for word recognition in noise", Proc, Interspeech2011, Florence, 2011.
28. Jurcicek, F., Keizer, S., Gasic, M., Mairesse, F., Thomson, B., Yu, K., Young, S., "Real User evaluation of spoken dialogue systems using Amazon Mechanical Turk", Proc, Interspeech2011, Florence, 2011.
29. Gelas, H., Abate, ST, Besacier, L., Pellegrino, F., "Quality assessment of crowdsourcing transcriptions for African languages", Proc Interspeech2011, Florence, 2011.
30. Bucholz, S., Latorre, J., "Crowdsourcing preference tests and how to detect cheating", Proc. Interspeech2011, Florence, 2011.
31. Lee, C., Glass, J., "A Transcription Task for Crowdsourcing with Automatic Quality Control", Proc. Interspeech2011, Florence, 2011.
32. McGraw, I., Glass, J., Seneff, S., "Growing a Spoken Language Interface on Amazon Mechanical Turk", Proc. Interspeech2011, Florence, 2011.