Using the Amazon mechanical turk for transcription of spoken language

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2010, Pages 5270-5273

  Marge, Matthew ^a   Banerjee, Satanjeev ^a   Rudnicky, Alexander I ^a  

^a Carnegie Mellon University ^*  (United States)

Author keywords

Crowd sourcing; Speech transcription

Indexed keywords

SPEECH TRANSMISSION;

AMAZON'S MECHANICAL TURKS; CROWD SOURCING; MECHANICAL TURKS; NON-NATIVE; RELIABLE METHODS; SPEECH TRANSCRIPTIONS; SPOKEN LANGUAGES; TRANSCRIPTION METHODS; VOTING SCHEMES; WORKERS';

TRANSCRIPTION;

EID: 78049407752     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2010.5494979     Document Type: Conference Paper

References (9)

1. A. Gruenstein, I. McGraw, and A. Sutherland, "A self-transcribing speech corpus: collecting continuous speech with an online educational game," in SLaTE Workshop, 2009.
2. I. McGraw, A. Gruenstein, and A. Sutherland, "A self-labeling speech corpus: Collecting spoken words with an online educational game," in Interspeech, 2009.
3. R. Snow, B. O'Connor, D. Jurafsky, and A. Y. Ng, "Cheap and fast - but is it good? Evaluating non-expert annotations for natural langauge tasks," in EMNLP, 2008.
4. C. Callison-Burch, "Fast, cheap, and creative: Evaluating translation quality using Amazon's Mechanical Turk," in EMNLP, 2009.
5. A. Kittur, E. H. Chi, and B. Suh, "Crowdsourcing user studies with Mechanical Turk," in CHI '08, Florence, Italy, 2008.
6. The NIST Rich Transcription Evaluation Project. [Online]. http://www.itl.nist.gov/iad/mig/tests/rt
7. C. Bennett and A. I. Rudnicky, "The Carnegie Mellon Communicator corpus," in ICSLP, 2002.
8. K. Zechner, "What did they actually say? Agreement and Disagreement among Transcribers of Non-Native Spontaneous Speech Responses in an English Proficiency Test," in SLaTE Workshop, 2009.
9. J. G. Fiscus, "A post-processing system to yield word error rates: Recognizer Output Voting Error Reduction (ROVER)," in ASRU Workshop, 1997.