Opinion mining of Spanish customer comments with non-expert annotations on Mechanical Turk

Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, Mturk 2010 at the 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2010 - Proceedings

Volumn , Issue , 2010, Pages 114-121

  Mellebeek, Bart ^a   Benavent, Francesc ^a   Grivolla, Jens ^a   Codina, Joan ^a   Costa Jussà, Marta R ^a   Banchs, Rafael ^a  

^a Barcelona Media Innovation Center   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

COST EFFECTIVENESS; CROWDSOURCING; DATA MINING;

AI SYSTEMS; AMAZON'S MECHANICAL TURKS; CROWDSOURCING PLATFORMS; DATA DRIVEN; EXPERT ANNOTATIONS; GLOBAL WORKFORCE; HUMAN ANNOTATIONS; HUMAN INTELLIGENCE; MECHANICAL TURKS; TRAINING DATA;

SENTIMENT ANALYSIS;

EID: 85089558256     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (8)

1. A. Esuli and F. Sebastiani. 2006. SentiWordNet: a publicly available lexical resource for opinion mining. In Proceedings of LREC, volume 6.
2. B. D Eugenio and M. Glass. 2004. The kappa statistic: A second look. Computational linguistics, 30(1):95101.
3. Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H. Witten. 2009. The weka data mining software: an update. SIGKDD Explor. Newsl., 11(1):10-18.
4. A. Kittur, E. H Chi, and B. Suh. 2008. Crowdsourcing user studies with mechanical turk.
5. W. Mason and D. J Watts. 2009. Financial incentives and the performance of crowds. In Proceedings of the ACM SIGKDD Workshop on Human Computation, pages 77-85.
6. A. K. McCallum. 2002. Mallet: A machine learning for language toolkit. http://mallet.cs.umass.edu.
7. R. Snow, B. O'Connor, D. Jurafsky, and A. Y Ng. 2008. Cheap and fastbut is it good?: evaluating non-expert annotations for natural language tasks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 254-263.
8. Q. Su, D. Pavlov, J. H Chow, and W. C Baker. 2007. Internet-scale collection of human-reviewed data. In Proceedings of the 16th international conference on World Wide Web, pages 231-240.