Guess what? A game for affective annotation of video using crowd sourcing

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 6974 LNCS, Issue PART 1, 2011, Pages 277-285

  Riek, Laurel D ^a   O'Connor, Maria F ^b   Robinson, Peter ^b  

^a University of Notre Dame   (United States)

^b UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

crowd sourcing; emotion annotation; human computation; social context annotation; video annotation

Indexed keywords

CROWD SOURCING; EMOTION ANNOTATION; HUMAN COMPUTATION; SOCIAL CONTEXT; VIDEO ANNOTATIONS;

COMPUTER VISION; INFORMATION RETRIEVAL;

INTELLIGENT COMPUTING;

EID: 80054844987     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-24600-5_31     Document Type: Conference Paper

References (24)

1. von Ahn, L., Dabbish, L.: Labeling images with a computer game. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 319-326. ACM, New York (2004)
2. Baron-Cohen, S., Golan, O., Wheelwright, S., Hill, J.J.: Mind reading: the interactive guide to emotions (2004), http://www.jkp.com/mindreading
3. Bavelas, J., McGee, D., Phillips, B., Routledge, R.: Microanalysis of communication in psychotherapy. Human Systems 11(1), 47-66 (2000)
4. Bernstein, M., Chi, E., Chilton, L., Hartmann, B., Kittur, A., Miller, R.: Crowdsourcing and Human Computation: Systems, Studies and Platforms (May 2011), http://crowdresearch.org/chi2011-workshop/ (last accessed April 15, 2011)
5. Burke, M.A., Young, P.: Norms, Customs, and Conventions. In: Benhabib, J., Bisin, A., Jackson, M. (eds.) Handbook for Social Economics. Elsevier, Amsterdam (2010)
6. Calvo, R., D'Mello, S.: Affect Detection: An Interdisciplinary Review of Models, Methods, and their Applications. IEEE Transactions on Affective Computing 1(1), 18-37 (2010)
7. Ganjisaffar, Y., Javanmardi, S., Lopes, C.: Leveraging crowdsourcing heuristics to improve search in wikipedia. In: Proceedings of the 5th International Symposium on Wikis and Open Collaboration, WikiSym 2009, pp. 27:1-27:2. ACM, New York (2009)
8. Golan, O., Baron-Cohen, S.: Systemizing empathy: Teaching adults with Asperger syndrome or high-functioning autism to recognize complex emotions using interactive multimedia. Development and Psychopathology 18(02), 591-617 (2006)
9. Hayes, A.: SPSS, SAS, and Mplus Macros and Code (2011), http://www.afhayes.com/spss-sas-and-mplus-macros-and-code.html (last accessed April 15, 2011)
10. Hayes, A.F., Krippendorff, K.: Answering the call for a standard reliability measure for coding data. Communication Methods and Measures (2007)
11. Hsueh, P.Y., Melville, P., Sindhwani, V.: Data quality from crowdsourcing: a study of annotation selection criteria. In: Proceedings of the NAACL HLT 2009 Workshop on Active Learning for Natural Language Processing, HLT 2009, pp. 27-35. Association for Computational Linguistics (2009)
12. Kazai, G., Lease, M.: TREC 2011 Crowdsourcing Track (November 2011), https://sites.google.com/site/treccrowd2011 (last accessed April 15, 2011)
13. Krippendorff, K.: Content Analysis: An Introduction to Its Methodology. Sage Publications, Thousand Oaks (2004)
14. McCreadie, R., Macdonald, C., Ounis, I.: Crowdsourcing a news query classification dataset. In: Proceedings of the ACM SIGIR 2010 Workshop on Crowdsourcing for Search Evaluation (CSE 2010), pp. 31-38 (2010)
15. Nowak, S., Rüger, S.: How reliable are annotations via crowdsourcing: a study about inter-annotator agreement for multi-label image annotation. In: Proceedings of the International Conference on Multimedia Information Retrieval, pp. 557-566. ACM, New York (2010)
16. O'Connor, M.F.: Automatic Understanding of Social Scenes. Master's thesis, University of Cambridge (2011)
17. Pantic, M., Rothkrantz, L.: Toward an affect-sensitive multimodal humancomputer interaction. Proceedings of the IEEE 91(9), 1370-1390 (2003)
18. Philippot, P., Feldman, R., Coats, E.: The social context of nonverbal behavior. Cambridge Univ. Pr., Cambridge (1999)
19. Riek, L.D., Robinson, P.: Challenges and opportunities in building socially intelligent machines. IEEE Signal Processing (2011)
20. Scherer, K., Banziger, T., Roesch, E.: A blueprint for affective computing: A source-book. Oxford University Press, Oxford (2010)
21. Sheng, V., Provost, F., Ipeirotis, P.: Get another label? improving data quality and data mining using multiple, noisy labelers. In: Proceeding of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 614-622. ACM, New York (2008)
22. Snow, R., O'Connor, B., Jurafsky, D., Ng, A.: Cheap and fast-but is it good?: evaluating non-expert annotations for natural language tasks. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 254-263. Association for Computational Linguistics (2008)
23. Tarasov, A., Delany, S., Cullen, C.: Using crowdsourcing for labelling emotional speech assets. In: W3C workshop on Emotion ML (2010)
24. Yan, R., Yang, J., Hauptmann, A.: Automatically labeling video data using multiclass active learning. In: 9th IEEE International Conference on Computer Vision, pp. 516-523. IEEE, Los Alamitos (2003)