Using crowdsourcing to capture complexity in human interpretations of multimedia content

Advances in Computer Vision and Pattern Recognition

Volumn 64, Issue , 2014, Pages 229-269

  Larson, Martha ^a   Melenhorst, Mark ^a   Menéndez, María ^b   Xu, Peng ^a  

^a DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

^b UNIVERSITY OF TRENTO   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84986213316     PISSN: 21916586     EISSN: 21916594     Source Type: Book Series    
DOI: 10.1007/978-3-319-05696-8_10     Document Type: Chapter

References (35)

1. Barthes R (1977) Image music text. Hill and Wang, New York
2. Beaver DI, Geurts B (2013) Presupposition. In: Zalta EN (ed) The Stanford encyclopedia of philosophy (Summer 2011 Edition). http://www.plato.stanford.edu/archives/fall2013/entries/presupposition
3. Borth D, Ji R, Chen T, Breuel T, Chang S-F (2013) Large-scale visual sentiment ontology and detectors using adjective noun pairs. In: Proceedings of the 21st ACM international conference on multimedia (MM’13). ACM, New York, pp 223-232
4. Chen K-T, Chang C-J, Wu C-C, Chang Y-C, Lei C-L (2010) Quadrant of euphoria: a crowd-sourcing platform for QoE assessment. IEEE Netw 24(2):28-35
5. Cockton G, Woolrich A (2001) Understanding inspection methods: lessons from an assessment of heuristic evaluation. In: Blandford A, Vanderdonckt J, Gray Ph (eds) People and computers XV-interaction without Frontiers, pp 171-191
6. Conotter V, Dang-Nguyen D-T, Boato G, Menéndez M, Larson M (2014) Assessing the impact of image manipulation on users’ perceptions of deception. In: Proc. SPIE 9014, Human Vision and Electronic Imaging XIX, 90140Y
7. Deng J, Dong W, Socher R, Li L-J, Li K, Li F-F (2009) ImageNet: a large-scale hierarchical image database. In: CVPR’09: IEEE conference on computer vision and pattern recognition, pp 248-255
8. Galli L, Fraternali P, Martinenghi D, Novak J (2012) A draw-and-guess game to segment images. In: SocialComm 2012 ASE/IEEE international conference on social computing, pp 914-917
9. Genevieve P (2012) SUN attribute database: discovering, annotating, and recognizing scene attributes. In Proceedings of the 2012 IEEE conference on computer vision and pattern recognition (CVPR), pp 2751-2758
10. Glaser BG, Strauss A (1967) Discovery of grounded theory. In: Strategies for qualitative research. Sociology Press, Mill Valley
11. Hossfeld T, Keimel C, Hirth M, Gardlo B, Habigt J, Diepold K, Tran-Gia P (2014) Best practices for QoE crowdtesting: QoE assessment with crowdsourcing. IEEE Trans Multimedia 16:541-558
12. Howe J (2006) The Rise of crowdsourcing wired. Wired Mag 14(06):1-6
13. Kennedy L, Hauptmann A (2006) LSCOM Lexicon definitions and annotations (Version 1.0). Computer science department, Paper 949. http://www.repository.cmu.edu/compsci/949
14. Liu D, Hua X-S, Yang L, Wang M, Zhang H-J (2009) Tag ranking. In: WWW 2009 Proceedings of the 18th international conference on world wide web, pp. 351-360
15. Mason W, Suri S (2012) Conducting behavioral research on Amazon’s Mechanical Turk. Behav Res Methods 44(1):1-23
16. Morris RR, Dontcheva M, Gerber EM (2012) Priming for better performance in microtask crowdsourcing environments. IEEE Internet Comput 16(5):13-19
17. Murphy GL (2002) The big book of concepts. The MIT Press, Cambridge
18. Nie L, Yan S, Wang M, Hong R, Chua T-S (2012) Harvesting visual concepts for image search with complex queries. In: Proceedings of the 20th ACM international conference on multimedia (MM’12). ACM, New York, pp 59-68
19. Nielsen J (1994) Enhancing the explanatory power of usability heuristics. In: Proceedings of the ACM CHI’94 conference, pp 152-158
20. Nielsen J, Landauer TKA (1993) Mathematical model of the finding of usability problems. In: Proceedings of ACM INTERCHI’93 conference, pp 206-213
21. Nowak S, Rueger S (2010) How reliable are annotations via crowdsourcing: a study about inter-annotator agreement for multi-label image annotation. In: MIR’10 Proceedings of the international conference on multimedia, information retrieval
22. Paolacci G, Chandler J, Ipeirotis PG (2010) Running experiments on amazon mechanical turk. Judgment Decision Making 5:5
23. Parikh D, Grauman K (2011) Interactively building a discriminative vocabulary of nameable attributes. In: Computer vision and pattern recognition (CVPR), pp 1681-1688
24. Prelec D (2004) A bayesian truth serum for subjective data. Science 306(5695):462-6
25. Quinn AJ, Bederson BB (2011) Human computation: a survey and taxonomy of a growing field. Proc ACM CHI 2011:1403-12
26. Rafferty P, Hidderly R (2005) Indexing multimedia and creative works. Ashgate, Farnham
27. Ross J, Lilly Irani M, Silberman S, Zaldivar A, Tomlinson B (2010) Who are the crowdworkers?: shifting demographics in mechanical turk. In: CHI’10 Extended abstracts on human factors in computing systems (CHI EA’10). ACM, New York, pp 2863-2872
28. Smeaton AF, Over P, Kraaij W (2006) Evaluation campaigns and TRECVid. In: Proceedings of the 8th ACM international workshop on multimedia information retrieval (MIR’06) pp. 321-330
29. Snoek CGM, Freiburg B, Oomen J, Ordelman R (2010) Crowdsourcing rock n’ roll multimedia retrieval. In: MM’10 Proceedings of the international conference on multimedia, pp. 1535-1538
30. Szabó ZG (2013) Compositionality. In: Zalta EN (ed) The stanford encyclopedia of philosophy. (Fall, Fall River). http://www.plato.stanford.edu/archives/fall2013/entries/compositionality
31. Van den Haak MJ, Jong MDT, Schellens PJ (2004) Employing think-aloud protocols and constructive interaction to test the usability of online library catalogues: a methodological comparison. Interact Comput 16(6):1153-70
32. Van der Geest T, Spyridakis JH (2000) Developing heuristics for web communication. Tech Commun 47(3):359-82
33. Vliegendhart R, Larson MA, Pouwelse JA (2012) Discovering user perceptions of semantic similarity in near-duplicate multimedia files. In: CrowdSearch 2012: First international workshop on crowdsourcing web search, pp 54-58
34. Wieneck L, Düring M, Sillaume G, Lallemand C, Croce V, Lazzaro M, Nucci F, Pasini C, Fraternali P, Tagliasacchi M, Melenhorst M, Novak J, Micheel I (2013) Building the social graph of the history of European integration. A pipeline for humanist-machine interaction in the digital humanities. In: Proceedings of the conference histoinformatics 2013, Kyoto
35. Xirong Li; Snoek CGM, Worring M, Smeulders AWM (2012) Harvesting social images for bi-concept search. IEEE Trans Multimedia 14(4):1091-1104