Applying human computation mechanisms to Information Retrieval

Proceedings of the ASIST Annual Meeting

Volumn 49, Issue 1, 2012, Pages 1-10

  Harris, Christopher G ^a   Srinivasan, Padmini ^a  

^a UNIVERSITY OF IOWA   (United States)

Author keywords

Crowd sourcing; Games with a purpose; GWAP; Human computation; Information Retrieval

Indexed keywords

EID: 84878607367     PISSN: None     EISSN: 15508390     Source Type: Journal    
DOI: 10.1002/meet.14504901050     Document Type: Article

References (31)

1. Alonso, O. and Mizzaro, S. (2009) Can we get rid of TREC assessors? Using Mechanical Turk for relevance assessment. In Proc SIGIR'09, ACM, New York, NY.
2. Bennett, P. N., Chickering, D. M. et al. (2009) Learning consensus opinion: mining data from a labeling game. In Proc. WWW '09. ACM, New York, NY pp 121-130.
3. Bernstein, M. S., Little, G. et. al. (2010) Soylent: a word processor with a crowd inside. In Proc UIST'10. ACM, New York, NY, pp 313-322.
4. Csikszentmihalyi, M. (1991). Flow: The psychology of optimal experience, Harper Perennial Press
5. Eickhoff, C., Harris, C. G., de Vries, A.P., and P. Srinivasan. (2012) Quality through flow and immersion: gamifying crowdsourced relevance assessments. In Proc. SIGIR 2012, ACM, New York, NY.
6. Finin, T., W. Murnane, et al. (2010). Annotating named entities in Twitter data with crowdsourcing. In Proc. NAACL HLT 2010, Association for Computational Linguistics, Stroudsburg, PA, USA. pp 80-88.
7. Franklin, M., D. Kossmann, et al. (2011). CrowdDB: Answering queries with crowdsourcing. In Proc. SIGMOD (2011). ACM, New York, USA. pp 61-72.
8. Grady, C. and M. Lease (2010). Crowdsourcing document relevance assessment with Mechanical Turk, ACL, Stroudsburg, PA, USA
9. Hacker, S. and L. von Ahn (2009). Matchin: eliciting user preferences with an online game, In Proc CHI'09, ACM. New York, USA, pp 1207-1216.
10. Kumar, A. and M. Lease (2011). Learning to rank from a noisy crowd. In Proc SIGIR'11, ACM, New York, pp 1221-1222.
11. Lancaster, F. W. and Warner, A. J. Information Retrieval Today. Information Resources Press, Arlington, VA, USA, 1993.
12. Lieberman, H., D. Smith, et al. (2007). Common Consensus: a web-based game for collecting commonsense goals. In Proc. IUI'07, ACM, New York,NY.
13. Little, G., Chilton, L. B., et. al. (2010). TurKit: human computation algorithms on mechanical turk. In Proc. UIST '10. ACM, New York, NY, pp. 57-66.
14. Luon, Y., C. Aperjis, et al. Rankr: A Mobile System for Crowdsourcing Opinions.
15. Ma, H., R. Chandrasekar, et al. (2009). Improving search engines using human computation games, In Proc. HCOMP'09, ACM, New York, NY.
16. McCann, R., A. Doan, et al. (2003). Building data integration systems: A mass collaboration approach, AAAI, New York, NY.
17. McGonigal, J. (2011). Reality is broken: Why games make us better and how they can change the world, Penguin Pr.
18. Milne, D., D. M. Nichols, et al. (2008). A competitive environment for exploratory query expansion, In JCDL'08, ACM, New York, NY.
19. Naveed, N., T. Gottron, et al. (2011). Searching Microblogs: Coping with Sparsity and DocumentQuality. In Proc. CIKM'11. ACM, New York, NY pp 183-188.
20. Ng, V. (2007). Semantic class induction and coreference resolution. In Proc. ACL'07, ACL, Stroudsburg, PA, USA pp. 536-543.
21. Nowak, S. and S. Rüger (2010). Reliable Annotations via Crowdsourcing. In Proc. MIR'10. ACM, New York, NY. pp. 557-566.
22. Paiement, J. F., J. G. Shanahan, et al. (2010). Crowd Sourcing Local Search Relevance. In Proc. CrowdConf 2010. ACM, New York, NY.
23. Ravi, S. and K. Knight (2009). Minimized models for unsupervised part-of-speech tagging, In Proc. ACL'09, ACL, Stroudsburg, PA, USA. pp 504-512.
24. Robson, C., S. Kandel, et al. (2011). Data collection by the people, for the people. In Proc. CHI'11. Vancouver, BC, Canada, ACM, New York, pp 25-28.
25. Su, Q., D. Pavlov, et al. (2007). Internet-scale collection of human-reviewed data. In Proc. WWW'07. Banff, Alberta, Canada, ACM, New York. NY, pp 231-240.
26. Urbano, J., J. Morato, et al. (2010). Crowdsourcing preference judgments for evaluation of music similarity tasks. In Proc. CSE'10, ACM, New York, NY. pp 9-16.
27. von Ahn, L. and L. Dabbish (2004). Labeling images with a computer game, In Proc. CHI'04, ACM, New York, NY.
28. von Ahn, L., M. Kedia, et al. (2006). Verbosity: a game for collecting common-sense facts, In Proc. CHI'06, ACM, New York, NY, pp 75-78.
29. Whitehill, J., P. Ruvolo, et al. (2009). "Whose vote should count more: Optimal integration of labels from labelers of unknown expertise." Advances in Neural Information Processing Systems 22: pp 2035-2043.
30. Yan, T., V. Kumar, et al. (2010). CrowdSearch: exploiting crowds for accurate real-time image search on mobile phones, In Proc. MobiSys'10, ACM, New York, NY.
31. Zuccon, G., T. Leelanupab, et al. (2011). Crowdsourcing Interactions. In CSDM'11. ACM, New York, NY p 35.