The annotation-validation (AV) model: Rewarding contribution using retrospective agreement

ACM International Conference Proceeding Series

Volumn , Issue , 2014, Pages 12-16

  Chamberlain, Jon ^a  

^a UNIVERSITY OF ESSEX   (United Kingdom)

Author keywords

Crowdsourcing; Gamification; User reward model

Indexed keywords

INFORMATION RETRIEVAL;

ANALYSIS OF DATA; CROWDSOURCING; GAMIFICATION; GOLD STANDARDS; HIGH QUALITY; LARGE DATASETS; TEXT ANNOTATIONS; USER PERFORMANCE;

COMPUTER SIMULATION;

EID: 84899809074     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2594776.2594779     Document Type: Conference Paper

References (14)

1. M.-D. Albakour, U. Kruschwitz, J. Niu, and M. Fasli. Autoadapt at the Session Track in TREC 2010. Gaithersburg, Maryland, USA, Nov. 2010.
2. A. Bosu, C. S. Corley, D. Heaton, D. Chatterji, J. C. Carver, and N. A. Kraft. Building reputation in stackover flow: An empirical investigation. In Proceedings of the 10th Working Conference on Mining Software Repositories, MSR '13, pages 89-92, Piscataway, NJ, USA, 2013. IEEE Press.
3. J. Chamberlain, K. Fort, U. Kruschwitz, L. Mathieu, and M. Poesio. ACM Transactions on Interactive Intelligent Systems, volume The People's Web Meets NLP: Collaboratively Constructed Language Resources, chapter Using Games to Create Language Resources: Successes and Limitations of the Approach. Springer, 2013.
4. J. Chamberlain and C. O'Reilly. User performance indicators in task-based data collection systems. In Proceedings of MindTheGap'14, 2014.
5. S. Deterding, D. Dixon, R. Khaled, and L. Nacke. From game design elements to gamefulness: Defining \gamification". In Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, MindTrek '11, pages 9-15, New York, NY, USA, 2011. ACM.
6. B. Kanefsky, N. Barlow, and V. Gulick. Can distributed volunteers accomplish massive data analysis tasks? Lunar and Planetary Science, XXXII, 2001.
7. H. Ma, R. Chandrasekar, C. Quirk, and A. Gupta. Improving search engines using human computation games. In Proceedings of the 18th ACM Conference on Information and Knowledge Management, CIKM '09, pages 275-284, New York, NY, USA, 2009. ACM.
8. M. Poesio, J. Chamberlain, U. Kruschwitz, L. Robaldo, and L. Ducceschi. Phrase detectives: Utilizing collective intelligence for internet-scale language resource creation. ACM Transactions on Interactive Intelligent Systems, 2013.
9. W. Rafelsberger and A. Scharl. Games with a purpose for social networking platforms. In Proceedings of the 20th ACM conference on Hypertext and Hypermedia, 2009.
10. V. C. Raykar, S. Yu, L. H. Zhao, G. H. Valadez, C. Florin, L. Bogoni, and L. Moy. Learning from crowds. Journal of Machine Learning Research, 11:1297-1322, Aug. 2010.
11. L. Schamber, M. Eisenberg, and M. S. Nilan. A re-examination of relevance: Toward a dynamic, situational definition. Information Processing &Management, 26(6):755-776, Nov. 1990.
12. K. Seaborn, P. Pennefather, and D. Fels. Reimagining leaderboards: Towards gamifying competency models through social game mechanics. In Proceedings of Gamification 2013: Gameful Design, Research, and Applications, pages 107-110, Stratford, Ontario, 2013. ACM.
13. L. von Ahn and L. Dabbish. Designing games with a purpose. Communications of the ACM, 51(8):58-67, 2008.
14. G. Zichermann and C. Cunningham. Gamification by Design: Implementing Game Mechanics in Web and Mobile Apps. O'Reilly Media, Inc., 2011.