Curiosity killed the cat, but makes crowdwork better

Conference on Human Factors in Computing Systems - Proceedings

Volumn , Issue , 2016, Pages 4098-4110

  Law, Edith ^a   Yin, Ming ^b   Goh, Joslin ^a   Chen, Kevin ^a   Terry, Michael ^a   Gajos, Krzysztof Z ^b  

^a UNIVERSITY OF WATERLOO   (Canada)

^b HARVARD UNIVERSITY   (United States)

Author keywords

Crowdsourcing; Curiosity; Motivation

Indexed keywords

HUMAN COMPUTER INTERACTION; HUMAN ENGINEERING; MOTIVATION;

AMAZON'S MECHANICAL TURKS; CRITICAL QUESTIONS; CURIOSITY; PERSONAL CHARACTERISTICS; TASK INTERFACE;

CROWDSOURCING;

EID: 85015044190     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2858036.2858144     Document Type: Conference Paper

References (66)

1. Anderson, S. Applying curiosity to interaction design: Tell me something I don't know, 2009. Last Retrieved January 7, 2016 from http://johnnyholland.org/2009/08/curiosity-and-interaction-design/.
2. Ariely, D., Kamenica, E., and Prelec, D. Man's search for meaning: The case of legos. Journal of Economic Behaviour and Organization 67, 3(2008), 671-677.
3. Beenen, G., Ling, K., Wang, X., Chang, K., Frankowski, D., Resnick, P., and Kraut, R. E. Using social psychology to motivate contributions to online communities. In Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work, CSCW'04, ACM (New York, NY, USA, 2004), 212-221.
4. Berlyne, D. An experimental study of human curiosity. British Journal of Psychology 45(1954), 256-265.
5. Berlyne, D. A theory of human curiosity. British Journal of Psychology 45(1954), 180-191.
6. Berlyne, D. Conflict, arousal and curiosity. McGraw-Hill, London, 1960.
7. Bowser, A., Hansen, D., He, Y., Boston, C., Reid, M., Gunnell, L., and Preece, J. Using gamification to inspire new citizen science volunteers. In Proceedings of the First International Conference on Gameful Design, Research, and Applications, Gamification'13, ACM (New York, NY, USA, 2013), 18-25.
8. Caplin, A., and Leahy, J. Psychological expected utility theory and anticipatory feelings. Quarterly Journal of Economics (2001), 55-79.
9. Chandler, D., and Kapelner, A. Breaking monotony with meaning: Motivation in crowdsourcing markets. Journal of Economic Behaviour and Organization 90(2013), 123-133.
10. Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenen, M., Leaver-Fay, A., Baker, D., Popovic, Z., and Players, F. Predicting protein structures with a multiplayer online game. Nature 466(August 2010), 756-760.
11. Dai, P., Rzeszotarski, J. M., Paritosh, P., and Chi, E. H. And now for something completely different: Improving crowdsourcing workflows with micro-diversions. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work, CSCW'15, ACM (New York, NY, USA, 2015), 628-638.
12. Easley, D., and Ghosh, A. Incentives, gamification, and game theory: An economic approach to badge design. In Proceedings of the Fourteenth ACM Conference on Electronic Commerce, EC'13, ACM (New York, NY, USA, 2013), 359-376.
13. Festinger, L. A theory of social comparison processes. Human Relations 7(1954), 117-140.
14. Feyisetan, O., Simperl, E., Van Kleek, M., and Shadbolt, N. Improving paid microtasks through gamification and adaptive furtherance incentives. In Proceedings of the 24th International Conference on World Wide Web, International World Wide Web Conferences Steering Committee (2015), 333-343.
15. Frank Bretz, T. H., and Westfall, P. Multiple Comparisons Using R. Chapman & Hall/CRC, 2011.
16. Gaver, W. W., Beaver, J., and Benford, S. Ambiguity as a resource for design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI'03, ACM (New York, NY, USA, 2003), 233-240.
17. Gollub, L. Information on conditioned reinforcement. Journal of the Experimental Analysis of Behaviour (2001), 361-372.
18. Grevet, C., Mankoff, J., and Anderson, S. D. Design and evaluation of a social visualization aimed at encouraging sustainable behavior. In Proceedings of the 2010 43rd Hawaii International Conference on System Sciences, HICSS'10, IEEE Computer Society (Washington, DC, USA, 2010), 1-8.
19. Harris, C. You're hired! an examination of crowdsourcing incentive models in human resource tasks. In ACM International Conference on Web Search and Data Mining, WSDM'11(2011), 15-18.
20. Ho, C.-J., Slivkins, A., Suri, S., and Vaughan, J. W. Incentivizing high quality crowdwork. In Proceedings of the 24th International Conference on World Wide Web, International World Wide Web Conferences Steering Committee (2015), 419-429.
21. Horton, J. J., and Chilton, L. B. The labor economics of paid crowdsourcing. In Proceedings of the 11th ACM Conference on Electronic Commerce, EC'10, ACM (New York, NY, USA, 2010), 209-218.
22. Hosmer, D. W., and Lemeshow, S. Goodness of fit tests for the multiple logistic regression model. Communications in Statistics: Theory and Methods 9, 10(1980), 1043-1069.
23. Huang, S.-W., and Fu, W.-T. Don't hide in the crowd!: Increasing social transparency between peer workers improves crowdsourcing outcomes. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI'13, ACM (New York, NY, USA, 2013), 621-630.
24. Jones, S. Curiosity and knowledge. Psychological Reports 45(1979), 639-642.
25. Kang, M., Hsu, M., Krajbich, I., Loewenstein, G., McClure, S., Wang, J., and Camerer, C. The wick in the candle of learning: epistemic curiosity activates reward circuitry and enhances memory. Psychological Science 20, 8(2009), 963-973.
26. Kinnaird, P., Dabbish, L., Kiesler, S., and Faste, H. Co-worker transparency in a microtask marketplace. In Proceedings of the 2013 Conference on Computer Supported Cooperative Work, CSCW'13, ACM (New York, NY, USA, 2013), 1285-1290.
27. Koffka, K. Principles of Gestalt Psychology. Harcourt-Brace, New York, 1935.
28. Kruskal, W., and Wallis, W. A. Use of ranks in one-criterion variance analysis. Journal of the American Statistical Association 47, 260(1952), 583-621.
29. Langer, K. Suspenseful Design: Engaging Emotionally with Complex Applications through Compelling Narratives. Master's thesis, University of Waterloo, Waterloo, Ontario, 2014.
30. Law, E., and von Ahn, L. Human Computation. Synthesis Lectures on Artificial Intelligence and Machine Learning, Morgan and Claypool, June 2011.
31. Litman, J. Curiosity and the pleasures of learning: Wanting and liking new information. Cognition and Emotion 19, 6(2005), 793-814.
32. Litman, J., Hutchins, T., and Russon, R. Epistemic curiosity, feeling-of-knowing, and exploratory behavior. Cognition and Emotion 19, 4(2005), 559-582.
33. Locke, E., and Latham, G. Building a practically useful theory of goal setting and task motivation: A 35 year odyssey. American Psychologist 57, 9(2002), 705-717.
34. Loewenstein, G. The psychology of curiosity: A review and reinterpretation. Cognition and Emotion 19, 6(2005), 793-814.
35. Malone, T. Toward a theory of intrinsically motivating instruction. Cognitive Science 4(1981), 333-369.
36. Markey, A., and Lowenstein, G. Curiosity. In International Handbook of Emotions in Eduction, R. Pekrun and L. Linnenbrink-Garcia, Eds. Rutledge, New York, 2014.
37. Marlow, J., and Dabbish, L. A. The effects of visualizing activity history on attitudes and behaviors in a peer production context. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work, CSCW'15, ACM (New York, NY, USA, 2015), 757-764.
38. Mason, W., and Watts, D. Financial incentives and the "performance of crowd". In ACM SIGKDD Workshop on Human Computation (HCOMP) (2009), 77-85.
39. McCullagh, P., and Nelder, J. Generalized Linear Modesl, 2 ed. Chapman & Hall/CRC, 1989.
40. Menon, S., and Soman, D. Managing the power of curiosity for effective web advertising strategies. Journal of Advertising 31, 3(2002), 1-14.
41. Metzger, W. Laws of Seeing. MIT Press, Cambridge, MA, 2006.
42. Miller, N. Liberalization of basic s-r concepts: Extensions to conflict behavior, motivation and social learning. Psychology: the study of a science (1959), 196-292.
43. Naylor, F. A state-trait curiosity inventory. Australian Psychology 16(1981), 172-183.
44. Osius, G., and Rojek, D. Normal goodness-of-fit tests for multinomial models with large degrees of freedom. Journal of the American Statistical Association 87, 140(1992), 1145-1152.
45. Pluck, G., and Johnson, H. Stimulating curiosity to enhance learning. GESJ: Education Sciences and Psychology 2, 19(2011), 1512-1801.
46. Raddick, M., Bracey, G., Gay, P., Lintott, C., Cardamone, C., Murray, P., Schawinski, K., Szalay, A., and Vandenberg, J. Galaxy zoo: Motivations of citizen scientists. Astronomy Education Review 12, 1 (2013).
47. Rashid, A. M., Ling, K., Tassone, R. D., Resnick, P., Kraut, R., and Riedl, J. Motivating participation by displaying the value of contribution. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI'06, ACM (New York, NY, USA, 2006), 955-958.
48. Rogstadius, J., Kostakos, V., Kittur, A., Smus, B., Laredo, J., and Vukovic, M. An assessment of intrinsic and extrinsic motivation on task performance in crowdsourcing markets. In AAAI Conference on Weblogs and Social Media (2011).
49. Rotman, D., Preece, J., Hammock, J., Procita, K., Hansen, D., Parr, C., Lewis, D., and Jacobs, D. Dynamic changes in motivation in collaborative citizen-science projects. In Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work, ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW) '12(2012), 217-226.
50. Ryan, R. Control and information in the interpersonal sphere: An extension of cognitive evaluation theory. Journal of Personality and Social Psychology 42(1982), 450-461.
51. Schmitt, F., and Lahroodi, R. The epistemic value of curiosity. Education Theory 8(2008), 125-148.
52. Segal, A., Simpson, R., Gal, Y., Homsy, V., Heartwood, M., Page, K., and Jirotka, M. Improving productivity in citizen science through controlled intervention. In WWW (2015), 331-337.
53. Shaw, A. D., Horton, J. J., and Chen, D. L. Designing incentives for inexpert human raters. In Proceedings of the ACM 2011 Conference on Computer Supported Cooperative Work, CSCW'11, ACM (New York, NY, USA, 2011), 275-284.
54. Shirk, J., Ballard, H., Wilderman, C., Phillips, T., Wiggins, A., Jordan, R., and Bonney, R. Public participation in scientific research: A framework for intentional design. Ecology and Society 17, 2(2012), 29-48.
55. Silvia, P. J. Curiosity and motivation. In Oxford Handbook of Motivation, R. M. Ryan, Ed. Oxford University Press, New York, 2014.
56. Speilberger, C., and Starr, L. Curiosity and exploratory behavior. Motivation, Theory and Research (1994), 221-243.
57. Teodoro, R., Ozturk, P., Naaman, M., Mason, W., and Lindqvist, J. The motivations and experiences of the on-demand mobile workforce. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work, CSCW'14, ACM (New York, NY, USA, 2014), 236-247.
58. Tieben, R., Bekker, T., and Schouten, B. Curiosity and interaction: Making people curious through interactive systems. In Proceedings of the 25th BCS Conference on Human-Computer Interaction, BCS-HCI'11, British Computer Society (Swinton, UK, UK, 2011), 361-370.
59. von Ahn, L., and Dabbish, L. Designing games with a purpose. Communications of the ACM 51, 8(2008), 58-67.
60. Wasserman, L. All of Statistics: A Concise Course in Statistical Inference. Springer-Verlag, Berlin, 2003.
61. Wilson, A., Burnett, M., Beckwith, L., Granatir, O., Casburn, L., Cook, C., Durham, M., and Rothermel, G. Harnessing curiosity to increase correctness in end-user programming. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI'03, ACM (New York, NY, USA, 2003), 305-312.
62. Yin, M., and Chen, Y. Bonus or not? learn to reward in crowdsourcing. In Proceedings of the 24th International Joint Conference on Artificial Intelligence, IJCAI'15(2015), 201-207.
63. Yin, M., Chen, Y., and Sun, Y. The effects of performance-contingent financial incentives in online labor markets. In Conference on Artificial Intelligence, AAAI'13(2013), 1191-1197.
64. Zeiler, M. Schedules of reinforcement. In Handbook of Operant Behaviour, W. Hong and J. Staddon, Eds. Prentice Hall, Englewood Cliffs, NJ, 1977.
65. Zhu, H., Kraut, R., and Kittur, A. Organizing without formal organization: Group identification, goal setting and social modeling in directing online production. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work, CSCW'12, ACM (New York, NY, USA, 2012), 935-944.
66. Zion, M., and Sadeh, I. Curiosity and open inquiry learning. Journal of Biological Education 41(2007), 162-8.