Atelier: Repurposing Expert crowdsourcing tasks as micro-internships

Conference on Human Factors in Computing Systems - Proceedings

Volumn , Issue , 2016, Pages 2645-2656

  Suzuki, Ryo ^a   Salehi, Niloufar ^b   Lam, Michelle S ^b   Marroquin, Juan C ^b   Bernstein, Michael S ^b  

^a UNIVERSITY OF COLORADO   (United States)

^b STANFORD UNIVERSITY   (United States)

Author keywords

Crowd work; Crowdsourcing; Micro internships

Indexed keywords

COMMERCE; HUMAN COMPUTER INTERACTION; HUMAN ENGINEERING;

BEST PRACTICES; BREAK DOWN; CROWD WORK; FIELD EXPERIMENT; MICRO-INTERNSHIPS; REPURPOSING; SKILL DEVELOPMENT; WORK EXPERIENCE;

CROWDSOURCING;

EID: 85014797833     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2858036.2858121     Document Type: Conference Paper

References (41)

1. Susan A Ambrose, Michael W Bridges, Michele DiPietro, Marsha C Lovett, and Marie K Norman. 2010. How learning works: Seven research-based principles for smart teaching. John Wiley & Sons.
2. Judd Antin and Aaron Shaw. 2012. Social desirability bias and self-reports of motivation: a study of Amazon Mechanical Turk in the US and India. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2925-2934.
3. Alan S Blinder. 2006. Offshoring: the next industrial revolution? FOREIGN AFFAIRS-NEW YORK 85, 2(2006), 113.
4. Benjamin S Bloom. 1984. The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational researcher (1984), 4-16.
5. Andrew Bosworth. 2009. Facebook Engineering Bootcamp. https://www.facebook.com/notes/177577963919/. (2009).
6. John D Bransford, Ann L Brown, and Rodney R Cocking. 1999. How people learn: Brain, mind, experience, and school. National Academy Press.
7. John Seely Brown, Allan Collins, and Paul Duguid. 1989. Situated cognition and the culture of learning. Educational researcher 18, 1(1989), 32-42.
8. Allan Collins, John Seely Brown, and Susan E Newman. 1989. Cognitive apprenticeship: Teaching the crafts of reading, writing, and mathematics. Knowing, learning, and instruction: Essays in honor of Robert Glaser 18(1989), 32-42.
9. Elizabeth A Davis and Naomi Miyake. 2004. Explorations of scaffolding in complex classroom systems. The Journal of the Learning Sciences 13, 3(2004), 265-272.
10. Mira Dontcheva, Robert R Morris, Joel R Brandt, and Elizabeth M Gerber. 2014. Combining crowdsourcing and learning to improve engagement and performance. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems. ACM, 3379-3388.
11. Steven Dow, Anand Kulkarni, Scott Klemmer, and Björn Hartmann. 2012. Shepherding the crowd yields better work. In Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work. ACM, 1013-1022.
12. George W. Furnas, Thomas K. Landauer, Louis M. Gomez, and Susan T. Dumais. 1987. The vocabulary problem in human-system communication. Commun. ACM 30, 11(1987), 964-971.
13. Philip J Guo. 2015. Codeopticon: Real-Time, One-To-Many Human Tutoring for Computer Programming (To appear). In Proceedings of the 28th annual ACM symposium on User interface software and technology. ACM.
14. Neha Gupta, David Martin, Benjamin V Hanrahan, and Jacki O'Neill. 2014. Turk-Life in India. In Proceedings of the 18th International Conference on Supporting Group Work. ACM, 1-11.
15. Daniel Haas, Jason Ansel, Lydia Gu, and Adam Marcus. 2015. Argonaut: Macrotask Crowdsourcing for Complex Data Processing. Proceedings of the VLDB Endowment 8, 12 (2015).
16. Pamela Hinds. 2002. Distributed work. MIT Press.
17. Jim Hollan and Scott Stornetta. 1992. Beyond being there. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 119-125.
18. Panagiotis G Ipeirotis. 2010. Demographics of Mechanical Turk. Working paper no CEDER-10-01. Available at: http://ssrn.com/abstract=1585030 (2010).
19. Nicolas Kaufmann and Thimo Schulze. 2011. Worker motivation in crowdsourcing and human computation. Education 17, 2009(2011), 6.
20. Aniket Kittur, Jeffrey V Nickerson, Michael Bernstein, Elizabeth Gerber, Aaron Shaw, John Zimmerman, Matt Lease, and John Horton. 2013. The future of crowd work. In Proceedings of the 2013 conference on Computer supported cooperative work. ACM, 1301-1318.
21. Marios Kokkodis and Panos Ipeirotis. 2014. The utility of skills in online labor markets. Thirty-Fifth International Conference on Information Systems, Auckland 2014 (2014).
22. Marios Kokkodis and Panagiotis G Ipeirotis. 2015. Reputation Transferability in Online Labor Markets. Management Science (2015).
23. Anand Kulkarni, Philipp Gutheim, Prayag Narula, David Rolnitzky, Tapan Parikh, and Björn Hartmann. 2012. Mobileworks: Designing for quality in a managed crowdsourcing architecture. Internet Computing, IEEE 16, 5(2012), 28-35.
24. Chinmay Kulkarni, Michael S Bernstein, and Scott Klemmer. 2015. PeerStudio: Rapid Peer Feedback Emphasizes Revision and Improves Performance. In Proceedings from The Second (2015) ACM Conference on Learning@ Scale. 75-84.
25. Ben Lafreniere, Andrea Bunt, Matthew Lount, and Michael A Terry. 2013. Understanding the Roles and Uses of Web Tutorials.. In ICWSM.
26. Min Liu. 1998. A study of engaging high-school students as multimedia designers in a cognitive apprenticeship-style learning environment. Computers in Human Behavior 14, 3(1998), 387-415.
27. Edwin A Locke. 1980. Goal Setting. Technical Report. DTIC Document.
28. Kurt Luther, Jari-Lee Tolentino, Wei Wu, Amy Pavel, Brian P Bailey, Maneesh Agrawala, Björn Hartmann, and Steven P Dow. 2015. Structuring, Aggregating, and Evaluating Crowdsourced Design Critique. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM, 473-485.
29. David Martin, Benjamin V Hanrahan, Jacki O'Neill, and Neha Gupta. 2014. Being a Turker. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing. ACM, 224-235.
30. Christine Nash and Dave Collins. 2006. Tacit knowledge in expert coaching: Science or art? Quest 58, 4(2006), 465-477.
31. Michael I Norton, Daniel Mochon, and Dan Ariely. 2012. The'IKEA effect': When labor leads to love. J Consum Psychol. 22:453460. doi: 10.1016/j.jcps.2011.08.002 (2012).
32. Seymour Papert. 1980. Mindstorms: Children, computers, and powerful ideas. Basic Books, Inc.
33. Daniel Rees Lewis, Emily Harburg, Elizabeth Gerber, and Matthew Easterday. 2015. Building Support Tools to Connect Novice Designers with Professional Coaches. In Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition. ACM, 43-52.
34. Daniela Retelny, Sébastien Robaszkiewicz, Alexandra To, Walter S Lasecki, Jay Patel, Negar Rahmati, Tulsee Doshi, Melissa Valentine, and Michael S Bernstein. 2014. Expert crowdsourcing with flash teams. In Proceedings of the 27th annual ACM symposium on User interface software and technology. ACM, 75-85.
35. Roger C Schank, Tamara R Berman, and Kimberli A Macpherson. 1999. Learning by doing. Instructional-design theories and models: A new paradigm of instructional theory 2(1999), 161-181.
36. Daniel L Schwartz and John D Bransford. 1998. A time for telling. Cognition and instruction 16, 4(1998), 475-5223.
37. C Addison Stone. 1998. The metaphor of scaffolding its utility for the field of learning disabilities. Journal of learning disabilities 31, 4(1998), 344-364.
38. Upwork. 2015. Freelancing in America: 2015. https://www.upwork.com/i/freelancinginamerica2015/. (2015).
39. Lev Vygotsky. 1978. Interaction between learning and development. Readings on the development of children 23, 3(1978), 34-41.
40. Rob Wass and Clinton Golding. 2014. Sharpening a tool for teaching: the zone of proximal development. Teaching in Higher Education 19, 6(2014), 671-684.
41. David Wood, Jerome S Bruner, Gail Ross, and others. 1976. The role of tutoring in problem solving. Journal of child psychology and psychiatry 17, 2(1976), 89-100.