User requirements of a crowd sourcing platform for researchers: Findings from a series of focus groups

Proceedings - Pacific Asia Conference on Information Systems, PACIS 2014

Volumn , Issue , 2014, Pages

  Schlagwein, Daniel ^a   Daneshgar, Farhad ^a  

^a UNIVERSITY OF NEW SOUTH WALES   (Australia)

Author keywords

Citizen science; Crowdsourcing platforms; Crowdsourcing research; Networked science; User requirements

Indexed keywords

DESIGN;

CITIZEN SCIENCE; CROWDSOURCING; CROWDSOURCING PLATFORMS; NETWORKED SCIENCE; USER REQUIREMENTS;

INFORMATION SYSTEMS;

EID: 84928593427     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (49)

1. Afuah, A., and Tucci, C.L. 2012. "Crowdsourcing As a Solution to Distant Search," Academy of Management Review (37:3), pp. 355-375.
2. Aguinis, H., and Lawal, S.O. 2012. "Conducting field experiments using eLancing's natural environment," Journal of Business Venturing (27:4), pp. 493-505.
3. Buhrmester, M., Kwang, T., and Gosling, S.D. 2011. "Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data," Perspectives on Psychological Science (6:1), pp. 3-5.
4. Carignani, A., and Negri, L. 2010. "We Don't Need No Education: Required Abilities in Online Labor Markets," International Conference on Information Systems (ICIS), St. Louis, MO, USA.
5. Chen, J.J., Menezes, N.J., and Bradley, A.D. 2011. "Opportunities for Crowdsourcing Research on Amazon Mechanical Turk," in: ACM Conference on Human Factors in Computing Systems (CHI) 2011, Workshop on Crowdsourcing and Human Computation.
6. Chen, K.-T., and Chang, C.-J. 2010. "Quadrant of Euphoria: A Crowdsourcing Platform for QoE Assessment," IEEE Network (3-4), pp. 28-35.
7. Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenen, M., Leaver-Fay, A., Baker, D., and Popovic, Z. 2010. "Predicting Protein Structures with a Multiplayer Online Game," Nature (466:7307), pp. 756-760.
8. Crowston, K. 2012. "Amazon Mechanical Turk: A Research Tool for Organizations and Information Systems Scholars," IFIP WG 8.2 Working Conference.
9. Crump, M.J., McDonnell, J.V., and Gureckis, T.M. 2013. "Evaluating Amazon's Mechanical Turk as a Tool for Experimental Behavioral Research," PLOS ONE (8:3), p. e57410.
10. Feller, J., Finnegan, P., Hayes, J., and O'Reilly, P. 2012. "'Orchestrating' Sustainable Crowdsourcing: A Characterisation of Solver Brokerages," Journal of Strategic Information Systems (21:3), pp. 216-232.
11. Felstiner, A. 2011. "Working the Crowd: Employment and Labor Law in the Crowdsourcing Industry," Berkeley Journal of Employment and Labor Law (32:1), pp. 143-143.
12. Frey, K., Lüthje, C., and Haag, S. 2011. "Whom Should Firms Attract to Open Innovation Platforms? The Role of Knowledge Diversity and Motivation.," Long Range Planning (44:5/6), pp. 397-420.
13. Gallaugher, J., and Ransbotham, S. 2010. "Social Media and Customer Dialog Management at Starbucks," MIS Quarterly Executive (9:4), pp. 197-212.
14. Heer, J., and Bostock, M. 2010. "Crowdsourcing Graphical Perception: Using Mechanical Turk to Assess Visualization Design," ACM Conference on Human Factors in Computing Systems (CHI) 2010, Atlanta, GA, USA.
15. Hevner, A.R., March, S.T., Park, J., and Ram, S. 2004. "Design Science in Information Systems Research," MIS Quarterly (28:1), pp. 75-106.
16. Howe, J. 2006. "The Rise of Crowdsourcing," Wired (14:6), pp. 176-183.
17. Hull, E., Jackson, K., and Dick, J. 2005. Requirements Engineering. London, UK: Springer.
18. Ipeirotis, P. 2010. "Demographics of Mechanical Turk," SSRN Working Paper 1585030).
19. Ipeirotis, P.G., Provost, F., and Wang, J. 2010. "Quality management on amazon mechanical turk," Proceedings of the ACM SIGKDD workshop on human computation: ACM, pp. 64-67.
20. Kawrykow, A., Roumanis, G., Kam, A., Kwak, D., Leung, C., Wu, C., Zarour, E., Sarmenta, L., Blanchette, M., and Waldispühl, J. 2012. "Phylo: A Citizen Science Approach for Improving Multiple Sequence Alignment," PloS one (7:3).
21. Kittur, A., Chi, E.H., and Suh, B. 2008. "Crowdsourcing User Studies With Mechanical Turk," ACM Conference on Human Factors in Computing Systems (CHI).
22. Kulkarni, A.P., Can, M., and Hartmann, B. 2011. "Turkomatic: Automatic Recursive Task and Workflow Design for Mechanical Turk," in: CHI 2011.
23. Kuppuswamy, V., and Bayus, B.L. 2013. "Crowdfunding Creative Ideas: The Dynamics of Projects Backers in Kickstarter," SSRN Working Paper 2234765), pp. 1-37.
24. Lakhani, K.R., Jeppesen, L.B., Lohse, P.A., and Panetta, J.A. 2007. "The Value of Openness in Scientific Problem Solving," Harvard Business School Working Paper 07-050).
25. Law, E., Dalton, C., Merrill, N., Young, A., and Gajos, K.Z. 2013. "Curio: A Platform for Supporting Mixed-Expertise Crowdsourcing," AAAI Conference on Human Computation and Crowdsourcing, Palm Springs, CA, USA.
26. Law, E., and von Ahn, L. 2011. "Human Computation," Synthesis Lectures on Artificial Intelligence and Machine Learning (5:3), pp. 1-121.
27. Leimeister, J.M., Huber, M., Bretschneider, U., and Krcmar, H. 2009. "Leveraging Crowdsourcing: Activation-Supporting Components for IT-Based Ideas Competition," Journal of Management Information Systems (26:1), pp. 197-224.
28. Little, G. 2009. "TurKit: Tools for Iterative Tasks on Mechanical Turk," IEEE Symposium on Visual Languages and Human-Centric Computing (IEEE VL/HCC), pp. 252-253.
29. Little, G., Chilton, L.B., Goldman, M., and Miller, R.C. 2010. "TurKit: Human Computation Algorithms on Mechanical Turk," ACM Symposium on User interface Software and Technology, New York, NY, USA.
30. Majchrzak, A., and Malhotra, A. forthcoming. "Towards an Information Systems Perspective and Research Agenda on Crowdsourcing for Innovation," Journal of Strategic Information Systems (n/a), pp. 1-12.
31. Marge, M., Banerjee, S., and Rudnicky, A.I. 2010. "Using the Amazon Mechanical Turk for Transcription of Spoken Language," IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP) 2010.
32. Mason, W., and Suri, S. 2011. "Conducting Behavioral Research on Amazon's Mechanical Turk," Behaviour Research Methods (44:1), pp. 1-23.
33. Mason, W., and Watts, D.J. 2010. "Financial Incentives and the Performance of Crowds," ACM SIGKDD Explorations Newsletter (11:2), pp. 100-108.
34. Nathan R. Prestopnik, K.C. 2011. "Gaming for (Citizen) Science: Exploring Motivation and Data Quality in the Context of Crowdsourced Science through the Design and Evaluation of a Social-Computational System," IEEE Seventh International Conference on e-Science Workshops (Conference).
35. Nielsen, M. 2011. Reinventing discovery: The new era of networked science. Princeton University Press.
36. O'hEocha, C., Wang, X., and Conboy, K. 2012. "The Use of Focus Groups in Complex and Pressurised IS Studies and Evaluation Using Klein & Myers' Principles for Interpretive Research," Information Systems Journal (22:3), pp. 235-256.
37. Paolacci, G., Chandler, J., and Ipeirotis, P.G. 2010. "Running Experiments on Amazon Mechanical Turk," Judgment and Decision Making (5:5), pp. 411-419.
38. Pedersen, J., Kocsis, D., Tripathi, A., Tarrell, A., Weerakoon, A., Tahmasbi, N., Xiong, J., Deng, W., Oh, O., and de Vreede, G.-J. 2013. "Conceptual Foundations of Crowdsourcing: A Review of IS Research," Hawaii International Conference on System Sciences (HICSS), pp. 579-588.
39. Peffers, K., Tuunanen, T., Rothenberger, M.A., and Chatterjee, S. 2007. "A Design Science Research Methodology for Information Systems Research," Journal Of Management Information Systems (24:3), pp. 45-77.
40. Quinn, A.J., and Bederson, B.B. 2011. "Human Computation: A Survey and Taxonomy of a Growing Field," ACM Conference on Human Factors in Computing Systems (CHI), Vancouver, Canada, pp. 1403-1412.
41. Raddick, M.J., Bracey, G., Gay, P.L., Lintott, C.J., Murray, P., Schawinski, K., Szalay, A.S., and Vandenberg, J. 2010. "Galaxy zoo: Exploring the motivations of citizen science volunteers," Astronomy Education Review (9:1), p. 010103.
42. Saxton, G.D., Oh, O., and Kishore, R. 2013. "Rules of crowdsourcing: Models, issues, and systems of control," Information Systems Management (30:1), pp. 2-20.
43. Steelman, Z.R., Hammer, B.I., and Limayem, M. 2014. "Data Collection in the Digital Age: Innovative Alternatives to Student Samples," (38:2), pp. 355-A320.
44. Stewart, D.W., Shamdasani, P.N., and Rook, D.W. 2006. Focus Groups: Theory and Practice (2e). Thousand Oaks, CA, USA: Sage Publications.
45. Tremblay, M.C., Hevner, A.R., and Berndt, D.J. 2010. "Focus Groups for Artefact Refinement and Evaluation in Design Research," Communications of the Association for Information Systems (26:1), pp. 599-618.
46. van Aken, J.E. 2004. "Management Research Based on the Paradigm of the Design Sciences: The Quest for Field-tested and Grounded Technological Rules," Journal of Management Studies (41:2), pp. 219-246.
47. van de Ven, A.H. 2007. Engaged Scholarship: A Guide for Organizational and Social Research, (1 ed.). Oxford University Press (United Kingdom):
48. Wiggins, A., and Crowston, K. 2011. "From Conservation to Crowdsourcing: A Typology of Citizen Science," Hawaii International Conference on System Sciences (HICSS).
49. Zheng, H., Li, D., and Hou, W. 2011. "Task Design, Motivation, and Participation in Crowdsourcing Contests," International Journal of Electronic Commerce (15:4), pp. 57-88.