Two's company, three's a crowd: A case study of crowdsourcing software development

Proceedings - International Conference on Software Engineering

Volumn , Issue 1, 2014, Pages 187-198

  Stol, Klaas Jan ^a   Fitzgerald, Brian ^a  

^a UNIVERSITY OF LIMERICK   (Ireland)

Author keywords

case study; challenges; Crowdsourcing; software development

Indexed keywords

CROWDSOURCING; PROGRAM DEBUGGING; QUERY PROCESSING; SOFTWARE ENGINEERING;

AGILE APPROACHES; AMAZON MECHANICAL TURKS; CHALLENGES; COMPLEX INDUSTRIES; DEVELOPMENT COSTS; DEVELOPMENT PROCESS; INDUSTRY CASE STUDIES; MULTI-NATIONAL CORPORATIONS;

SOFTWARE DESIGN;

EID: 84994141774     PISSN: 02705257     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2568225.2568249     Document Type: Conference Paper

References (87)

1. Ågerfalk, P. J. and Fitzgerald, B. 2008. Outsourcing to an unknown worforce: Exploring opensourcing as a global sourcing strategy, MIS Quarterly, 32, 2. 2] AppStori. http: //www. appstori. com.
2. Archak, N. 2010. Money, Glory and Cheap Talk: Analyzing Strategic Behavior of Contestants in Simultaneous Crowdsourcing Contests on TopCoder. com. Proc. WWW.
3. Aurum, A., Jeffery, R., Wohlin, C. and Handzic, M. 2003. Managing Software Engineering Knowledge, Springer.
4. Baddoo, N. and Hall, T. 2002. Motivators of Software Process Improvement: an analysis of practitioners'views, J Syst Softw, 62, 2, 85-96.
5. Beecham, S., Baddoo, N., Hall, T., Robinson, H. and Sharp, H. 2008. Motivation in Software Engineering: A systematic literature review, Inform Software Tech, 50, 9-10.
6. Begel, A., Bosch, J. and Storey, M. A. 2013. Social Networking Meets Software Development: Perspectives from GitHub, MSDN, Stack Exchange, and TopCoder, IEEE Software, 30, 1.
7. Begel, A., Herbsleb, J. D. and Storey, M.-A. 2012. The Future of Collaborative Software Development. Proc. Computer Supported Cooperative Work.
8. Bjørnson, F. O. and Dingsøyr, T. 2008. Knowledge management in software engineering: A systematic review of studied concepts, findings and research methods used, Inform Software Tech, 50, 11.
9. Boehm, B. 2006. A View of 20th and 21st Century Software Engineering. Proc. International Conference on Software Engineering. Shanghai, China. ACM. 12-29.
10. Boehm, B. W. 1981. Software Engineering Economics, Pearson Education.
11. Bonabeau, E. 2009. Decisions 2. 0: The Power of Collective Intelligence, MIT Sloan Manage Rev, 50, 2, 45-52.
12. Boudreau, K. J., Lacetera, N. and Lakhani, K. R. 2011. Incentives and Problem Uncertainty in Innovation Contests: An Empirical Analysis, Management Science, 57, 5.
13. Brabham, D. C. 2008. Crowdsourcing as a Model for Problem Solving: An Introduction and Cases, Convergence, 14, 1.
14. Brabham, D. C. 2012. The Myth of Amateur Crowds: A critical discourse analysis of crowdsourcing coverage, Information, Communication & Society, 15, 3.
15. Brabham, D. C. 2013. Crowdsourcing, MIT Press.
16. Brooks, F. P. 1995. The mythical man-month: essays on software engineering, Addison-Wesley.
17. Chanal, V. and Caron-Fasan, M. L. 2010. The Difficulties involved in Developing Business Models open to Innovation Communities: the Case of a Crowdsourcing Platform, M@n@gement, 13, 4.
18. Chandler, D. and Kapelner, A. 2013. Breaking monotony with meaning: Motivation in crowdsourcing markets, Journal of Economic Behavior & Organization, 90, 123-133.
19. Dabbish, L., Farzan, R., Kraut, R. and Postmes, T. 2012. Fresh Faces in the Crowd: Turnover, Identity, and Commitment in Online Groups. Proc. CSCW. ACM.
20. Desouza, K. C. and Evaristo, J. R. 2004. Managing Knowledge in Distributed Projects, Commun. ACM, 47, 4.
21. DiPalantino, D. and Vojnovic, M. 2009. Crowdsourcing and all-pay auctions. Proc. 10th ACM Conf. Electronic Commerce.
22. Doan, A., Ramakrishnan, R. and Halevy, A. Y. 2011. Crowdsourcing systems on the World-Wide Web, Commun. ACM, 54, 4.
23. Dow, S. P., Kulkarni, A., Klemmer, S. R. and Hartmann, B. 2012. Shepherding the Crowd Yields Better Work. Proc. Computer-Supported Cooperative Work. ACM.
24. Ebner, W., Leimeister, M., Bretschneider, U. and Krcmar, H. 2008. Leveraging the Wisdom of Crowds: Designing an ITsupported Ideas Competition for an ERP Software Company. Proc. 41st Hawaii International Conference System Sciences.
25. Erickson, L. B. 2012. Leveraging the Crowd as a Source of Innovation: Does Crowdsourcing Represent a New Model for Product and Service Innovation? Proc. SIGMIS Computers and People Research. ACM.
26. Erickson, L. B., Petrick, I. and Trauth, E. M. 2012. Organizational Uses of the Crowd: Developing a Framework for the Study of Crowdsourcing. Proc. SIGMIS-CPR.
27. Estellés-Arolas, E. and González-Ladrón-de-Guevara, F. 2012. Towards an Integrated Crowdsourcing Definition, Journal of Information Science, 38, 2.
28. Faridani, S., Hartmann, B. and Ipeirotis, P. G. 2011. What's the Right Price? Pricing Tasks for Finishing on Time. Proc. AAAI Workshop on Human Computation.
29. Feller, J., Finnegan, P., Fitzgerald, B. and Hayes, J. 2008. From Peer Production to Productization: A Study of Socially Enabled Business Exchanges in Open Source Service Networks, Inform Syst Res, 19, 4.
30. Feller, J. and Fitzgerald, B. 2002. Understanding Open Source Software Development, Pearson Education Ltd.
31. Fitzgerald, B. 2006. The Transformation of Open Source Software, MIS Quarterly, 30, 3.
32. Fitzgerald, B. 2012. Software Crisis 2. 0, IEEE Comput., 45.
33. Fitzgerald, B., Stol, K., O'Sullivan, R. and O'Brien, D. 2013. Scaling Agile Methods to Regulated Environments: An Industry Case Study. Proc. 35th International Conference on Software Engineering. San Francisco, CA, USA. IEEE.
34. Greengard, S. 2011. Following the Crowd, Commun. ACM, 54, 2, 20-22.
35. Herbsleb, J. D. and Grinter, R. E. 1999. Splitting the Organization and Integrating the Code: Conway's Law Revisited. Proc. 21st Int'l Conf Software Engineering.
36. Herbsleb, J. D. and Mockus, A. 2003. An Empirical Study of Speed and Communication in Globally Distributed Software Development, IEEE Trans Softw Eng, 29, 6.
37. Hetmank, L. 2013. Components and Functions of Crowdsourcing Systems-A Systematic Literature Review. Proc. 11th Int'l Conf. Wirtschaftsinformatik.
38. Hoffmann, L. 2009. Crowd Control, Commun. ACM, 52, 3.
39. Horton, J. J. and Chilton, L. B. 2010. The Labor Economics of Paid Crowdsourcing. Proc. Conf. Electronic Commerce.
40. Howe, J. http: //www. crowdsourcing. com.
41. Howe, J. 2006. The Rise of Crowdsourcing, Wired, 14.
42. Howe, J. 2008. Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business, Crown Business.
43. Ipeirotis, P. G. 2010. Analyzing the Amazon Mechanical Turk marketplace, XRDS, 17, 2, 16-21.
44. Ipeirotis, P. G. and Paritosh, P. K. 2011. Managing Crowdsourced Human Computation. WWW.
45. Jouret, G. 2009. Inside Cisco's Search for the Next Big Idea, Harvard Business Review, 87, 9, 43-45.
46. Kaufmann, N., Schulze, T. and Veit, D. 2011. More than fun and money. Worker Motivation in Crowdsourcing-A Study on Mechanical Turk. Proc. 17th AMCIS.
47. Kazman, R. and Chen, H.-M. 2009. The Metropolis Model: A new Logic for Development of crowdsourced systems, Commun. ACM, 52, 7.
48. Kinnaird, P., Dabbish, L., Kiesler, S. and Faste, H. 2013. Co-Worker Transparency in a Microtask Marketplace. Proc. Computer Supported Coordination Work.
49. Kittur, A. 2010. Crowdsourcing, Collaboration and Creativity, XRDS, 17, 2.
50. Kittur, A., Nickerson, J. V., Bernstein, M. S., Gerber, E. M., Shaw, A., Zimmerman, J., Lease, M. and Horton, J. J. 2013. The Future of Crowd Work. Proc. CSCW. ACM.
51. Kittur, A., Smus, B., Khamkar, S. and Kraut, R. E. 2011. CrowdForge: Crowdsourcing Complex Work. Proc. ACM Symposium on User Interface Software and Technology.
52. Kraut, R. E. and Streeter, L. A. 1995. Coordination in Software Development, Commun. ACM, 38, 3.
53. Kulkarni, A., Can, M. and Hartmann, B. 2012. Collaboratively Crowdsourcing Workflows with Turkomatic. Proc. Computer-Supported Cooperative Work.
54. Lakhani, K. R., Garvin, D. A. and Lonstein, E. 2010. TopCoder (A): Developing Software through Crowdsourcing, Harvard Business School 610-032.
55. Lakhani, K. R. and Panetta, J. A. 2007. The Principles of Distributed Innovation, Innovations: Technology, Governance, Globalization, 2, 3.
56. LaToza, T. D., Towne, W. B., van der Hoek, A. and Herbsleb, J. D. 2013. Crowd Development. Proc. 6th CHASE Workshop. San Francisco, CA, USA. IEEE.
57. Malone, T. W. and Crowston, K. 1994. The Interdisciplinary Study of Coordination, ACM Comput Surv, 26, 1.
58. Mao, K., Yang, Y., Li, M. and Harman, M. 2013. Pricing Crowdsourcing-Based Software Development Tasks. Proc. 35th International Conference on Software Engineering.
59. Mason, W. and Watts, D. J. 2009. Financial Incentives and the Performance of Crowds'. Proc. KDD-HCOMP. ACM.
60. Mockus, A., Fielding, R. and Herbsleb, J. D. 2000. A Case Study of Open Source Software Development. Proc. ICSE.
61. Musson, R., Richards, J., Fisher, D., Bird, C., Bussone, B. and Ganguly, S. 2013. Leveraging the crowd: how 48, 000 users helped improve Lync performance, IEEE Softw., 30, 4.
62. Naparat, D. and Finnegan, P. 2013. Crowdsourcing Software Requirements and Development: A Mechanism-based Exploration of Opensourcing'. Proc. 19th AMCIS.
63. Parnas, D. L. 1972. On the criteria to be used in decomposing systems into modules, Commmun. ACM, 15, 12.
64. Pilz, D. and Gewald, H. 2013. Does Money Matter? Motivational Factors for Participation in Paid-and Non-Profit-Crowdsourcing Communities. Proc. 11th Int'l Conf. Wirtschaftsinformatik.
65. Raymond, E. S. 2001. The Cathedral & the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary, O'Reilly Media.
66. Rouse, A. C. 2010. A Preliminary Taxonomy of Crowdsourcing. Proc. Australasian Conference on Information Systems (ACIS).
67. Runeson, P., Höst, M., Rainer, A. and Regnell, B. 2012. Case Study Research in Software Engineering: Guidelines and Examples, Wiley.
68. Savage, N. 2012. Gaining Wisdom from Crowds, Commun. ACM, 55, 3, 13-15.
69. Schenk, E. and Guittard, C. 2009. Crowdsourcing: What can be outsourced to the crowd, and why?
70. Schenk, E. and Guittard, C. 2011. Towards a Characterization of Crowdsourcing Practices, Journal of Innovation Economics, 1, 7.
71. Schwarz, A., Mehta, M., Johnson, N. and Chin, W. W. 2007. Understanding Frameworks and Reviews: A Commentary to Assist us in Moving Our Field Forward by Analyzing Our Past, Database Adv Inform Syst, 38, 3.
72. Seaman, C. 1999. Qualitative Methods in Empirical Studies of Software Engineering, IEEE Trans Softw Eng, 24, 4.
73. Simula, H. 2013. The Rise and Fall of Crowdsourcing? 46th Hawaii International Conference on System Sciences.
74. Singer, Y. and Mittal, M. 2013. Pricing Mechanisms for Crowdsourcing Markets. Proc. WWW.
75. Stol, K. and Fitzgerald, B. 2014. Research Protocol for a Case Study of Crowdsourcing Software Development. http: //staff. lero. ie/stol/publications, University of Limerick.
76. Surowiecki, J. 2005. The Wisdom of Crowds: Why the Many Are Smarter Than the Few, Abacus.
77. Tajedin, H. and Nevo, D. 2013. Determinants of success in crowdsourcing software development. Proc. SIGMIS Computer and People Research. Cincinnati, OH, USA.
78. Tamburri, D., Lago, P. and van Vliet, H. 2013. Organizational social structures for software engineering, ACM Comput Surveys, 46, 1.
79. TopCoder. http: //www. topcoder. com.
80. TopCoder. http: //www. topcoder. com/whatiseoi/.
81. TopCoder. 2010. 10 Burning Questions on Crowdsourcing: Your starting guide to open innovation and crowdsourcing success. http: //www. topcoder. com/blog/10-burningquestions-on-crowdsourcing-and-open-innovation.
82. Tung, Y.-H. and Tsenga, S.-S. 2013. A novel aproach to collaborative testing in a crowdsourcing environment, J Syst Softw, 86, 8.
83. uTest. http: //www. utest. com.
84. Vukovic, M. 2009. Crowdsourcing for Enterprises. SERVICES.
85. Wolfson, S. M. and Lease, M. 2011. Look Before You Leap: Legal Pitfalls of Crowdsourcing. ASIST Annual Meeting.
86. Yin, R. K. 2003. Case Study Research, 3rd ed., SAGE.
87. Zhao, Y. and Zhu, Q. 2012. Evaluation on crowdsourcing research: Current status and future direction, Inf Syst Front, April.