An evaluation framework for software crowdsourcing

Frontiers of Computer Science

Volumn 7, Issue 5, 2013, Pages 694-709

  Wu, Wenjun ^a   Tsai, Wei Tek ^b,c   Li, Wei ^a  

^a BEIHANG UNIVERSITY   (China)

^b ARIZONA STATE UNIVERSITY   (United States)

^c TSINGHUA UNIVERSITY   (China)

Author keywords

competition rules; crowdsourcing; game theory; software engineering

Indexed keywords

COMPUTER SOFTWARE SELECTION AND EVALUATION; CROWDSOURCING; FUNCTION EVALUATION; GAME THEORY; SOFTWARE ENGINEERING; SOFTWARE QUALITY;

COMPETITION RULES; DIVERSITY OF SOLUTIONS; EVALUATION FRAMEWORK; GAME THEORY MODELS; OBJECTIVE FUNCTIONS; PRIMARY FACTORS; SHARP CONTRAST; SYSTEMATIC ANALYSIS;

QUALITY CONTROL;

EID: 84884904054     PISSN: 20952228     EISSN: 20952236     Source Type: Journal    
DOI: 10.1007/s11704-013-2320-2     Document Type: Article

References (28)

1. Doan A, Ramakrishnan R, Halevy A Y. Crowdsourcing systems on the World-Wide Web. Communications of the ACM, 2011, 54(4): 86-96.
2. Lakhani K, Garvin D, Lonstein E. Topcoder (a): developing software through crowdsourcing. Harvard Business School General Management Unit Case, 2010. Available at SSRN: http://ssrn. com/abstract=2002884.
3. uTest. https://www. utest. com/.
4. Bosch J. From software product lines to software ecosystems. In: Proceedings of the 13th International Software Product Line Conference. 2009, 111-119.
5. Jansen S, Finkelstein A, Brinkkemper S. A sense of community: a research agenda for software ecosystems. In: Proceedings of the 31st International Conference on Software Engineering-Companion Volume. 2009, 187-190.
6. Apple Store Metrics. http://148apps. biz/app-store-metrics/, 2012.
7. AppStori. http://appstori. com/, 2012.
8. Kittur A. Crowdsourcing, collaboration and creativity. XRDS, 2010, 17(2): 22-26.
9. Constantinescu R, Iacob I M. Capability maturity model integration. Journal of Applied Quantitative Methods, 2007, 2(1): 187.
10. Atwood M. Military standard: defense system software development. Department of Defense, USA, 1988.
11. Schenk E, Guittard C. Crowdsourcing: what can be outsourced to the crowd, and why. In: Workshop on Open Source Innovation, Strasbourg, France. 2009.
12. Tong R, Lakhani K. Public-private partnerships for organizing and executing prize-based competitions. Berkman Center Research Publication, 2012. Available at SSRN: http://ssrn. com/abstract=2083755.
13. Algorithm Development Through Crowdsourcing. http://catalyst. harvard. edu/services/crowdsourcing/, 2012.
14. Archak N, Sundararajan A. Optimal design of crowdsourcing contests. In: Proceedings of the 30th International Conference on Information Systems. 2009, 1-16.
15. Wu T W, Li W. Creative software crowdsourcing. Creative Software Crowdsourcing: From Components and Algorithm Development to Project Concept Formations, 2013.
16. Baldwin C Y, Clark K B. The architecture of participation: does code architecture mitigate free riding in the open source development model? Management Science, 2006, 52(7): 1116-1127.
17. Rand D G, Dreber A, Ellingsen T, Fudenberg D, Nowak MA. Positive interactions promote public cooperation. Science, 2009, 325(5945): 1272-1275.
18. Herbrich R, Minka T, Graepel T. TrueSkill™: a bayesian skill rating system. In: Proceedings of the 2006 Annual Conference of Advances in Neural Information Processing Systems. 2007, 19: 569-576.
19. TopCoder Inc. http://apps. topcoder. com/wiki/display/tc/algorithm+competition+rating+system, 2013.
20. Apple App Store Review Guidelines. https://developer. apple. com/appstore/guidelines. html, 2010.
21. Archak N. Money, glory and cheap talk: analyzing strategic behavior of contestants in simultaneous crowdsourcing contests on topcoder. com. In: Proceedings of the 19th International Conference on World Wide Web. 2010, 21-30.
22. DiPalantino D, Vojnovic M. Crowdsourcing and all-pay auctions. In: Proceedings of the 10th ACM Conference on Electronic Commerce. 2009, 119-128.
23. Horton J J, Chilton L B. The labor economics of paid crowdsourcing. In: Proceedings of the 11th ACM Conference on Electronic Commerce. 2010, 209-218.
24. Bacon D F, Chen Y, Parkes D, Rao M. A market-based approach to software evolution. In: Proceedings of the 24th ACM SIGPLAN Conference Companion on Object Oriented Programming Systems Languages and Applications. 2009, 973-980.
25. Bullinger A C, Moeslein K. Innovation contests-where are we? In: Proceedings of the 16th Americas Conference on Information Systems. 2010.
26. Leimeister J M, Huber M, Bretschneider U, Krcmar H. Leveraging crowdsourcing: activation-supporting components for it-based ideas competition. Journal of Management Information Systems, 2009, 26(1): 197-224.
27. Kazman R, Chen HM. The metropolis model a new logic for development of crowdsourced systems. Communications of the ACM, 2009, 52(7): 76-84.
28. Bratvold D, Armstrong C. http://www. dailycrowdsource. com, 2013.