Information extraction and manipulation threats in crowd-powered systems

Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW

Volumn , Issue , 2014, Pages 248-256

  Lasecki, Walter S ^a   Teevan, Jaime ^b   Kamar, Ece ^b  

^a University of Rochester   (United States)

^b MICROSOFT RESEARCH   (United States)

Author keywords

Crowdsourcing; Extraction; Manipulation; Privacy; Security

Indexed keywords

AUTOMATION; DATA PRIVACY; EXTRACTION; INTERACTIVE COMPUTER SYSTEMS;

AMAZON'S MECHANICAL TURKS; AUTOMATED SYSTEMS; COORDINATED ATTACK; CROWDSOURCING; EXTRACTING INFORMATION; MANIPULATION; MECHANICAL TURKS; SECURITY;

COMPUTER SUPPORTED COOPERATIVE WORK;

EID: 84898987373     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2531602.2531733     Document Type: Conference Paper

References (30)

1. Bernstein, M.S., Brandt, J.R., Miller, R.C. and Karger, D.R. Crowds in two seconds: Enabling realtime crowdpowered interfaces. In Proceedings of UIST 2011.
2. Bernstein, M.S., Little, G., Miller, R.C., Hartmann, B., Ackerman, M.S., Karger, D.R., Crowell, D. and Panovich, K. Soylent: A word processor with a crowd inside. In Proceedings of UIST 2010.
3. Bigham, J.P., Jayant, C., Ji, H., Little, G., Miller, A., Miller, R.C., Miller, R., Tatarowicz, A., White, B., White, S. and Yeh, T. VizWiz: nearly real-time answers to visual questions. In Proceedings of UIST 2010.
4. Chen, X., Bennett, P.N., Collins-Thompson, K. and Horvitz, E. Pairwise ranking aggregation in a crowdsourced setting. In Proceedings of WSDM 2013.
5. Deng, J., Krause, J., and Fei-Fei, L. Fine-Grained Crowdsourcing for Fine-Grained Recognition. In Proceesings of CVPR 2013.
6. Evanini, K. and Zechner, K. Using crowdsourcing to provide prosodic annotations for non-native speech. In Proceedings of Interspeech 2011.
7. Featured requester: Discover how comScore benefits from mechanical Turk. The Mechanical Turk Blog, March 2013. http://bit.ly/Ye64Sb
8. Harris, Christopher G. Dirty Deeds Done Dirt Cheap: A Darker Side to Crowdsourcing. In SocialCom 2011.
9. Kamar, E., Hacker, S., Lintott, C. and Horvitz, E. Combining human and machine learning intelligence in large-scale crowdsourcing: Principles, methods, and studies. MSR-TR-2012-58, 2012.
10. Kittur, A. and Kraut, R.E. Harnessing the wisdom of crowds in Wikipedia: Quality through coordination. In Proceedings of CSCW 2008.
11. Kittur, A., Nickerson, J.V., Bernstein, M., Gerber, E., Shaw, A., Zimmerman, J., Lease, M., and Horton, J. The future of crowd work. In Proceedings of CSCW 2013.
12. Kokkalis, N., Köhn, T., Pfeiffer, C., Chornyi, D., Bernstein, M.S. and Klemmer, S.R. EmailValet: Managing email overload through private, accountable crowdsourcing. In Proceedings of CSCW 2013.
13. Le, J., Edmonds, A., Hester,V., and Biewald, L. Ensuring quality in crowdsourced search relevance evaluation. In Proceedings of SIGIR 2010 Workshop on Crowdsourcing for Search Evaluation.
14. Little, G., Chilton, L.B., Goldman, M. and Miller, R.C. TurKit: Human computation algorithms on mechanical turk. In Proceedings of UIST 2010.
15. Little, G. and Sun, Y-A. Human OCR. CHI 2011 Workshop on Crowdsourcing and Human Computation.
16. Lasecki, W.S., Miller, C., Sadilek, A., Abumoussa, A., Borrello, D., Kushalnagar, R. and Bigham, J.P. Real-time captioning by groups of non-experts. In Proceedings of UIST 2012.
17. Lasecki, W.S., Murray, K.I., White, S., Miller, R.C. and Bigham, J.P. Real-time crowd control of existing interfaces. In Proceedings of UIST 2011.
18. Lasecki, W.S., Song, Y.C., Kautz, H. and Bigham, J.P. Real-time crowd labeling for deployable activity recognition. In Proceedings CSCW 2013.
19. Lasecki, W.S., Thiha, P., Zhong, Y., Brady, E. and Bigham, J.P. Answering Visual Questions with Conversational Crowd Assistants. In Proceedings of ASSETS 2013.
20. Lasecki, W.S., Wesley, R., Nichols, J., Kulkari, A., Allen, J.F. and Bigham, J.P. Chorus: A Crowd-Powered Personal Assistant. In Proceedings of UIST 2013.
21. Lasecki, W.S., White, S.C., Murray, K.I. and Bigham, J.P. Crowd Memory: Learning in the collective. In Proceedings of Collective Intelligence 2012.
22. Mason, W. and Watts, D.J. Financial incentives and the performance of crowds. In Proceedings of HComp 2009.
23. Massively multiplayer pong. http://collisiondetection.net, 2006.
24. Menton, C. and Singh, P. Manipulation can be hard in tractable voting systems even for constant-sized coalitions. CoRR abs/1108.4439.
25. Sun, Y-A. Roy, S. and Little, G. Beyond independent agreement: A tournament selection approach for quality assurance of human computation tasks. In HComp 2011.
26. Sweeney, L. Uniqueness of simple demographics in the U.S. population. In LIDAP-WP4, CMU. 2000.
27. Quinn, A.J. and Bederson, B.B. Human computation: A survey and taxonomy of a growing field. In Proceedings of CHI 2011.
28. Van Boskrik, S. US interactive marketing forecast, 2011 to 2016. Forrester. August 2011.
29. Von Ahn, L. Games with a purpose. Comp. 39(6), 2006.
30. Zaidan, O.F. and Callison-Burch, C. Crowdsourcing translation: professional quality from non-professionals. In Proceedings of ACL-HLT 2011