Strategies for crowdsourcing social data analysis

Conference on Human Factors in Computing Systems - Proceedings

Volumn , Issue , 2012, Pages 227-236

  Willett, Wesley ^a   Heer, Jeffrey ^b   Agrawala, Maneesh ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b Stanford University   (United States)

Author keywords

Crowdsourcing; Information visualization; Social data analysis

Indexed keywords

ALTERNATIVE APPROACH; CROWDSOURCING; DATA ANALYSIS TOOL; DATA SETS; FEATURE-ORIENTED; HIGH QUALITY; INFORMATION VISUALIZATION; SUBTASKS;

INFORMATION SYSTEMS;

HUMAN ENGINEERING;

EID: 84862069681     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2207676.2207709     Document Type: Conference Paper

References (33)

1. Ahmad, S., Battle, A., Malkani, Z., and Kamvar, S. The jabberwocky programming environment for structured social computing. In Proc. UIST (2011).
2. Bernstein, M. S., Little, G., Miller, R. C., Hartmann, B., Ackerman, M. S., Karger, D. R., Crowell, D., and Panovich, K. Soylent: a word processor with a crowd inside. In Proc. UIST (2010), 313-322.
3. Boud, D. Enhancing learning through self assessment. Routledge, 1995.
4. Chandler, D., and Kapelner, A. Breaking monotony with meaning: Motivation in crowdsourcing markets. U. of Chicago mimeo (2010).
5. Danis, C. M., Viegas, F. B., Wattenberg, M., and Kriss, J. Your place or mine?: Visualization as a community component. In Proc. CHI (2008), 275-284.
6. Heer, J., and Agrawala, M. Design considerations for collaborative visual analytics. Information Visualization 7, 1 (2008), 49-62.
7. Heer, J., and Bostock, M. Crowdsourcing graphical perception: Using mechanical turk to assess visualization design. In Proc. CHI (2010), 203-212.
8. Heer, J., Viégas, F., and Wattenberg, M. Voyagers and voyeurs: Supporting asynchronous collaborative visualization. Comm. of the ACM 52, 1 (2009), 87-97.
9. Hill, W. C., and Hollan, J. D. Deixis and the future of visualization excellence. In Proc. of IEEE Visualization (1991), 314-320, 431.
10. Hurley, C. B., and Oldford, R. W. Pairwise Display of High-Dimensional Information via Eulerian Tours and Hamiltonian Decompositions. JCGS 19, 4 (2010), 861-886.
11. Ipeirotis, P. Demographics of mechanical turk. New York University, Tech. Rep (2010).
12. Kittur, A., Chi, E. H., and Suh, B. Crowdsourcing user studies with mechanical turk. In Proc. CHI (2008), 453-456.
13. Kittur, A., Smus, B., and Kraut, R. Crowdforge: crowdsourcing complex work. In CHI Extended Abstracts, ACM (2011), 1801-1806.
14. Kong, N., Heer, J., and Agrawala, M. Perceptual guidelines for creating rectangular treemaps. IEEE TVCG 16 (2010), 990-998.
15. Kulkarni, A., Can, M., and Hartman, B. Collaboratively Crowdsourcing Workflows with Turkomatic. In Proc. CSCW (2012).
16. Lee, E.-K., Cook, D., Klinke, S., and Lumley, T. Projection pursuit for exploratory supervised classification. JCGS 14, 4 (2005), 831-846.
17. Little, G., Chilton, L., Goldman, M., and Miller, R. Turkit: tools for iterative tasks on mechanical turk. In Proc. SIGKDD, ACM (2009).
18. Luther, K., Counts, S., Stecher, K., Hoff, A., and Johns, P. Pathfinder: an online collaboration environment for citizen scientists. In Proc. CHI (2009), 239-248.
19. Mason, W., and Watts, D. J. Financial incentives and the "performance of crowds". SIGKDD Explor. Newsl. 11 (May 2010), 100-108.
20. Nielsen, J. Participation inequality: Encouraging more users to contribute. Jakob Nielsen's Alertbox (2006). http://www.useit.com/alertbox/ participation inequality.html.
21. Oleson, D., Sorokin, A., Laughlin, G., Hester, V., Le, J., and Biewald, L. Programmatic gold: Targeted and scalable quality assurance in crowdsourcing. In Proc. HComp (2011).
22. Pirolli, P., and Card, S. The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. The Analyst 2005, 1-6.
23. Quinn, A. J., and Bederson, B. B. Human Computation: A Survey and Taxonomy of a Growing Field. In Proc. CHI (2011).
24. Russell, D. M., Stefik, M. J., Pirolli, P., and Card, S. K. The cost structure of sensemaking. In Proc. CHI (1993), 269-276.
25. Shaw, A. D., Horton, J. J., and Chen, D. L. Designing incentives for inexpert human raters. In Proc. CSCW (2011), 275-284.
26. Sorokin, A., and Forsyth, D. Utility data annotation with amazon mechanical turk. In Computer Vision and Pattern Recognition Workshops (2008), 1-8.
27. Swivel. http://www.swivel.com.
28. Tableau. http://www.tableausoftware.com.
29. Viégas, F., Wattenberg, M., McKeon, M., Van Ham, F., and Kriss, J. Harry potter and the meat-filled freezer: A case study of spontaneous usage of visualization tools. In Proc. HICSS, Citeseer (2008).
30. Viégas, F., Wattenberg, M., Van Ham, F., Kriss, J., and McKeon, M. Manyeyes: a site for visualization at internet scale. IEEE TVCG 13, 6 (2007), 1121-1128.
31. Whittaker, S., Terveen, L., Hill, W., and Cherny, L. The dynamics of mass interaction. In Proc. CSCW (1998), 257-264.
32. Willett, W., Heer, J., Agrawala, M., and Hellerstein, J. CommentSpace: Structured Support for Collaborative Visual Analysis. In Proc. CHI (2011).
33. Wills, G., and Wilkinson, L. Autovis: automatic visualization. Information Visualization 9 (March 2010), 47-69.