Voyant: Generating structured feedback on visual designs using a crowd of non-experts

Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW

Volumn , Issue , 2014, Pages 1433-1444

  Xu, Anbang ^a   Huang, Shih Wen ^a   Bailey, Brian P ^a  

^a University of Illinois at Urbana Champaign   (United States)

Author keywords

Creativity; Critique; Crowdsourcing; Design; Feedback

Indexed keywords

COMPUTER SUPPORTED COOPERATIVE WORK; FEEDBACK; INTERACTIVE COMPUTER SYSTEMS;

COORDINATED VIEWS; CREATIVITY; CRITIQUE; CROWDSOURCING; EFFECTIVE SOLUTION; FEEDBACK TYPE; VISUAL DESIGN; VISUAL ELEMENTS;

DESIGN;

EID: 84898948908     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2531602.2531604     Document Type: Conference Paper

References (45)

1. Baldonado, M.Q.W., Woodruff, A., &Kuchinsky, A., Guidelines for Using Multiple Views in Information Visualization. In AVI, (2000), 110-119.
2. Bernstein, M.S., Brandt, J., Miller, R.C., &Karger, D.R., Crowds in Two Seconds: Enabling Realtime Crowd-Powered Interfaces. In UIST, (2011), 33-42.
3. Bernstein, M.S., Little, G., Miller, R.C., Hartmann, B., Ackerman, M.S., Karger, D.R., Crowell, D., et al., Soylent: A Word Processor with a Crowd Inside. In UIST, (2010), 313-322.
4. Cross, N. Designerly Ways of Knowing. Design Studies, 3, 5, (1982), 221-227.
5. Dannels, D.P. and Martin, K.N. Critiquing Critiques: A Genre Analysis of Feedback Across Novice to Expert Design Studios. Journal of Business and Technical Communication, 22, 2, (2008), 135-159.
6. Diehl, M. and Stroebe, W. Productivity Loss in Brainstorming Groups: Towards the Solution of a Riddle. Journal of Personality and Social Psychology, 53, 3, (1987), 497-509.
7. Dow, S.P., Fortuna, J., Schwartz, D., Altringer, B., &Klemmer, S., Prototyping Dynamics: Sharing Multiple Designs Improves Exploration, Group Rapport, and Results. In CHI, (2011), 2807-2816.
8. Dow, S.P., Gerber, E., &Wong, A., A Preliminary Study of Using Crowds in the Classroom. In CHI, (2013), 227-236.
9. Dow, S.P., Heddleston, K., &Klemmer, S.R., The Efficacy of Prototyping Under Time Constraints. In Creativity &Cognition, (2009), 165-174.
10. Dow, S.P., Kulkarni, A., Klemmer, S., &Hartmann, B., Shepherding the Crowd Yields Better Work. In CSCW, (2012), 1013-1022.
11. Dunne, C., Riche, N.H., Lee, B., Metoyer, R.A., &Robertson, G.G., GraphTrail: Analyzing Large Multivariate, Heterogeneous Networks while Supporting Exploration History. In CHI, (2012), 1663-1672.
12. Elkins, J. Art Critiques: A Guide. New Academia Publishing, Washington DC, 2012.
13. Feldman, E.B. The Teacher as Model Critic. Journal of Aesthetic Education, 7, 1, (1973), 50-57.
14. Feldman, E.B. Varieties of Visual Experience: Art as Image and Idea. Prentice-Hall, Englewood Cliffs, N.J., 1981.
15. Fischer, G., Nakakoji, K., Ostwald, J., Stahl, G., &Sumner, T., Embedding Computer-Based Critics in the Contexts of Design. In CHI, (1993), 157-164.
16. Fogarty, J., Hudson, S.E., Atkeson, C.G., Avrahami, D., Forlizzi, J., Kiesler, S., Lee, J.C., et al. Predicting Human Interruptibility with Sensors. TOCHI, 12, 1, (2005), 119-146.
17. Heer, J., Viégas, F.B., &Wattenberg, M., Voyagers and Voyeurs: Supporting Asynchronous Collaborative Information Visualization. In CHI, (2007), 1029-1038.
18. Hundhausen, C.D., Fairbrother, D., &Petre, M. An Empirical Study of the "Prototype Walkthrough": A Studio-Based Activity for HCI Education. TOCHI, 19, 4, (2012), 1-36.
19. Kittur, A., Chi, E.H., &Suh, B., Crowdsourcing User Studies With Mechanical Turk. In CHI, (2008), 453-456.
20. Kittur, A., Nickerson, J.V., Bernstein, M.S., Gerber, E.M., Shaw, A., Zimmerman, J., Lease, M., et al., The Future of Crowd Work. In CSCW, (2013), 1301-1318.
21. Komarov, S., Reinecke, K., &Gajos, K.Z., Crowdsourcing Performance Evaluations of User Interfaces. In CHI, (2013), 207-216.
22. Kress, G.R. and Leeuwen, T.v. Reading Images: The Grammar of Visual Design. Routledge, 1996.
23. Kulkarni, C. and Klemmer, S. Learning Design Wisdom by Augmenting Physical Studio Critique With Online Self-Assessment, Stanford University technical report, 2012.
24. Nijstad, B.A. and Stroebe, W. How the Group Affects the Mind: A Cognitive Model of Idea Generation in Groups. Personality and Social Psychology Review, 10, 3, (2006), 186-213.
25. Park, C.H., Son, K., Lee, J.H., &Bae, S.-H., Crowd vs. Crowd: Large-Scale Cooperative Design through Open Team Competition. In CSCW, (2013), 1275-1284.
26. Paulus, P. and Yang, H.-C. Idea Generation in Groups: A Basis for Creativity in Organizations. Organizational Behavior and Human Decision Processes, 82, 1, (2000), 76-87.
27. Reinecke, K., Yeh, T., Miratrix, L., Mardiko, R., Zhao, Y., Liu, J., &Gajos, K.Z., Predicting Users' First Impressions of Website Aesthetics With a Quantification of Perceived Visual Complexity and Colorfulness. In CHI, (2013), 2049-2058.
28. Rosenholtz, R., Twarog, N.R., Schinkel-Bielefeld, N., &Wattenberg, M., An Intuitive Model of Perceptual Grouping for HCI Design. In CHI, (2009), 1331-1340.
29. Schön, D.A. Designing as Reflective Conversation with the Materials of a Design Situation. Knowledge-Based Systems, 5, 1, (1992), 3-14.
30. Strauss, A.L. Qualitative Analysis for Social Scientists. Cambridge University Press, 1987.
31. Tohidi, M., Buxton, W., Baecker, R., &Sellen, A., Getting the Right Design and the Design Right. In CHI, (2006), 1243-1252.
32. Wang, H.C., Fussell, S.R., &Cosley, D., From Diversity to Creativity: Stimulating Group Brainstorming with Cultural Differences and Conversationally-Retrieved Pictures. In CSCW, (2011), 265-274.
33. Willett, W., Heer, J., &Agrawala, M., Strategies for Crowdsourcing Social Data Analysis. In CHI, (2012), 227-236.
34. Williams, R. The Non-designer's Design Book. Peachpit Press, 2008.
35. Xu, A. and Bailey, B.P., A Crowdsourcing Model for Receiving Design Critique. In CHI Extended Abstracts, (2011), 1183-1188.
36. Xu, A. and Bailey, B.P., A Reference-Based Scoring Model for Increasing the Findability of Promising Ideas in Innovation Pipelines. In CSCW, (2012), 1183-1186.
37. Xu, A. and Bailey, B.P., What Do You Think? A Case Study of Benefit, Expectation, and Interaction in a Large Online Critique Community. In CSCW, (2012), 295-304.
38. Yu, L. and Nickerson, J.V., Cooks or Cobblers? Crowd Creativity through Combination. In CHI, (2011), 1393-1402.
39. Zheng, X.S., Chakraborty, I., Lin, J.J., &Rauschenberger, R., Correlating Low-level Image Statistics with Users' Rapid Aesthetic and Affective Judgments of Web Pages. In CHI, (2009), 1-10.
40. Does This JC Penney Tea Kettle Look Like Hitler?, NBC's Today.com, http: //www.today.com/news/does-j-c-penney-teakettle-look-hitler-6C10100642, Retrieved on May 28 2013.
41. Core77. http: //www.core77.com.
42. Dribbble. http: //dribbble.com.
43. Feedbackarmy. http: //www.feedbackarmy.com.
44. Fivesecondtest. http: //fivesecondtest.com.
45. Usabilla. http: //www.usabilla.com.