Structuring, aggregating, and evaluating crowdsourced design critique

CSCW 2015 - Proceedings of the 2015 ACM International Conference on Computer-Supported Cooperative Work and Social Computing

Volumn , Issue , 2015, Pages 473-485

  Luther, Kurt ^a   Tolentino, Jari Lee ^b   Wu, Wei ^c   Pavel, Amy ^c   Bailey, Brian P ^d   Agrawala, Maneesh ^c   Hartmann, Björn ^c   Dow, Steven P ^a  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

^d UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

Author keywords

Design; critique; feedback; social computing; crowdsourcing

Indexed keywords

COMPUTER SUPPORTED COOPERATIVE WORK; GROUPWARE; INTERACTIVE COMPUTER SYSTEMS;

DESIGN CRITIQUES; DESIGN PROCESS; GAINING ACCESS; HIGH QUALITY; SOCIAL COMPUTING; WEB-BASED SYSTEM;

DESIGN;

EID: 84968747313     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2675133.2675283     Document Type: Conference Paper

References (33)

1. 99designs. http://www.99designs.com/.
2. Barrett, T. A comparison of the goals of studio professors conducting critiques and art education goals for teaching criticism. Studies in art education (1988), 22-27.
3. Bransford, J. D., Brown, A. L., and Cocking, R. R., Eds. How people learn: brain, mind, experience, and school. National Academy Press, Washington, D.C., 2000.
4. Dannels, D., Gaffney, A., and Martin, K. Beyond content, deeper than delivery: What critique feedback reveals about communication expectations in design education. Int. J. Schol. Teach. & Learn. 2, 2 (2008).
5. Dannels, D. P., and Martin, K. N. Critiquing critiques a genre analysis of feedback across novice to expert design studios. Jo. Bus. & Tech. Comm. 22, 2 (2008), 135-159.
6. Dondis, D. A. A primer of visual literacy. MIT Press, Cambridge, Mass., 1973.
7. Dow, S., Fortuna, J., Schwartz, D., Altringer, B., Schwartz, D., and Klemmer, S. Prototyping dynamics: sharing multiple designs improves exploration, group rapport, and results. In Proc. CHI 2011 (2011), 2807-2816.
8. Dow, S., Gerber, E., and Wong, A. A pilot study of using crowds in the classroom. In Proc. CHI 2013, CHI 13 (New York, NY, USA, 2013), 227-236.
9. Dow, S. P., Glassco, A., Kass, J., Schwarz, M., Schwartz, D. L., and Klemmer, S. R. Parallel prototyping leads to better design results, more divergence, and increased self-efficacy. ACM Trans. Comput.-Hum. Interact. 17, 4 (Dec. 2010), 18:1-18:24.
10. Easterday, M. W., Rees Lewis, D., Fitzpatrick, C., and Gerber, E. M. Computer supported novice group critique. In Proc. of DIS 14 (2014).
11. Feedback army. http://www.feedbackarmy.com/.
12. Feldman, E. Practical Art criticism. Prentice Hall, Englewood Cliffs, NJ, 1994.
13. Five second test. http://www.fivesecondtest.com/.
14. Forrst. http://www.forrst.com/.
15. Klemmer, S. R., Hartmann, B., and Takayama, L. How bodies matter: five themes for interaction design. In Proceedings of DIS, ACM (2006), 140-149.
16. Kulkarni, C., and Klemmer, S. R. Learning design wisdom by augmenting physical studio critique with online self-Assessment. Tech. rep., Stanford U., 2012.
17. Lidwell, W., Holden, K., and Butler, J. Universal principles of design. Rockport Pub, 2010.
18. Marlow, J., and Dabbish, L. From rookie to all-star: Professional development in a graphic design community of practice. In Proc. CSCW 2014 (2014).
19. Nichols, A. L., and Maner, J. K. The good-subject effect: investigating participant demand characteristics. The Journal of general psychology 135, 2 (Apr. 2008), 151-165
20. Nielsen, J., and Molich, R. Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 90, ACM (New York, NY, USA, 1990), 249-256.
21. Reynolds, G. Presentation Zen: Simple ideas on presentation design and delivery. New Riders, 2011.
22. Sadler, D. R. Formative assessment and the design of instructional systems. Instr. Sci. 18, 2 (1989), 119-144.
23. Schön, D. Educating the Reflective Practitioner. Jossey-Bass Publishers, San Francisco, 1990.
24. Shneiderman, B. The eyes have it: A task by data type taxonomy for information visualizations. In Proc. VL/HCC 1996 (1996), 336-343.
25. Soloway, E., Guzdial, M., and Hay, K. E. Learner-centered design: The challenge for HCI in the 21st century. interactions 1 (April 1994), 36-48.
26. Tinapple, D., Olson, L., and Sadauskas, J. Critviz: Web-based software supporting peer critique in large creative classrooms. Bulletin of the IEEE Technical Committee on Learning Technology 15, 1 (2013), 29.
27. Tohidi, M., Buxton, W., Baecker, R., and Sellen, A. Getting the right design and the design right. In Proceedings of CHI, ACM (2006), 1243-1252.
28. Wattenberg, M., and Kriss, J. Designing for social data analysis. IEEE Transactions on Visualization and Computer Graphics 12, 4 (2006), 549-557.
29. Willett, W., Heer, J., and Agrawala, M. Strategies for crowdsourcing social data analysis. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 12, ACM (New York, NY, USA, 2012), 227-236.
30. Willett, W., Heer, J., Hellerstein, J., and Agrawala, M. Commentspace: structured support for collaborative visual analysis. In Proc. CHI 2011 (2011), 3131-3140.
31. Williams, R. The Non-Designers design book. Peachpit Press, 2008.
32. Xu, A., and Bailey, B. What do you think?: A case study of benefit, expectation, and interaction in a large online critique community. In Proc. CSCW 2012 (2012), 295-304.
33. Xu, A., Bailey, B. P., and Huang, S.-W. Voyant: Generating structured feedback on visual designs using a crowd of non-experts. In Proc. CSCW 2014 (2014).