Potential limitations of verbal protocols in design experiments

Proceedings of the ASME Design Engineering Technical Conference

Volumn 5, Issue , 2010, Pages 287-296

  Chiu, I ^a   Shu, L H ^b  

^a RYERSON UNIVERSITY   (Canada)

^b UNIVERSITY OF TORONTO   (Canada)

Author keywords

Design cognition; Design stimulation and creativity; Experimental methodology; Verbal protocols

Indexed keywords

COGNITIVE PROCESS; DESIGN COGNITION; DESIGN STUDIES; EXPERIMENTAL METHODOLOGY; METHODOLOGICAL APPROACH; STUDY DESIGN; VERBAL PROTOCOLS;

EXPERIMENTS; VIBRATIONS (MECHANICAL);

DESIGN;

EID: 80054968434     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1115/DETC2010-28675     Document Type: Conference Paper

References (56)

1. Alexiou, K., Zamenopoulos, T., Johnson, J.H., 2009, Exploring the neurological basis of design cognition using brain imaging: some preliminary results, Design Studies 30:623-647.
2. Altshuller, G.S., Shulyak, L., 1996, And Suddenly the Inventor Appeared: TRIZ, the Theory of Inventive Problem Solving, 2/e, Technical Innovation Center, Worcester, MA.
3. Atman, C.J, Bursic K., 1996, "Teaching Engineering Design: Can Reading a Textbook Make a Difference?" Research in Engineering Design, 7/7:240-250.
4. Atman, C.J., Cardella, M.E., Turns, J., Adams, R., 2004, Comparing freshman and senior engineering design processes: an in-depth follow-up study, Design Studies, 26:325-357.
5. Bainbridge, L., Beishon, J., Hemming, J.H., Splaine, M., 1968, A Study of Real-Time Human Decision-Making Using a Plant Simulator, OR, Special Conference Issue: Decision-Making 19:91-106.
6. Bainbridge, L., 1991, Chapter 7: Verbal Protocol Analysis, in J.R. Wilson and E.N. Corlett's (eds.) Evaluation of Human Work, Taylor & Francis.
7. Benami, O., Jin, Y., 2002, Creative stimulation in conceptual design, ASME DETC/CIE, Montreal, Canada, DETC2002/DTM-34023.
8. Besemer S.P., Treffinger D.J., 1981, Analysis of Creative Products: Review and Synthesis, J. Creative Behavior, 15:158-178.
9. Bhaskar, R., Simon, H. A., 1977, Problem solving in semantically rich domains: An example from engineering thermodynamics, Cognitive Science, 1:193-215.
10. Brereton, M., McGarry, B., 2000, An Observational Study of How Objects Support Engineering Design Thinking and Communication: Implications for the design of tangible media, CHI, 2/1:217-224
11. Brown, D.C., 2008, Guiding Computational Design Creativity Research, in J. Gero's (ed.) Studying Design Creativity, Springer.
12. Charlton, S. G., 1996, Mental Workload Test and Evaluation, in T.G. O'Brien and S.G. Charlton's (eds.) Handbook of Human Factors Testing and Evaluation, Earlbaum.
13. Chiu, I., Shu, L.H., 2008a, Use of opposite-relation lexical stimuli in concept generation, Annals of the CIRP 57/1:149-152.
14. Chiu, I., Shu, L.H., 2008b, Effects of Dichotomous Lexical Stimuli in Concept Generation, ASME IDETC/CIE, New York City, NY, USA, August 3-6, 2008. DETC2008-49372 (DTM).
15. Chiu, I., 2010, Quantifying Oppositely and Similarly Related Semantic Stimuli on Concept Creativity, University of Toronto, Ph.D. Thesis.
16. Cross, N., Christiaans, H., Drost, K., 1996, Introduction: The Delft Protocols Workshop, in N. Cross, H. Christiaans, K. Drost's (eds.) Analysing Design Activity, John Wiley & Sons, West Sussex, UK.
17. Cross, N., 2006, Designerly Ways of Knowing, Springer-Verlag, London, UK.
18. De Bono, E.,1970, Lateral thinking: creativity step by step, Harper & Row.
19. Deubel, P., 2003, An investigation of behaviorist and cognitive approaches to instructional multimedia design, Journal of Educational Multimedia and Hypermedia 12/1:63-90.
20. rd Edition, McGraw-Hill, NY.
21. Dong, A, 2006, Concept formation as knowledge accumulation: a computational linguistics study, Artificial Intelligence for Engineering Design Analysis & Manufacturing, 20/1:35-53.
22. Drews, F. A., Yazdani, H., Godfrey, C.N., Cooper, J.M., Strayner, D.L., 2009, Text Messaging During Simulated Driving, Human Factors: The Journal of the Human Factors and Ergonomics Society, accessed online at http://hfs.sagepub.com on January 17, 2010.
23. Duquette, L., 2009, Statistical Consultant, Department of Statistics, University of Toronto, Personal communication.
24. rd ed., Holt Rinehart & Winston
25. Ericsson, K. A., Simon, H.A., 1993, Protocol Analysis: Verbal Reports as Data, MIT Press, Cambridge, MA.
26. Ericsson K. A., 2002, Protocol analysis and Verbal Reports on Thinking, An updated and extracted version from Ericsson, accessed online at http://www.psy.fsu.edu/faculty/ericsson/ericsson.proto.thnk.html
27. Gero, J.S., McNeill, T., 1998, An approach to the analysis of design protocols, Design Studies, 19:21-61.
28. Gordon, S.E., 1992, Implications of Cognitive Theory for Knowledge Acquisition, in R.R. Hoffman's (ed.) The Psychology of Expertise, Cognitive Research and Empirical AI, Springer-Verlag, New York, NY.
29. Gordon, W. J. J., 1961, Synectics, The Development of Creative Capacity, Harper & Row, New York, NY.
30. Hart, S.G, Staveland, L.E.,1988, Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P.A. Hancock, N. Meshkati (eds.) Human Mental Workload, 139-183. Elsevier Science: NorthHolland
31. Hubka, V., Eder W. E., 1996, Design Science-Introduction to the Needs, Scope and Organization of Engineering Design Knowledge, Springer-Verlag, London, UK.
32. Kim, Y.S., Jin, S.T., Lee, S.W., 2006, Design Activities and Personal Creativity Characteristics: A Case Study of Dual Protocol Analysis Using design Information and Process. ASME IDECT/CIE, Philadelphia, PA, DETC2006-99654(DTM).
33. Köhler, W., 1959, Gestalt Psychology Today, American Psychologist, 14:727-734.
34. nd Edition, Ideation Int'l Inc., Southfield, MI.
35. Lutfey, K.E., Campbell, S.M., Renfrew, M.R., Marceau, L.D., Roland, M., McKinlay, J.B., 2008, How are patient characteristics relevant for physicians' clinical decision making in diabetes? Social Science & Medicine, 67:1391-1399.
36. Merriam-webster.com, 2008, Merriam-Webster Online Dictionary, http://merriam-webster.com
37. Nagai, Y. and T. Taura, 2006, Formal Description of Concept-Synthesizing Process for Creative Design, in Design Computing and Cognition '06, Ed. J.S. Gero, 2006.
38. Nisbett, R., Wilson, T., 1977, Telling more than we can know: Verbal reports on mental processes, Psychological Review, 84:231-259.
39. Obtata, T., Daimon, T., Kawashima, H., 1993, A Cognitive Study of In-vehicle Navigation Systems: Applying Verbal Protocol Analysis to Usability Evaluation, Proceedings of IEEE - IEEE Vehicle Navigation & Information Systems Conference, Ottawa.
40. Rasmussen, J., Jensen, A., 1974, Mental procedures in real life tasks. A case study in electronics trouble shooting, Ergonomics, 17:293-30.
41. Rittel, H.W.J., Webber, M. M., 1984, "Planning Problems are Wicked Problems", in N. Cross' (ed.) Developments in Design Methodology, John Wiley & Sons, Bath, England, pp 135-144.
42. Segers, N., 2004, Computational representations of words & representations of words & associations in architectural design, development of a system support creative design, Bouwstenen 78, Technische Universiteit Eindhoven, Ph.D. Thesis.
43. Shah, J., Kulkarni, S., Vargas-Hernandez, N., 2000, Evaluation of Idea Generation Methods for Conceptual Design: Effectiveness Metrics & Design of Experiments, J. Mech. Des., 122:377-384.
44. Simon, H., 1969, The Sciences of the Artificial, MIT Press, Cambridge, MA.
45. Srinivasan, V., Chakrabarti, A., 2009, An Empirical Evaluation of Novelty-Sapphire Relationship, ASME IDETC/CIE, San Diego, CA, USA, August 30-September 2, 2009. DETC2009-86668 (DTM).
46. Tang, Y., and Zeng., Y., 2009, Quantifying Designer's Mental Stress in the Conceptual Design Process Using Kinesics Study, Proc of International Conference on Engineering Design, ICED'09, Stanford University, Stanford, CA, August 24-27, 2009.
47. Thomas, J.C., Carroll, J.M., 1984, The Psychological Study of Design, in N. Cross's (ed.) Developments in Design Methodology, pp 221-235.
48. Torrance, E.P.,1974, Torrance Tests of Creative Thinking, Scholastic Testing Service, Inc.
49. Tseng, I., Cagan, J., Moss, J. and Kotovsky, K., 2008, Overcoming Blocks in Conceptual Design: The Effects of Open Goals and Analogical Similarity on Idea Generation, ASME IDETC/CIE, Brooklyn, NY, Aug 3-6, DETC2008-49276(DTM).
50. Wickens, C.D., Gordon, S., Liu Y., 1997, Introduction to Human Factors Engineering, Longman, New York, NY.
51. rd Edition, Prentice Hall, Upper Saddle River, NJ.
52. WordNet, 3.0, http://www.cogsci.princeton.edu/~wn.
53. Ullman, D., 2003, The Mechanical Design Process, Third Edition, McGraw-Hill, New York, NY.
54. Visser, W., 2006, The Cognitive Artifacts of Designing, Lawrence Erlbaum Associates, Publishers, Mahwah, NJ.
55. Yang, M.C., Cham, J.G., 2007, An analysis of sketching skill and its role in early stage engineering design, J. Mech. Des., 129:476-482.
56. Zeng, Y., 2009, Environmentally Based Design: A Method for Innovative Design, Chalmers Design Seminar, University of Toronto, March 18, 2009.