Reconceptualizing think-aloud methodology: Refining the encoding and categorizing techniques via contextualized perspectives

Computers in Human Behavior

Volumn 19, Issue 1, 2003, Pages 95-115

  Yang, Shu Ching ^a  

^a NATIONAL SUN YAT SEN UNIVERSITY   (Taiwan)

Author keywords

Categorization; Encoding; Ill structured domain; Protocol analysis; Think aloud methodology

Indexed keywords

DATA PROCESSING; ENCODING (SYMBOLS); MATHEMATICAL MODELS; PROBLEM SOLVING;

PROTOCOL ANALYSIS;

PSYCHOLOGY COMPUTING;

EID: 0037212460     PISSN: 07475632     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0747-5632(02)00011-0     Document Type: Article

References (98)

1. Afflerbach P., Johnson P. On the use of verbal reports in reading research. Journal of Reading Behavior. 16(4):1984;307-332.
2. Albrechtsen D. One writer two languages: a case study of a 15-year-old student's writing process in Danish and English. International Journal of Applied Linguistics. 7(2):1997;223-250.
3. Bailey W.A., Kay E.J. Toward the standard analysis of verbal data. IEEE Systems, Man, and Cybernetics Proceedings. 1986;582-587.
4. Ball C.T., Langholtz H.J. Resource-allocation strategies: a verbal protocol analysis. Organizational Behavior & Human Decision Processes. 76(1):1998;70-89.
5. Barber A.E., Roehling M.V. Job postings and the decision to interview: a verbal protocol analysis. Journal of Applied Psychology. 78(5):1993;845-857.
6. Bartlett F.C. Remembering. 1932;Cambridge University Press, Cambridge.
7. Baumann J.F., Jones L.A., Seifert-Kessell N. Using thinking-alouds to enhance children's comprehension monitoring abilities. The Reading Teacher. 47(3):1993;184-193.
8. Beishuizen J., van der Schalk J., Le Grand J. No correlation between inferencing causal relations and text comprehension? Learning and Instruction. 9(1):1999;37-56.
9. Benson P.J. Problems in picturing text: a study of visual/verbal problem solving. Technical Communication Quarterly. 6(2):1997;141-160.
10. Bereiter C., Bird M. Use of thinking aloud in identification and teaching of reading comprehension strategies. Cognition and Instruction. 2:1985;131-156.
11. Biggs S.F., Rosman A.J., Sergenian G.K. Methodological issues in judgment and decision-making research: concurrent verbal protocol validity and simultaneous traces of process. Journal of Behavioral Decision Making. 6:1993;187-206.
12. Bordage G., Lemieux M. Semantic structures and diagnostic thinking of experts and novices. Academic Medicine. 66(9):1991;70-72.
13. Bracewell R.J., Breuleux A. Substance and romance in analyzing think-aloud protocols. Smagorinsky P. Speaking about writing: reflections on research methodology. 1994;55-88 Sage, Thousand Oaks, CA.
14. Breuleux A. The analysis of writers' think-aloud protocols: developing a principled coding scheme for ill-structured tasks. Denhiere G., Rossi J.P. Text and text processing. 1991;333-362 North-Holland, Amsterdam.
15. Chi M.T.H., Bassok M., Lewis M., Reimann P.L., Glaser R. Self explanations: how students study and use examples in learning to solve problems. Cognitive Science. 13:1989;145-182.
16. Cohen A.D. Verbal reports as a source of insights into second language learner strategies. Applied Language Learning. 7(1&2):1996;5-24.
17. Crain-Thoreson C., Lippman M.Z. Windows on comprehension: reading comprehension processes as revealed by two think-aloud procedures. Journal of Educational Psychology. 89(4):1997;579-592.
18. Crain-Thoreson C., Lippman M.Z., McClendon-Magnuson D. Windows on comprehension: reading comprehension processes as revealed by two think-aloud procedures. Journal of Educational Psychology. 89(4):1997;579-591.
19. Cummings, A.L. (1988). Protocol analysis of social problem solving in counselor trainees. Paper presented at the 96th Annual Meeting of the American Psychological Association (ERIC Document reproduction service, number ED303699).
20. Davidson-Shivers, G.V., Shorter, L., & Jordoan, K. (1997). Children's learning strategies, encoding processes, and navigational decisions in a hypermedia concept. In O. Abel, N.J. Maushak, & K.E. Wright (Eds.), Proceedings of selected research and development presentations at the 1997 National Convention of the Association for Educational Communications and Technology (pp. 21-29). Iowa State University, Ames, IA.
21. Davis, J.N., & Bistodeau, L. (1993). How do L1 and L2 reading differ? Evidence from think aloud protocols. Modern Language Journal, 77(4), 459-472.
22. Dobrin D.N. Protocols once more. College English. 48:1986;713-726.
23. Dobrin D.N. Wither wisdom? Smagorinsky P. Speaking about writing: reflections on research methodology. 1994;275-289 Sage, Thousand Oaks, CA.
24. Draper S., Anderson A. The significance of dialogue in learning and observing learning. Computers and Education. 17(1):1991;93-107.
25. Edwards D., Mercer N. Common knowledge: the development of understanding in the classroom. 1987;Methuen, London, New York.
26. Ericsson K.A., Simon H.A. Protocol analysis: verbal reports as data. 1984/1993;MIT Press, Cambridge.
27. Ericsson K.A., Simon H.A. Protocol analysis. van Dijk T.A. Handbook of discourse analysis: dimensions of discourse. 1985;259-268 Academic Press, London.
28. Gardner H. The mind's new science: a history of cognitive revolution. 1985;Basic Books, New York.
29. Garner R. Verbal-report data on cognitive and metacognitive strategies. Weinstein C.E., Goetz E.T., Alexander P.A. Learning and study strategies: issues in assessment, instruction, and evaluation. 1988;63-76 Academic Press, San Diego.
30. Ghiselin M. Classification as an evolutionary problem. Costall A., Still A. Cognitive psychology in question. 1987;70-86 Harvester Press, Brighton, Sussex.
31. Gillespie D. The mind's we: contextualism in cognitive psychology. 1992;Southern Illinois University Press, Carbondale.
32. Grant-Davie K. Coding data issues of validity, reliability, and interpretation. Kirsch G., Sullivan P.A. Methods and methodology in composition research. 1992;270-286 Southern Illinois University Press, Carbondale.
33. Gray S.H. Hypertext and the technology of conversation: orderly situation choices. 1993;Greenwood Press, Westport, CT.
34. Green J., Franquiz M., Dixon C. The myth of the objective transcript: transcribing as a situated act. TESOL Quarterly. 31(1):1997;172-176.
35. Grotjahn R. On the methodological basis of introspective methods. Farch G., Kasper G. Introspection in second language research. 1987;54-81 Multilingual Matters, Clevedon, England.
36. Guthrie J., Britten T., Barker K. Roles of document structure, cognitive strategy, and awareness in searching for information. Reading Research Quarterly. 26:1991;300-324.
37. Hayes J.R., Flower L.S. Uncovering cognitive processes in writing: an introduction to protocol analysis. Mosenthal L., Tamor S., Walmsley A. Research on writing: principles and methods. 1983;207-220 Longman, New York.
38. Hayes L., Flower L. Identifying the organization of writing processes. Gregg L., Steinberg E. Cognitive processes in writing: an interdisciplinary approach. 1980;Erlbaum, Hillsdale, NJ.
39. Hewett T.T., Scott S. The use of thinking-out-loud and protocol analysis in development of a process model of interactive database searching. Bullinger H.J., Shackel B. Human-computer interaction. 1987;51-56 Elsevier Science Publishers.
40. Highhouse S. A verbal protocol analysis of choice under ambiguity. Journal of Economic Psychology. 15(4):1994;621-636.
41. Homa, D. (1984). On the nature of categories. In G.H. Bower (Ed.), The psychology of learning and motivation (Vol. 18; pp. 49-94). New York, Academic Press.
42. Janssen D., van Waes L., van den Bergh H. Effects of thinking aloud on writing processes. Levy C.M., Ransdell S. The science of writing: theories, methods, individual differences, and applications. 1996;Lawrence Erlbaum Associates, Mahwah, NJ.
43. Jenkins J.J. Four parts to remember: a tetrahedral model of memory experiments. Cermak L.S., Craik F.I.M. Levels of processing in human memory. 1979;429-446 Lawrence Erlbaum Associates, Hillsdale, NJ.
44. Johnson K.E. Cognitive strategies and second-language writers: a re-evaluation of sentence-combining. Journal of Second Language Writing. 1(1):1992;61-75.
45. Kang H.-W. Effects of culture-specific knowledge upon ESL reading comprehension. School of Education Review. 4:1992;93-105.
46. Kasper G. Analyzing verbal protocols. TESOL Quarterly. 32(2):1998;358-362.
47. Kormos J. The use of verbal reports in L2 research: verbal reports in L2 speech production research. TESOL Quarterly. 32(2):1998;353-358.
48. Kucan L., Beck I.L. Thinking aloud and reading comprehension research: inquiry, instruction, and social interaction. Review of Educational Research. 67(3):1997;271-299.
49. Lakoff G. Women, fire, and dangerous things. 1987;University of Chicago Press, Chicago.
50. Leow R.P. Attention, awareness, and foreign language behavior. Language Learning. 47(3):1997;467-505.
51. Lewis, C. (1992). Using the think-aloud method in cognitive interface design (IBM Technical Report RC 9265 (#40713)). Yorktown Heights, NY: IBM Thomas J. Watson Research Center.
52. Long D.L., Bourg T. Thinking aloud: telling a story about a story. Discourse Processes. 21(3):1996;329-339.
53. Loxterman J.A., Beck I.L., McKeown M.G. The effects of thinking aloud during reading on students' comprehension of more or less coherent text. Reading Research Quarterly. 29(4):1994;352-367.
54. Lund, M.A. (1985). Evaluating the user interface: the candid camera approach. In L. Borman, & B. Curtis (Eds.), Proceedings of CHI '85 human factors in computing systems (pp. 1-10). New York: ACM.
55. Mack R.L., Lewis C.H., Carroll J.M. Learning to use word processors: problems and prospects. ACM Transactions on Office Information Systems. 1(3):1983;254-271.
56. Matsumoto K. Verbal-report data and introspective methods in second language research: state of the art. RELC Journal. 24(1):1993;32-60.
57. Moore P. Information problem solving: a wider view of library skills. Contemporary Educational Psychology. 20(1):1995;1-31.
58. Neisser U. Cognitive psychology. 1967;Prentice-Hall, Englewood Cliffs, NJ.
59. Neisser U. Concepts and conceptual development: ecological and intellectual factors in categorization. 1987;Cambridge University Press, New York.
60. Newell A., Simon H.A. Human problem solving. 1972;Prentice-Hall, Englewood Cliffs, NJ.
61. Paley K.S. The college application essay: a rhetorical paradox. Assessing Writing. 3(1):1996;85-105.
62. Perl, S. (1984). Coding the composing process: a guide for teachers and researchers (ERIC Document 240609).
63. Poltrock, S.E., & Nasr, M.G. (1989). Protocol analysis: a tool for analyzing human-computer interaction (MCC Technical Report. No. ACT-HI 186-89).
64. Potter J., Wetherell M. Discourse and social psychology: beyond attitudes and behavior. 1994;Sage Publications, London; Newbury Park, CA.
65. Pressley M., Afflerbach P. Verbal protocols of reading: the nature of constructively responsive reading. 1995;Lawrence Erlbaum Associates, Hillsdale, NJ.
66. Ransdell S. Generating thinking-aloud protocols impact on the narrative writing of college students. The American Journal of Psychology. 108:1995;89-98.
67. Rosch E.H. Principles of categorization. Rosch E.H., Lloyd B.B. Cognition and categorization. 1978;27-48 Erlbaum, Hillsdale, NJ.
68. Sanderson P.M. Verbalizable knowledge and skilled task performance: association, dissociation and mental models. Journal of Experimental Psychology: Learning, Memory and Cognition. 15:1989;729-747.
69. Sanderson P.M., James J.M., Seidler K.S. SHAPA: an interactive software environment for protocol analysis. Ergonomics. 32:1989;1271-1302.
70. Sanderson P.M., Verhage A.G., Fuld R.B. State-space and verbal-protocol methods for studying the human operator in process control. Ergonomics. 32:1989;1343-1372.
71. Schkade D.A., Payne J.W. How people respond to contingent valuation questions: a verbal protocol analysis of willingness to pay for an environmental regulation. Journal of Environmental Economics & Management. 26(1):1994;88-110.
72. Schwegler R.A., Shamoon L.K. Meaning attribution in ambiguous texts in sociology. Bazerman C., Paradis J. Textual dynamics of the professions. 1991;216-233 University of Wisconsin Press, Madison.
73. Sienot M. Pretesting web sites: a comparison between the plus-minus method and the think-aloud method for the world wide web. Journal of Business and Technical Communication. 11(4):1997;469-482.
74. Smagorinsky P. The reliability and validity of protocol analysis. Written Communication. 6(4):1989;463-479.
75. Smagorinsky P. The writer's knowledge and the writing process: a protocol analysis. Research in the Teaching of English. 25(3):1991;339-364.
76. Smagorinsky P. Think-aloud protocol analysis: beyond the black box. Smagorinsky P. Speaking about writing: reflections on research methodology. 1994;3-19 Sage, Thousand Oaks, CA.
77. Smith J., Staudinger U.M. Occupational settings facilitating wisdom-related knowledge: the sample case of clinical. Journal of Consulting & Clinical Psychology. 62(5):1994;989-1000.
78. Sonnentag S. Expertise in professional software design: a process study. Journal of Applied Psychology. 83(5):1998;703-726.
79. Spiro, R.J., Coulson, R.L., Feltovich, P.J., & Anderson, D.K. (1988). Cognitive flexibility theory: advanced knowledge acquisition in ill-structured domains. In V. Patel (Ed.), Tenth Annual Conference of the Cognitive Science Society (pp. 375-383). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
80. Stricklan, J. (1987). Evaluating computer-tutors: a protocol study. Paper presented at the Annual Meeting of the Conference on College Composition and Communication (ERIC Document 280029).
81. Swanson H.L., O'Connor J.E., Cooney J.B. An information processing analysis of expert and novice teachers' problem solving. American Educational Research Journal. 27(3):1990;533-556.
82. Swarts H., Flower L.S., Hays J.R. Designing protocol studies of the writing process: an introduction. Beach R., Bridwell L.S. New directions in composition research: perspectives in writing research. 1984;53-71 Guilford, New York.
83. Teasley S.D. The role of talk in children's peer collaborations. Developmental Psychology. 31(2):1995;207-220.
84. Upton T.A. "Yuk, the skin of insects!" Tracking sources of errors in second language reading comprehension. Journal of College Reading and Learning. 29(1):1998;5-20.
85. van der Linden E. Does Feedback enhance computer-assisted language learning? Computers and Education. 21(1-2):1993;61-65.
86. Voss J.F. Learning and transfer in subject-matter learning: a problem-solving model. International Journal of Educational Research. 11:1987;607-622.
87. Wade S. Using thinking alouds to assess comprehension. The Reading Teacher. 44(6):1990;442-451.
88. Ward L., Traweek D. Application of a metacognitive strategy to assessment, intervention, and consultation: a think-aloud technique. Journal of School Psychology. 31(4):1993;469-485.
89. Waxman T.G. Scripted thinking and faulty problem representation: the effects of theoretical orientation, level of experience, and temporal order on causal judgment. Canadian Journal of Counseling. 25(2):1991;146-169.
90. Wedman J., Folger T. Analysis of analogical problem-solving processes via think-aloud protocols. Journal of Research and Development in Education. 30(1):1996;51-62.
91. Whitney P., Budd D. Think-aloud protocols and the study of comprehension. Discourse Processes. 21(3):1996;341-351.
92. Wilson, M., & Thomas, S. (1995). Holy smoke! I missed something here: cultural experience and the construction of meaning. English Education, 27(1), 53-64.
93. Wineburg S.S. Historical problem solving: a study of the cognitive processes used in the evaluation of documentary and pictorial evidence. Journal of Educational Psychology. 83(1):1991;73-87.
94. Wittgenstein, L. (1968). Philosophical investigations (3rd ed; G.E.M. Anscombe, Trans.). New York, Macmillan (Original work published 1953).
95. Yang S.C. A dynamic model of reading-linking-to-writing for problem-solving within a constructive hypermedia learning environment. Journal of Educational Multimedia and Hypermedia. 5(3/4):1996;283-302.
96. Yang S.C. Information-seeking as problem-solving. Library and Information Science Research. 19(1):1997;71-92.
97. Yang S.C. Divulging intertextual processes in the problem-solving of hypermedia. Journal of Interactive Learning Research. 10(1):1999;87-120.
98. Yang S.C. Multidimensional taxonomy of learners cognitive processing in discourse synthesis with hypermedia. Computers in Human Behavior. 18(1):2002;37-68.