Assessing concurrent think-aloud protocol as a usability test method: A technical communication approach

IEEE Transactions on Professional Communication

Volumn 53, Issue 3, 2010, Pages 202-215

  Cooke, Lynne ^a  

^a WEST CHESTER UNIVERSITY   (United States)

Author keywords

Cognitive psychology; communication; technical communication

Indexed keywords

TESTING; USABILITY ENGINEERING;

COGNITIVE PSYCHOLOGY; DATA COLLECTION METHOD; GAIN INSIGHT; GROWING BODIES; REAL- TIME; TASK PERFORMANCE; TECHNICAL COMMUNICATIONS; TEST METHOD; THINK-ALOUD PROTOCOL; USABILITY TESTS;

EYE MOVEMENTS;

EID: 77956195591     PISSN: 03611434     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPC.2010.2052859     Document Type: Article

References (53)

1. A. J. Hornof, "Visual search and mouse-pointing in labeled versus unlabeled two-dimensional visual hierarchies," ACM Trans. Comput.-Human Interact., vol. 8, no. 3, pp. 171-197, 2001.
2. M. D. Byrne, "Cognitive architecture," in The Human-Computer Interaction Handbook, J. A. Jacko and A. Sears, Eds. New York: Lawrence Erlbaum Associates, 2003, pp. 97-117.
3. J. R. Anderson, The Rules of the Mind. Hillsdale, NJ: Lawrence Erlbaum Associates, 1993.
4. G. M. Olson and J. S. Olson, "Human-computer interaction: Psychological aspects of the human use of computing," Annu. Rev. Psychol., vol. 54, pp. 491-516, 2003.
5. J. M. Carroll, "Human-computer interaction: Psychology as a science of design," Annu. Rev. Psychol., vol. 48, pp. 61-83, 1997.
6. K. Holtzblatt, "Contextual design," in The Human-Computer Interaction Handbook, J. A. Jacko and A. Sears, Eds. New York: Lawrence Erlbaum Associates, 2003, pp. 941-963.
7. J. Ramey, "Usability testing: Conducting the test procedure itself," in Proc. Int. Professional Communication Conf., 1987, pp. 131-134.
8. J. Ramey, "A self-reporting methodology for rapid data analysis in usability testing," in Proc. Int. Professional Communication Conf., 1988, pp. 147-150.
9. J. Redish, "Reading to learn to do," IEEE Trans. Prof. Commun., vol. 32, no. 4, pp. 289-293, Dec., 1989.
10. P. Sullivan, "Beyond a narrow conception of usability testing," IEEE Trans. Prof. Commun., vol. 32, no. 4, pp. 256-264, Dec., 1989.
11. P. Sullivan, "User protocols: Tools for building an understanding of users," in Proc. Int. Prof. Commun. Conf., 1988, pp. 259-263.
12. K. A. Ericsson and H. A. Simon, Protocol Analysis: Verbal Aspects as Data, revised ed. Cambridge, MA: MIT Press, 1993.
13. K. A. Ericsson and H. A. Simon, Protocol Analysis. Cambridge, MA: MIT Press, 1980.
14. K. S. Bowers, G. Regehr, C. Balthazard, and K. Parker, "Intuition in the context of discovery," Cognit. Psychol., vol. 22, no. 1, pp. 72-110, 1990.
15. L. L. Jacoby, S. D. Lindsay, and J. P. Toth, "Unconscious influences revealed: Attention, awareness, and control," Amer. Psychol., vol. 47, no. 6, pp. 802-809, 1992.
16. T. D. Wilson and J. W. Schooler, "Thinking too much: Introspection can reduce the quality of preferences and decisions," J. Personal. Social Psychol., vol. 60, no. 2, pp. 181-192, 1991.
17. J. W. Schooler, S. Ohlsson, and K. Brooks, "Thoughts beyond words: When language overshadows insight," J. Exper. Psychol., vol. 122, no. 2, pp. 166-183, 1993.
18. J. E. Russo, E. J. Johnson, and D. L. Stephens, "The validity of verbal protocols," Memory Cognit., vol. 17, no. 6, pp. 759-769, 1989.
19. M. T. Boren and J. Ramey, "Thinking aloud: Reconciling theory and practice," IEEE Trans. Prof. Commun., vol. 43, no. 3, pp. 261-278, Sep., 2000.
20. L. Flower and J. R. Hayes, "A cognitive process theory of writing," Coll. Compos. Commun., vol. 32, no. 4, pp. 365-387, 1981.
21. R. J. Connors, "Composition studies and science," Coll. English, vol. 44, no. 8, pp. 1-20, 1982.
22. M. Cooper and M. Holtzman, "Talking about protocols," Coll. Comp. Commun., vol. 34, no. 3, pp. 284-293, 1983.
23. E. R. Steinberg, "Protocols, retrospective reports, and the stream of consciousness," Coll. English, vol. 48, no. 7, pp. 697-712, 1986.
24. J. Nielsen, Usability Engineering. London, UK: Academic Press, 1993.
25. B. Schneiderman, Designing the User Interface: Strategies for Effective Human-Computer Interaction, 3rd ed. Reading, MA: Addison-Wesley, 1998.
26. J. C. Dumas and J. S. Redish, A Practical Guide to Usability Testing. New York: Ablex, 1994.
27. J. Rubin, Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. New York: Wiley, 1994.
28. E. Krahmer and N. Ummelen, "Thinking about thinking aloud: A comparison of verbal protocols for usability testing," IEEE Trans. Prof. Commun., vol. 47, no. 3, pp. 105-117, Sep., 2004.
29. M. J. Van Den Haak, M. D. T. De Jong, and P. J. Schellens, "Retrospective versus concurrent think-aloud protocols: Testing the usability of an online library catalogue," Behav. Inf. Technol., vol. 22, no. 5, pp. 339-351, 2003.
30. M. J. Van Den Haak, M. D. T. De Jong, and P. J. Schellens, "Employing think-aloud protocols and constructive interaction to test the usability of an online library," Interact. Comput., vol. 16, no. 6, pp. 1153-1170, 2004.
31. M. J. Van Den Haak, M. D. T. De Jong, and P. J. Schellens, "Constructive interaction: An analysis of verbal interaction in a usability setting," IEEE Trans. Profess. Commun., vol. 49, no. 4, pp. 311-324, Dec., 2006.
32. M. J. Van Den Haak, M. D. T. De Jong, and P. J. Schellens, "Evaluation of an informational Web site: Three variants of the think-aloud method compared," Tech. Commun., vol. 54, no. 1, pp. 58-71, 2007.
33. D. Rhenius and G. Deffner, "Evaluation of concurrent thinking aloud using eye-tracking data," in Proc. 34th Human Factors Soc. Annu. Meet., 1990, pp. 1265-1269.
34. M. Hertzum, K. D. Hansen, and H. H. K. Andersen, "Scrutinising usability evaluation: Does thinking aloud affect behaviour and mental workload?," Behav. Inf. Technol., vol. 28, no. 2, pp. 165-181, 2009.
35. V. A. Bowers and H. L. Snyder, "Concurrent versus retrospective verbal protocol for comparing Windows usability," in Proc. 34th Human Factors Soc. Annu. Meet., 1990, pp. 1270-1274.
36. L. R. Peterson, "Concurrent verbal activity," Psychol. Rev., vol. 76, no. 4, pp. 376-386, 1969.
37. R. J. K. Jacob and S. K. Karn, "Eye tracking in human-computer interaction and usability research: Ready to deliver the promises," in Cognitive and Applied Aspects of Eye Movement Research, J. Hyönä, R. Radach, and H. Deubel, Eds. Amsterdam, The Netherlands: Elsevier, 2003, pp. 573-605.
38. L. Cooke, "Eye tracking: How it works and how it relates to usability," Tech. Commun., vol. 52, no. 4, pp. 456-463, 2005.
39. L. Cooke, "How do users search web home pages? An eye-tracking study of multiple navigation menus," Tech. Commun., vol. 55, no. 2, pp. 176-194, 2008.
40. W. P. Eveland and S. Dunwoody, "Examining information processing on the World Wide Web using think aloud protocols," Media Psychol., vol. 2, no. 3, pp. 219-244, 2004.
41. W. A. Scott, "Reliability of content analysis: The case of nominal scale coding," Public Opin. Quart., vol. 19, no. 3, pp. 321-325, 1955.
42. G. O. Wesolowsky, Multiple Regression and Analysis of Variance. New York: Wiley, 1976.
43. K. Rayner, "Eye movements in reading and information processing: 20 years of research," Psychol. Bull., vol. 124, no. 3, pp. 372-422, 1998.
44. K. A. Ericsson, "Concurrent verbal reports on reading and text compression," Text, vol. 8, no. 4, pp. 295-325, 1988.
45. J. Redish, Letting Go of the Words: Writing Web Content That Works. Amsterdam, The Netherlands: Elsevier, 2007.
46. S. Krug, Don't Make Me Think: A Common Sense Approach to Web Usability, 2nd ed. Berkeley, CA: New Rider's Press, 2005.
47. J. Nielsen and H. Loranger, Prioritizing Web Usability. Berkeley, CA: New Rider's Press, 2006.
48. F. Engle, "Visual conspicuity, visual search and fixation tendencies of the eye," Vis. Res., vol. 17, no. 1, pp. 95-108, 1977.
49. J. Findlay, "Visual search: Eye movements and peripheral vision," Optomet. Vis. Sci., vol. 72, no. 7, pp. 461-466, 1995.
50. J. M. Henderson and A. Hollingworth, "Eye movements during scene viewing: An overview," in Eye Guidance in Reading and Scene Perception, G. Underwood, Ed. Amsterdam, The Netherlands: Elsevier, 1998, pp. 269-293.
51. H. H. Clark and J. E. Fox Tree, "Using uh and um in spontaneous speaking," Cognition, vol. 84, no. 1, pp. 73-111, 2002.
52. E. Goffman, "Radio talk," in Forms of Talk, E. Goffman, Ed. Philadelphia, PA: Univ. of Pennsylvania Press, 1981, pp. 197-327.
53. M. De Jong and P. J. Schellens, "Toward a document evaluation methodology: What does research tell us about the validity and reliability of evaluation methods?," IEEE Trans. Prof. Commun., vol. 43, no. 3, pp. 242-260, Sep., 2000.