Cueing retrospective verbal reports in usability testing through eye-movement replay

People and Computers XXI HCI.But Not as We Know It - Proceedings of HCI 2007: The 21st British HCI Group Annual Conference

Volumn 1, Issue , 2007, Pages

  Eger, Nicola ^a   Ball, Linden J ^b   Stevens, Robert ^c   Dodd, Jon ^c  

^a Sony Ericsson Software Applications Laboratory   (United Kingdom)

^b LANCASTER UNIVERSITY   (United Kingdom)

^c Bunnyfoot Ltd   (United Kingdom)

Author keywords

Eye tracking; Online usability testing; Verbal protocols

Indexed keywords

EYE TRACKING; ONLINE SEARCHING; SEARCH ENGINES; USABILITY ENGINEERING;

COMPLETION TIME; COMPLEX SEARCHES; EXPERIMENTAL VALIDATIONS; ONLINE USABILITY TESTING; THINKING ALOUD; USABILITY PROBLEMS; USABILITY TESTING; VERBAL PROTOCOLS;

EYE MOVEMENTS;

EID: 84860500688     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.14236/ewic/hci2007.13     Document Type: Conference Paper

References (34)

1. Ahlum-Heath, M. E., and Divesta, F. J. The effects of conscious controlled verbalizations of a cognitive strategy on transfer in problem solving. Memory & Cognition, 14 (1986), 281-285.
2. Austin J., and Delaney, P. F. Protocol analysis as a tool for behavior analysis. The Analysis of Verbal Behavior, 15 (1998), 41-56.
3. Bainbridge, L. Verbal reports as evidence of the process operator's knowledge. International Journal of Human-Computer Studies, 51 (1999), 213-238.
4. Barnum, C. M. Usability Testing and Research. Longman, New York, 2002.
5. Berardi-Coletta, B., Buyer, L. S., Dominowski, R. L., and Rellinger, E. R. Metacognition and problem solving: A process-oriented approach. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21 (1995), 205-223.
6. Berry, D. C., and Broadbent, D. E. Explanation and verbalization in a computer search task. Quarterly Journal of Experimental Psychology, 39A (1987), 585-609.
7. Boren, M. T., and Ramey, J. Thinking aloud: Reconciling theory and practice. IEEE Transactions on Professional Communication, 43 (2000), 261-278.
8. Bowers, V.A., and Snyder, H.L. Concurrent versus retrospective verbal protocols for comparing window usability. In Proceedings of the Human factors society 34th annual meeting. HFES Press, 1990, 1270-1274.
9. Broadbent, D. E. Effective decisions and their verbal justification. Philosophical Transactions of the Royal Society of London, B327 (1990), 493-502.
10. Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., and Glaser, R. Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13 (1989), 145-182.
11. Cowen, L., Ball., L. J., and Delin, J. An eye-movement analysis of web-page usability. In People and Computers XVI (Proceedings of HCI 2002). Springer, London, 2002, 317-335.
12. Davies, S. P. Effects of concurrent verbalisation on design problem solving. Design Studies, 16 (1995), 102-116.
13. Dickson, J., McLennan, J., and Omodei, M. M. Effects of concurrent verbalisation on a time-pressured dynamic decision task. Journal of General Psychology, 127 (2000), 217-228.
14. Dumas, J. S., and Redish, J. C. A Practical Guide to Usability Testing (Revised Edition). Intellect Press, Exeter, UK, 1999.
15. Ericsson, A. K., and Simon, H. A. Protocol Analysis: Verbal Reports as Data. MIT Press, Cambridge, MA, 1984.
16. Ericsson, K. A., and Simon, H. A. Protocol Analysis: Verbal Reports as Data. MIT Press, Cambridge, MA, 1993.
17. Gagne, R. M., and Smith Jr., E. C. A study of the effects of verbalization on problem solving. Journal of Experimental Psychology, 63 (1962), 12-18.
18. Hoc, J. M., and Leplat, J. Evaluation of different modalities of verbalization in a sorting task. International Journal of Man-Machine Studies, 18 (1983), 283-306.
19. Jacob, R. J. K., and Karn, K. S. Eye tracking in humancomputer interaction and usability research: Ready to deliver the promises. In J. Hyona, R. Radach, and H. Deubel (eds.), The mind's eye: Cognitive and applied aspects of eye movements. Elsevier London, UK, 2003, 573-605.
20. Kirk, P., and Ashcraft, M. H. Telling stories: The perils and promise of using verbal reports to study math strategies. Journal of Experimental Psychology: Learning, Memory, and Cognition, 27 (2001), 157-175.
21. McCarthy, J. D., Sasse, M. A., and Riegelsberger, J. "Could I have the menu please?": An eyetracking study of design conventions. In People and Computers XVII (Proceedings of HCI 2003). Springer, London, 2003, 401-414.
22. Nisbett, R. E., and Wilson, T. D. Telling more than we know: Verbal reports on mental processes. Psychological Review, 84 (1977), 231-259.
23. Nielsen, J. Usability Engineering. Academic Press, Boston, MA, 1993.
24. Nielsen, J., Clemmensen, T., and Yssing, C. Getting access to what goes on in people's heads: Reflections on the think-aloud technique. In Proceedings of the second Nordic conference on Human-computer interaction (NordiCHI 2002). ACM Press, New York, NY, 2002, 101-110.
25. Poole, A., and Ball, L. J. Eye tracking in HCI and usability research. In C. Ghaoui (ed.), Encyclopedia of humancomputer interaction. Idea Group Inc., Pennsylvania, 2006.
26. Poole, A., Ball, L. J., and Phillips, P. In search of salience: A response time and eye movement analysis of bookmark recognition. In People and Computers XVIII (Proceedings of HCI 2004). Springer, London, 2004, 363-378.
27. Robinson, K. M. The validity of verbal reports in children's subtraction. Journal of Educational Psychology, 93 (2001), 211-222.
28. Rubin, J. Handbook of Usability Testing: How to Plan Design and Conduct Effective Tests. Wiley, New York, 1994.
29. Russo, J. E., Johnson, E. J. and Stephens, D. L. The validity of verbal protocols. Memory & Cognition, 17 (1989), 759-769.
30. Schooler, J. W., Ohlsson, S., and Brooks, K. Thoughts beyond words: When language overshadows insight. Journal of Experimental Psychology: General, 122 (1993), 166-183.
31. Schuck, J. R., and Leahy, W. R. A comparison of verbal non-verbal reports of fragmenting visual images. Perception and Psychophysics, 1 (1966), 191-192.
32. Taylor, K.L., and Dionne, J. P. Accessing problem solving strategy knowledge: The complementary use of concurrent verbal protocols and retrospective debriefing. Journal of Educational Psychology, 29 (2000), 413-425.
33. Teague, R., De Jesus, K., and Nunes-Ueno, M. Concurrent vs. post task usability test ratings. In Proceedings of the SIGCHI conference on Human factors in computing systems (CHI 2001). ACM Press, New York, NY, 2001, 289-290.
34. Van den Haak, M. J., de Jong, M. D. T., and Jan Schellens, P. Retrospective vs. concurrent think-aloud protocols: testing the usability of an online library catalogue. Behaviour & Information Technology, 22 (2003), 339-351.