Measuring spontaneous and instructed evaluation processes during Web search: Integrating concurrent thinking-aloud protocols and eye-tracking data

Learning and Instruction

Volumn 21, Issue 2, 2011, Pages 220-231

  Gerjets, Peter ^a   Kammerer, Yvonne ^a   Werner, Benita ^a  

^a KNOWLEDGE MEDIA RESEARCH CENTER   (Germany)

Author keywords

Evaluation criteria; Eye tracking; Thinking aloud; Web search

Indexed keywords

EID: 78650176302     PISSN: 09594752     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.learninstruc.2010.02.005     Document Type: Article

References (35)

1. Borlund P. The concept of relevance in IR. Journal of the American Society for Information Science and Technology 2003, 54:913-925.
2. Brand-Gruwel S., Gerjets P. Instructional support for enhancing students' information problem solving ability. Computers in Human Behavior 2008, 24:615-622.
3. Brand-Gruwel S., Wopereis I., Vermetten Y. Information problem solving by experts and novices: analysis of a complex cognitive skill. Computers in Human Behavior 2005, 21:487-508.
4. Brumby D.P., Howes A. Strategies for guiding interactive search: an empirical investigation into the consequences of label relevance for assessment and selection. Human-Computer Interaction 2008, 23:1-46.
5. Crystal A., Greenberg J. Relevance criteria identified by health information users during web searches. Journal of the American Society for Information Science and Technology 2006, 57:1368-1382.
6. Cutrell E., Guan Z. What are you looking for? An eye-tracking study of information usage in Web search. Proceedings of the SIGCHI conference on human factors in computing systems 2007, 407-416. ACM Press, New York. M.B. Rosson, D.J. Gilmore (Eds.).
7. De Koning B.B., Tabbers H.K., Rikers R.M.J.P., Paas F. Attention guidance in learning from complex animation: seeing is understanding?. Learning and Instruction 2010, 20:111-122.
8. Ericsson K.A., Simon H.A. Protocol analysis: Verbal reports as data 1993, MIT Press, Cambridge, MA.
9. Gerjets P., Hellenthal-Schorr T. Competent information search in the World Wide Web: development and evaluation of a web training for pupils. Computers in Human Behavior 2008, 24:693-715.
10. Granka L., Feusner M., Lorigo L. Eye monitoring in online search. Passive eye monitoring: Algorithms, applications and experiments 2008, 283-304. Springer, Berlin. R.I. Hammoud (Ed.).
11. Granka L., Joachims T., Gay G. Eye-tracking analysis of user behavior in WWW search. Proceedings of the 27th annual ACM SIGIR conference on research and development in information retrieval 2004, 478-479. ACM Press, New York. M. Sanderson, K. Järvelin, J. Allan, P. Bruza (Eds.).
12. Guan Z., Cutrell E. An eye tracking study of the effect of target rank on Web search. Proceedings of the SIGCHI conference on human factors in computing systems 2007, 417-420. ACM Press, New York. M.B. Rosson, D.J. Gilmore (Eds.).
13. Designing information spaces: The social navigation approach 2003, Springer, London. K. Höök, D. Benyon, A.J. Munro (Eds.).
14. Jarodzka H., Scheiter K., Gerjets P., Van Gog T. In the eyes of the beholder: how experts and novices interpret dynamic stimuli. Learning and Instruction 2010, 20:146-154.
15. Joachims T., Granka L., Pan B., Hembrooke H., Gay G. Accurately interpreting click through data as implicit feedback. Proceedings of the 28th annual international ACM SIGIR conference on research and development in information retrieval 2005, 154-161. ACM Press, New York. R. Baeza-Yates, N. Ziviani, G. Marchionini, A. Moffat, J. Tait (Eds.).
16. Kammerer Y., Wollny E., Gerjets P., Scheiter K. How authority-related epistemological beliefs and salience of source information influence the evaluation of web search results - an eye tracking study. Proceedings of the 31st annual conference of the cognitive science society 2009, 2158-2163. Cognitive Science Society, Austin, TX. N.A. Taatgen, H. van Rijn (Eds.).
17. Maglaughlin K.L., Sonnenwald D.H. User perspectives on relevance criteria: a comparison among relevant, partially relevant, and not-relevant judgments. Journal of the American Society for Information Science and Technology 2002, 53:327-342.
18. Marchionini G. Information seeking in electronic environments 1995, Cambridge University Press, Cambridge, UK.
19. Miller C.S., Remington R.W. Modeling information navigation: implications for information architecture. Human-Computer Interaction 2004, 19:225-271.
20. Pan B., Hembrooke H., Joachims T., Lorigo L., Gay G., Granka L. In Google we trust: users' decisions on rank, position, and relevance. Journal of Computer-Mediated Communication 2007, 12:801-823.
21. Pirolli P., Card S.K. Information foraging. Psychological Review 1999, 106:643-675.
22. Pirolli P., Fu W.-T.F. SNIF-ACT: a model of information foraging on the World Wide Web. Proceedings of the 9th international conference on user modeling 2003, 45-54. Springer, Johnstown, PA. P. Brusilovsky, A.T. Corbett, F. De Rosis (Eds.).
23. Rayner K. Eye movements in reading and information processing: 20 years of research. Psychological Bulletin 1998, 124:372-422.
24. Rieh S.Y. Judgment of information quality and cognitive authority in the web. Journal of the American Society for Information Science and Technology 2002, 53:145-161.
25. Rouet J.-F. The skills of document use: From text comprehension to Web-based learning 2006, Erlbaum, Mahwah, NJ.
26. Rouet J.-F., Favart M., Britt M.A., Perfetti C.A. Studying and using multiple documents in history: effects of discipline expertise. Cognition and Instruction 1997, 15:85-106.
27. Savolainen R., Kari J. User-defined relevance criteria in web searching. Journal of Documentation 2005, 62:685-707.
28. Schamber L. Relevance and information behaviour. Annual review of information science and technology (ARIST) 1994, 3-48. Learned Information, Medford, NJ. M.E. Wiliams (Ed.).
29. Stadtler M., Bromme R. Dealing with multiple documents on the WWW: the role of metacognition in the formation of documents models. Computer-Supported Collaborative Learning 2007, 2:191-210.
30. Sundar S.S., Knobloch-Westerwick S., Hastall M.R. News cues: information scent and cognitive heuristics. Journal of the American Society for Information Science and Technology 2007, 58:366-378.
31. Tombros A., Ruthven I., Jose J.M. How users assess web pages for information-seeking. Journal of the American Society for Information Science and Technology 2005, 56:327-344.
32. Van Gog T., Paas F., Van Merriënboer J.J.G. Uncovering expertise-related differences in troubleshooting performance: combining eye movement and concurrent verbal protocol data. Applied Cognitive Psychology 2005, 19:205-221.
33. Van Gog T., Paas F., Van Merriënboer J.J.G., Witte P. Uncovering the problem solving process: cued retrospective reporting versus concurrent and retrospective reporting. Journal of Experimental Psychology: Applied 2005, 11:237-244.
34. Walraven A. Becoming a critical websearcher: Effects of instruction to foster transfer 2008, Unpublished doctoral dissertation, Open University of the Netherlands, Heerlen, The Netherlands, Retrieved from. http://hdl.handle.net/1820/1734.
35. Wilson T.D. Models in information behaviour research. Journal of Documentation 1999, 55:249-270.