Source evaluation of domain experts and novices during Web search

Journal of Computer Assisted Learning

Volumn 33, Issue 3, 2017, Pages 234-251

  Brand Gruwel, S ^a   Kammerer, Y ^b   van Meeuwen, L ^a   van Gog, T ^c,d  

^a OPEN UNIVERSITY OF THE NETHERLANDS   (Netherlands)

^b KNOWLEDGE MEDIA RESEARCH CENTER   (Germany)

^c ERASMUS UNIVERSITY ROTTERDAM   (Netherlands)

^d UTRECHT UNIVERSITY   (Netherlands)

Author keywords

cued retrospective report; source evaluation; thinking aloud; web search

Indexed keywords

EID: 85008182419     PISSN: 02664909     EISSN: 13652729     Source Type: Journal    
DOI: 10.1111/jcal.12162     Document Type: Article

References (55)

1. Braasch, J. L. G., Lawless, K. A., Goldman, S. R., Manning, F. H., Gomez, K. W., & MacLeod, S. M. (2009). Evaluating search results: An empirical analysis of middle school students' use of source attributes to select useful sources. Journal of Educational Computing Research, 41, 63–82.
2. Braasch, J. L. G., Rouet, J.-F., Vibert, N., & Britt, M. (2012). Readers' use of source information in text comprehension. Memory & Cognition, 40, 450–465. doi:10.3758/s13421-011-0160-6.
3. Braasch, J. L. G., Bråten, I., Strømsø, H. I., Anmarkrud, Ø., & Ferguson, L. E. (2013). Promoting secondary school students' evaluation of source features of multiple documents. Contemporary Educational Psychology, 38, 180–195. doi:10.1016/j.cedpsych.2013.03.003.
4. Brand-Gruwel, S., & Stadtler, M. (2011). Solving information-based problems: Searching, selecting and evaluating information. Learning and Instruction, 21, 175–179.
5. Brand-Gruwel, S., Wopereis, I., & Vermetten, Y. (2005). Information problem solving by experts and novices: Analysis of a complex cognitive skill. Computers in Human Behavior, 21, 487–508.
6. Brand-Gruwel, S., Wopereis, I., & Walraven, A. (2009). A descriptive model of Information Problem Solving while using Internet. Computers & Education, 53, 1207–1217.
7. Bråten, I., Strømsø, H. I., & Salmeron, L. (2011). Trust and mistrust when students read multiple information sources about climate change. Learning and Instruction, 21, 180–192.
8. Conrad, F. G., Blair, J., & Tracy, E. (1999). Verbal reports are data! A theoretical approach to cognitive interviews, Proceedings of the Federal Committee on Statistical Methodology Research Conference, Tuesday B Sessions, Arlington, VA, 11–20.
9. Cutrell, E., & Guan, Z. (2007). What are you looking for? An eye-tracking study of information usage in Web search. The Proceedings of ACM CHI’07, New York: ACM Press. 407-416-1061.
10. Ericsson, K. A., & Kintsch, W. (1995). Long-term working memory. Psychological Review, 102, 211–245.
11. Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press.
12. Gerjets, P., Kammerer, Y., & Werner, B. (2011). Measuring spontaneous and instructed evaluation processes during web search: Integrating concurrent thinking-aloud protocols and eye-tracking data. Learning and Instruction, 21, 220–231.
13. Goldman, S. R. (2012). Adolescent literacy: Learning and understanding content. Future of Children, 22, 89–116.
14. Goldman, S. R., Braasch, J. L. G., Wiley, J., Graesser, A. C., & Brodowinska, K. (2012). Comprehending and learning from Internet sources: Processing patterns of better and poorer learners. Reading Research Quarterly, 47, 356–381. doi:10.1002/RRQ.027.
15. Higgins, J. A., & Johnson, M. K. (2012). Some thoughts on the interaction between perception and reflection. In J. M. Wolfe & L. Robertson (Eds.), From perception to consciousness: Searching with Anne Treisman (pp. 390–397). New York: Oxford University Press.
16. Holmqvist, K., Nyström, N., Andersson, R., Dewhurst, R., Jarodzka, H., & Van de Weijer, J. (2011). Eye tracking: A comprehensive guide to methods and measures. Oxford, UK: Oxford University Press.
17. Hölscher, C., & Strube, G. (2000). Web search behavior of Internet experts and newbies. Computer Networks, 33, 337–346.
18. Jamali, H. R., & Asadi, S. (2010). Google and the scholar: The role of Google in scientists' information-seeking behaviour. Online Information Review, 34, 282–294.Baslev Baslev
19. Johnson, M. K., & Hirst, W. (1993). MEM: Memory subsystems as processes. In A. F. Collins, S. E. Gathercole, M. A. Conway & P. E. Morris (Eds.), Theories of memory (pp. 241–286). Sussex, UK: Erlbaum.
20. Kammerer, Y., & Gerjets, P. (2012). Effects of search interface and internet-specific epistemic beliefs on source evaluations during web search for medical information: An eye-tracking study. Behaviour & Information Technology, 31, 83–97.
21. Kammerer, Y., & Gerjets, P. (2013). The role of thinking-aloud instructions and prior domain knowledge in information processing and source evaluation during Web search. In M. Knauff, M. Pauen, N. Sebanz & I. Wachsmuth (Eds.), Proceedings of the 35th Annual Conference of the Cognitive Science Society (pp. 716–721). Austin, TX: Cognitive Science Society.
22. Kammerer, Y., & Gerjets, P. (2014). The role of search result position and source trustworthiness in the selection of Web search results when using a list or a grid interface. International Journal of Human Computer Interaction, 30, 177–191.
23. Kobayashi, K. (2009). The influence of topic knowledge, external strategy use, and college experience on students' comprehension of controversial texts. Learning and Individual Differences, 19, 130–134.
24. Kuhlthau, C. (1993). Seeking meaning: A process approach to library and information services. Norwood, NJ: Ablex.
25. Kuusela, H., & Paul, P. (2000). A comparison of concurrent and retrospective verbal protocol analysis. The American Journal of Psychology, 113, 387–404.
26. Lawless, K. A., Goldman, S. R., Gomez, K., Manning, F., & Braasch, J. (2012). Assessing multiple source comprehension through Evidence Centered Design. In J. P. Sabatini, T. O'Reilly & E. R. Albro (Eds.), Reaching an understanding: Innovations in how we view reading assessment (pp. 3–18). Lanham, MD: R & L Education.
27. MaKinster, J. G., Beghetto, R. A., & Plucker, J. A. (2002). Why can't I find Newton's third law? Case studies of students' use of the web as a science resource. Journal of Science Education and Technology, 11, 155–172.
28. Marchionini, G. (1995). Information-seeking in electronic environments. New York: Cambridge University.
29. Mason, L., Ariasi, N., & Boldrin, A. (2011). Epistemic beliefs in action: Spontaneous reflections about knowledge and knowing during online information searching and their influence on learning. Learning and Instruction, 21, 137–151.
30. McCrudden, M. T., & Schraw, G. (2007). Relevance and goal-focusing in text processing. Educational Psychology Review, 19, 113–139.
31. Metzger, M. J., Flanagin, A. J., & Zwarun, L. (2003). College student Web use, perceptions of information credibility, and verification behavior. Computers & Education, 41, 271–290.
32. Mosenthal, P. B. (1998). Defining prose task characteristics for use in computer-adaptive testing and instruction. American Educational Research Journal, 35, 269–307.
33. Pan, B., Hembrooke, H., Joachims, T., Lorigo, L., Gay, G., & Granka, L. (2007). In Google we trust: Users' decisions on rank, position, and relevance. Journal of Computer-Mediated Communication, 12, 801–823.
34. Purcell, K., Heaps, A., Buchanan, J., & Friedrich, L. (2013). How teachers are using technology at home and in their classrooms. Washington, DC: Pew Research Center's Internet & American Life Project.
35. Rouet, J. -F. (2006). The skills of document use: From text comprehension to web-based learning. Mahwah, NJ: Erlbaum.
36. Rouet, J.-F., Britt, M. A., Mason, R. A., & Perfetti, C. A. (1996). Using multiple sources of evidence to reason about history. Journal of Educational Psychology, 88, 478–493. doi:10.1037/0022-0663.88.3.478.
37. Rouet, J. -F., Favart, M., Britt, M. A., & Perfetti, C. A. (1997). Studying and using multiple documents in history: Effects of discipline expertise. Cognition and Instruction, 15, 85–106.
38. Salmerón, L., Kammerer, Y., & García-Carrión, P. (2013). Searching the Web for conflicting topics: Page and user factors. Computers in Human Behavior, 29, 2161–2171.
39. Stadtler, M., Scharrer, L., Brummernhenrich, B., & Bromme, R. (2013). Dealing with Uncertainty: Readers' memory for and use of conflicting information from science texts as function of presentation format and source expertise. Cognition and Instruction, 31, 130–150. doi:10.1080/07370008.2013.769996.
40. Stanford, J., Tauber, E.R., Fogg, B.J., & Marable, L. (2002). Experts vs. online consumers: A comparative credibility study of health and finance Web sites. Consumer WebWatch Research Report.
41. Strømsø, H. I., Bråten, I., Britt, M. A., & Ferguson, L. (2013). Spontaneous sourcing among students reading multiple documents. Cognition and Instruction, 31, 176–203. doi:10.1080/07370008.2013.769994.
42. Tabatabai, D., & More, B. M. (2005). How experts and novices search the Web. Library and Information Science Research, 27, 222–248.
43. Van Gog, T. (2006). Uncovering the problem-solving process to design effective worked examples (Doctoral Dissertation), Open University of The Netherlands, Heerlen, The Netherlands.
44. Van Gog, T., Paas, F., Van Merriënboer, J. J. G., & Witte, P. (2005). Uncovering the problem-solving process: Cued retrospective reporting versus concurrent and retrospective reporting. Journal of Experimental Psychology: Applied, 11, 237–244.
45. Van Gog, T., Kester, L., Nievelstein, F., Giesbers, B., & Paas, F. (2009). Uncovering cognitive processes: Different techniques that can contribute to cognitive load research and instruction. Computers in Human Behavior, 25, 325–331.
46. Van Someren, M. W., Barnard, Y. F., & Sandberg, J. A. C. (1994). The think aloud method: A practical guide to modeling cognitive processes. London: Academic Press.
47. Van Strien, J., Brand-Gruwel, S., & Boshuizen, H. P. A. (2014). Dealing with conflicting information from multiple nonlinear texts: Effects of prior attitudes. Computers in Human Behavior, 32, 101–111.
48. Von der Mühlen, S., Richter, T., Schmid, S., Schmidt, E. M., & Berthold, K. (2016). Judging the plausibility of arguments in scientific texts: A student–scientist comparison. Thinking and Reasoning, 22, 221–249.
49. Walhout, J., Brand-Gruwel, S., Van Dijk, M., Jarodzka, H., De Groot, R., & Kirschner, P. A. (2015). Navigating through hypertext: Do sex and type of navigational support matter? Computers in Human Behavior, 46, 218–227.
50. Walraven, A., Brand-Gruwel, S., & Boshuizen, H. P. (2009). How students evaluate information and sources when searching the World Wide Web for information. Computers & Education, 52, 234–246.
51. Wiley, J., Goldman, S. R., Graesser, A. C., Sanchez, C. A., Ash, I., & Hemmerich, J. (2009). Source evaluation, comprehension, and learning in Internet science inquiry tasks. American Educational Research Journal, 46, 1060–1160. doi:10.3102/0002831209333183.
52. Wilson, T. D. (1999). Models in information behaviour research. Journal of Documentation, 55, 249–270.
53. Wineburg, S. (1991). Historical problem solving: A study of the cognitive processes used in the evaluation of documentary and pictorial evidence. Journal of Educational Psychology, 83, 73–87.
54. Wopereis, I., Brand-Gruwel, S., & Vermetten, Y. (2008). The effect of embedded instruction on solving information problems. Computers in Human Behavior, 24, 738–752.
55. Wyatt, D., Pressley, M., El-Dinary, P. B., Stein, S., Evans, P., & Brown, R. (1993). Comprehension strategies, worth and credibility monitoring, and evaluations: Cold and hot cognition when experts read professional articles that are important to them. Learning and Individual Differences, 5, 49–72.