Untangling search task complexity and difficulty in the context of interactive information retrieval studies

Journal of Documentation

Volumn 70, Issue 6, 2014, Pages 1118-1140

  Wildemuth, Barbara ^a   Freund, Luanne ^b   Toms, Elaine G ^c  

^a UNIVERSITY OF NORTH CAROLINA   (United States)

^b UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^c UNIVERSITY OF SHEFFIELD   (United Kingdom)

Author keywords

Experimental methods; Interactive information retrieval; Measurement; Search task complexity; Search task difficulty

Indexed keywords

EID: 84910677448     PISSN: 00220418     EISSN: None     Source Type: Journal    
DOI: 10.1108/JD-03-2014-0056     Document Type: Article

References (78)

1. Albertson, D. (2010), “Analyzing user interaction with the ViewFinder video retrieval system”, Journal of the American Society for Information Science and Technology, Vol. 61 No. 2, pp. 238-252.
2. Albertson, D. and Meadows, C. III (2011), “Situated topic complexity in interactive video retrieval”, Journal of the American Society for Information Science & Technology, Vol. 62 No. 9, pp. 1676-1695.
3. Anderson, L.W. and Krathwohl, D.A. (2001), ATaxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objective, Longman, New York, NY.
4. Arapakis, I., Jose, J.M. and Gray, P.D. (2008), “Affective feedback: an investigation into the role of emotions in the information seeking process”, Proceedings of the 31st ACM SIGIR Conference on Research and Development of Information Retrieval, pp. 395-402.
5. Bailey, E.W., Kelly, D. and Gyllstrom, K. (2009), “Undergraduates’ evaluations of assigned search topics”, Proceedings 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 812-813.
6. Barsky, E. and Bar-Ilan, J. (2012), “The impact of task phrasing on the choice of search keywords and on the search process and success”, Journal of the American Society for Information Science & Technology, Vol. 63 No. 10, pp. 1987-2005.
7. Bell, D. and Ruthven, I. (2004), “Searcher’s assessments of task complexity for web searching”, 26th European Conference on IR Research (ECIR 2004), Advances in Information Retrieval, pp. 57-71.
8. Bilal, D. (2001), “Children’s use of the Yahooligans! Web search engine: II. Cognitive and physical behaviors on research tasks”, Journal of the American Society for Information Science and Technology, Vol. 52 No. 2, pp. 118-136.
9. Borgman, C.L. (1986), “The user’s mental model of an information retrieval system: an experiment on a prototype online catalog”, International Journal of Man-Machine Studies, Vol. 24 No. l, pp. 47-64.
10. Borlund, P. (2003), “The IIR evaluation model: a framework for evaluation of interactive information retrieval systems”, Information Research, Vol. 8 No. 3, paper 152.
11. Borlund, P. and Schneider, J.W. (2010), “Reconsideration of the simulated work task situation: a context instrument for evaluation of information retrieval interaction”, Proceedings of Information Interactions in Context (IIiX), pp. 155-164.
12. Byström, K. and Hansen, P. (2005), “Conceptual framework for tasks in information studies”, Journal of the American Society for Information Science & Technology, Vol. 56 No. 10, pp. 1050-1061.
13. Byström, K. and Jarvelin, K. (1995), “Task complexity affects information seeking and use”, Information Processing & Management, Vol. 31 No. 2, pp. 191-213.
14. Campbell, D.J. (1988), “Task complexity: a review and analysis”, Academy of Management Review, Vol. 13 No. 1, pp. 40-52.
15. Capra, R., Marchionini, G., Oh, J.S., Stutzman, F. and Zhang, Y. (2007), “Effects of structure and interaction style on distinct search tasks”, Proceedings of the 7th ACM/IEEE Joint Conference on Digital Libraries, JCDL 2007, pp. 442-451.
16. Chaplin, J.P. (1985), Dictionary of Psychology, Laurel, New York, NY.
17. Debowski, S. (2001), “Wrong way: go back! An exploration of novice search behaviours while conducting an information search”, The Electronic Library, Vol. 19 No. 6, pp. 371-382.
18. Ding, R. and Ma, F. (2013), “Assessment of university student web searching competency by a task-based online test: a case study at Wuhan University, China”, The Electronic Library, Vol. 31 No. 3, pp. 359-375.
19. Edyburn, D.L. (1988), “Examining the successful retrieval of information by students using online databases”, School Library Media Quarterly, Vol. 16 No. 4, pp. 256-259.
20. Egan, D.E., Remde, J.R., Gomez, L.M., Landauer, T.K., Eberhardt, J. and Lochbaum, C.C. (1989), “Formative design-evaluation of SuperBook”, ACM Transactions on Information Systems, Vol. 7 No. 1, pp. 30-57.
21. Elsweiler, D. and Ruthven, I. (2007), “Towards task-based personal information management evaluations”, Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 23-30.
22. Evans, B.M., Kairam, S. and Pirolli, P. (2010), “Do your friends make you smarter?: an analysis of social strategies in online information seeking”, Information Processing & Management, Vol. 46 No. 6, pp. 679-692.
23. Feild, H.A., Allan, J. and Jones, R. (2010), “Predicting searcher frustration”, Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 34-41.
24. Ford, N., Miller, D. and Moss, N. (2002), “Web search strategies and retrieval effectiveness: an empirical study”, Journal of Documentation, Vol. 58 No. 1, pp. 30-48.
25. Ford, N., Miller, D. and Moss, N. (2005), “Web search strategies and human individual differences: cognitive and demographic factors, internet attitudes, and approaches”, Journal of the American Society for Information Science and Technology, Vol. 56 No. 7, pp. 741-756.
26. Foss, E., Druin, A., Brewer, R., Lo, P., Sanchez, L., Golub, E. and Hutchinson, H. (2012), “Children’s search roles at home: implications for designers, researchers, educators, and parents”, Journal of the American Society for Information Science and Technology, Vol. 63 No. 3, pp. 558-573.
27. Fraser, L. and Locatis, C. (2001), “Effects of link annotations on search performance in layered and unlayered hierarchically organized information spaces”, Journal of the American Society for Information Science & Technology, Vol. 52 No. 14, pp. 1255-1261.
28. Freund, L. and Wildemuth, B.M. (2014), “Documenting and studying the use of assigned search tasks: RepAST”, Proceedings of the Annual Meeting of the American Society for Information Science & Technology, Vol. 51.
29. Gwizdka, J. and Spence, I. (2006), “What can searching behavior tell us about the difficulty of information tasks? A study of web navigation”, Proceedings of the Annual Meeting of the American Society for Information Science & Technology, Vol. 43 No. 1, pp. 1-22.
30. He, D., Wu, D., Yue, Z., Fu, A. and Vo, K.T. (2012), “Undergraduate students’ interaction with online information resources in their academic tasks”, Aslib Proceedings: New Information Perspectives, Vol. 64 No. 6, pp. 615-640.
31. Hirsh, S.G. (1996), “The effect of domain knowledge on elementary school children, information retrieval behavior on an automated library catalog”, unpublished doctoral dissertation, Department of Library and Information Science, University of California, Los Angeles, CA.
32. Hirsh, S.G. (1997), “How do children find information on different types of tasks? Children’s use of the science library catalog”, Library Trends, Vol. 45 No. 4, pp. 725-745.
33. Hughes-Morgan, K. and Wilson, M.L. (2012), “Information vs interaction: examining different interaction models over consistent metadata”, Proceedings of the 4th Information Interaction in Context Symposium, pp. 72-81.
34. Iivonen, M. (1995a), “Consistency in the selection of search concepts and search terms”, Information Processing & Management, Vol. 31 No. 2, pp. 173-190.
35. Iivonen, M. (1995b), “Factors lowering the consistency in online searching”, Proceedings of 19th International Online Information Meeting, Online Information, pp. 101-107.
36. Ingwersen, P. and Jarvelin, K. (2005), The Turn: Integration of Information Seeking and Retrieval in Context, Springer, Dordrecht.
37. Jansen, B.J., Booth, D. and Smith, B. (2009), “Using the taxonomy of cognitive learning to model online searching”, Information Processing & Management, Vol. 45 No. 6, pp. 643-663.
38. Joho, H. and Jose, J.M. (2006), “Slicing and dicing the information space using local contexts”, Proceedings of the 1st International Conference on Information Interaction in Context (IIiX), pp. 66-74.
39. Kammerer, Y., Nairn, R., Pirolli, P. and Chi, E.H. (2009), “Signpost from the masses: learning effects in an exploratory social tag search browser”, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 625-634.
40. Khan, K. and Locatis, C. (1998), “Searching through cyberspace: the effects of link cues and correspondence on information retrieval from hypertext on the world wide web”, Journal of the American Society for Information Science, Vol. 49 No. 14, pp. 1248-1253.
41. Kelly, D. (2009), “Methods for evaluating interactive information retrieval systems with users”, Foundations and Trends in Information Retrieval, Vol. 3 Nos 1/2, pp. 1-224.
42. Kim, J. (2009), “Describing and predicting information-seeking behavior on the web”, Journal of the American Society for Information Science & Technology, Vol. 60 No. 4, pp. 679-693.
43. Kim, Y.-M. and Rieh, S.Y. (2005), “Dual-task performance as a measure for mental effort in library searching and web searching”, Proceedings of the 68th Annual Meeting of the American Society for Information Science & Technology (unpublished).
44. Kumpulainen, S. and Jarvelin, K. (2010), “Information interaction in molecular medicine: integrated use of multiple channels”, Proceedings of the Third Symposium on Information Interaction in Context (IIiX’10), pp. 95-104.
45. Large, A., Beheshti, J., Breuleux, A. and Renaud, A. (1994), “A comparison of information retrieval from print and CD-ROM versions of an encyclopedia by elementary school children”, Information Processing & Management, Vol. 30 No. 4, pp. 499-513.
46. Lazonder, A.W., Biemans, H.J.A. and Wopereis, I.G.J.H. (2000), “Differences between novice and experienced users in searching information on the World Wide Web”, Journal of the American Society for Information Science, Vol. 51 No. 6, pp. 576-581.
47. Li, Y. (2009), “Exploring the relationships between work task and search task in information search”, Journal of the American Society for Information Science & Technology, Vol. 60 No. 2, pp. 275-291.
48. Liu, J., Kim, C.S. and Creel, C. (2013), “Why do users feel search task difficult?”, Proceedings of the American Society for Information Science & Technology, Vol. 50 No. 1, pp. 1-4.
49. Liu, J., Gwizdka, J., Liu, C. and Belkin, N. (2010), “Predicting task difficulty for different task types”, Proceedings of the American Society for Information Science & Technology, Vol. 47 (unpublished).
50. Liu, J., Liu, C., Cole, M., Belkin, N.J. and Zhang, X. (2012), “Exploring and predicting search task difficulty”, Proceedings of the 21st ACM International Conference on Information and Knowledge Management, pp. 1313-1322.
51. Liu, P. and Li, Z. (2012), “Task complexity: a review and conceptualization framework”, International Journal of Industrial Ergonomics, Vol. 42, pp. 553-568.
52. Lopatovska, I. (2009), “Searching for good mood: examining relationships between search task and mood”, Proceedings of the American Society for Information Science & Technology, Vol. 46 No. 1, pp. 1-13.
53. Mack, T., Manoff, M., Miller, T.J. and Smith, A.D. (2004), “Designing for experts: how scholars approach an academic library web site”, Information Technology & Libraries, Vol. 23 No. 1, pp. 16-22.
54. Madden, A.D., Eaglestone, B., Ford, N.J. and Whittle, M. (2007), “Search engines: a first step to finding information: preliminary findings from a study of observed searches”, Information Research, Vol. 12 No. 2, paper 294.
55. Marchionini, G. and Liebscher, P. (1991), “Performance in electronic encyclopedias: implications for adaptive systems”, Proceedings of the American Society for Information Science, Vol. 54, pp. 39-48.
56. Maynard, D.C. and Hakel, M.D. (1997), “Effects of objective and subjective task complexity on performance”, Human Performance, Vol. 10 No. 4, pp. 303-330.
57. Mirel, B., Tonks, J.S., Song, J., Meng, F., Xuan, W. and Ameziane, R. (2013), “Studying PubMed usages in the field for complex problem solving: implications for tool design”, Journal of the American Society for Information Science & Technology, Vol. 64 No. 5, pp. 874-892.
58. Pharo, N. (2002), The SST Method Schema: A Tool for Analysing Work, Task-Based Web Information Search Processes, University of Tampere, Tampere.
59. Pirolli, P., Card, S.K. and van der Wege, M.M. (2003), “The effects of information scent on visual search in the hyperbolic tree browser”, ACM Transactions on Computer-Human Interaction, Vol. 10 No. 1, pp. 20-53.
60. Pirolli, P., Schank, P., Hearst, M. and Diehl, C. (1996), “Scatter/gather browsing communicates the topic structure of a very large text collection”, Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, pp. 213-220.
61. Sahib, N.G., Tombros, A. and Stockman, T. (2012), “A comparative analysis of the information-seeking behavior of visually impaired and sighted searchers”, Journal of the American Society for Information Science & Technology, Vol. 63 No. 2, pp. 377-391.
62. Schacter, J., Chung, G.K.W.K. and Dorr, A. (1998), “Children’s internet searching on complex problems: performance and process analyses”, Journal of the American Society for Information Science, Vol. 49 No. 9, pp. 840-849.
63. Sharit, J., Hernandez, M.A., Czaja, S.J. and Pirolli, P. (2008), “Investigating the roles of knowledge and cognitive abilities in older adult information seeking on the web”, ACM Transactions on Computer-Human Interaction, Vol. 15 No. 1, pp. 1-25.
64. thInformation Interaction in Context (IIiX) Symposium, pp. 110-119.
65. Singer, G., Norbisrath, U. and Lewandowski, D. (2013), “Ordinary search engine users carrying out complex search tasks”, Journal of Information Science, Vol. 39 No. 3, pp. 346-358.
66. Toms, E.G., O’Brien, H., Mackenzie, T., Jordan, C., Freund, L., Toze, S., Dawe, E. and MacNutt, A. (2008), “Task effects on interactive search: the query factor”, Focused Access to XML Documents. Lecture Notes in Computer Science, Vol. 4862, pp. 359-372.
67. Topi, H. and Lucas, W. (2005), “Searching the web: operator assistance required”, Information Processing & Management, Vol. 41 No. 2, pp. 383-403.
68. Vakkari, P. (2001), “Changes in search tactics and relevance judgements when preparing a research proposal: a summary of the findings of a longitudinal study”, Information Retrieval, Vol. 4 Nos 3/4, pp. 295-310.
69. Vakkari, P. (2003), “Task-based information searching”, Annual Review of Information Science & Technology, Vol. 37 No. 1, pp. 413-464.
70. Vakkari, P. and Huuskonen, S. (2012), “Search effort degrades search output but improves task outcome”, Journal of the American Society for Information Science & Technology, Vol. 63 No. 4, pp. 657-670.
71. Vakkari, P., Pennanen, M. and Serola, S. (2003), “Changes of search terms and tactics while writing a research proposal: a longitudinal study”, Information Processing & Management, Vol. 39 No. 3, pp. 445-463.
72. Wacholder, N. and Liu, L. (2006), “User preference: a measure of query-term quality”, Journal of the American Society for Information Science & Technology, Vol. 57 No. 12, pp. 1566-1580.
73. White, M.D. and Iivonen, M. (2002), “Assessing the level of difficulty of web search questions”, Library Quarterly, Vol. 72 No. 2, pp. 207-233.
74. Wildemuth, B.M. and Freund, L. (2009), “Search tasks and their role in studies of search behaviors”, Paper presented at HCIR 2009: Bridging Human-Computer Interaction and Information Retrieval, Washington, DC, October 23, available at: http://ils.unc.edu/searchtasks/publication/publication_1.pdf
75. Wildemuth, B.M., Freund, L. and Toms, E.G. (2014), “Studies of search task complexity or difficulty”, SILS Technical Report 2014-01, School of Information & Library Science, University of North Carolina, Chapel Hill, NC.
76. Williams, M.E., Kinnucan, M., Smith, L.C., Lannom, L. and Cho, D. (1986), “Comparative analysis of online retrieval interfaces”, Proceedings of the 49th ASIS Annual Meeting, pp. 365-370.
77. Wu, W., Kelly, D., Edwards, A. and Arguello, J. (2012), “Grannies, tanning beds, tattoos and NASCAR: evaluation of search tasks with varying levels of cognitive complexity”, Proceedings of the 4th Information Interaction in Context Symposium, pp. 254-257.
78. Zhang, Y. (2012), “The impact of task complexity on people’s mental models of MedlinePlus”, Information Processing & Management, Vol. 48 No. 1, pp. 107-119.