Task-based information retrieval: Structuring undergraduate history essays for better course evaluation using essay-type visualizations

Journal of the American Society for Information Science and Technology

Volumn 58, Issue 9, 2007, Pages 1227-1241

  Leide, John E ^a   Cole, Charles ^a   Beheshti, Jamshid ^a   Large, Andrew ^a   Yang, Lin ^a  

^a MCGILL UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

CURRICULA; ENGINEERING EDUCATION; INFORMATION SYSTEMS; INFORMATION THEORY; QUERY LANGUAGES; STUDENTS;

ESSAY-TYPE VISUALIZATIONS; TASK-FOCUSED QUERY-FORMULATION DEVICES; TOPIC-BASED QUERY;

INFORMATION RETRIEVAL;

EID: 34547306627     PISSN: 15322882     EISSN: 15322890     Source Type: Journal    
DOI: 10.1002/asi.20598     Document Type: Article

References (112)

1. Alemayehu, N. (2003). Analysis of performance variation using query expansion. Journal of the American Society for Information Science and Technology, 54(5), 379-391.
2. Annett, J., Duncan, K.D., Stammers, R.B., & Gray, M.J. (1971). Task analysis. London: HMSO.
3. Baker, S. (1991). The practical stylist (3rd ed.). New York: HarperCollins.
4. Bartlett, J.C. (2004). Connecting bioinformatics analysis to scientific practice: An integrated information behaviour and task analysis approach. Unpublished doctoral dissertation, Toronto: University of Toronto Faculty of Information Studies.
5. Bates, M.J. (1979). Information search tactics. Journal of the American Society for Information Science, 30, 205-214.
6. Bates, M.J. (1990). Where should the person stop and the information search interface start? Information Processing and Management, 26(5), 575-801.
7. Belkin, N., Cool, C., Stein, A., & Thiel, U. (1995). Cases, scripts and information-seeking strategies: On the design of interactive information retrieval systems. Expert Systems With Applications, 9(3), 379-395.
8. Belkin, N.J., & Marchetti, P.G. (1990). Determining the functionality and features of an intelligent interface to an information retrieval system. Proceedings of the 13th International Conference on Research and Development in Information Retrieval: Universite libre de Bruxelles (pp. 151-177). Brussels, Belgium: Universite libre de Bruxelles.
9. Belkin, N.J., Oddy, R.N., & Brooks, H.M. (1982a). ASK for information retrieval: Part I. Background and theory. Journal of Documentation, 38(2), 61-71.
10. Belkin, N.J., Oddy, R.N., & Brooks, H.M. (1982b). ASK for information retrieval: Part II. Results of a design study. Journal of Documentation, 38(3), 145-164.
11. Belkin, N.J., Seeger, R., & Wersig, G. (1983). Distributed expert problem treatment as a model for information system analysis and design. Journal of Information Science, 5(5), 153-167.
12. Benyon, D.R. (1992). The role of task analysis in systems design. Interacting With Computers, 4(2), 102-123.
13. Byström, K., & Järvelin, K. (1995). Task complexity affects information seeking and use. Information Processing and Management, 31(2), 191-213.
14. Card, S., Moran, T., & Newell, A. (1983). The psychology of human-computer interaction. Hillsdale, NJ: Erlbaum.
15. Case, D.O. (2002). Looking for information: A survey of research on information seeking, needs, and behavior. Amsterdam: Academic Press.
16. Cole, C. (1994). Operationalizing the notion of information as a subjective construct. Journal of the American Society for Information Science, 45(7), 465-476.
17. Cole, C. (2000). Inducing expertise in history doctoral students via enabling information retrieval design. Library Quarterly, 70(1), 86-109.
18. Cole, C., Beheshti, J., Leide, J.E., & Large, A. (2005). Interactive information retrieval: Bringing the user to a selection state. In A. Spink & C. Cole (Eds.), New directions in cognitive information retrieval (pp. 13-41). Dordrecht, The Netherlands: Springer.
19. Cole, C., Cantero, P., & Sauve, D. (1998). Intelligent information retrieval: Diagnosing information need: Part II. Uncertainty expansion in a proto-type of a diagnostic IR tool. Information Processing & Management, 34(6), 721-737.
20. Cole, C., Kennedy, L., & Carter, S. (1996). The optimization of online searches through the labeling of a dynamic, situation-dependent information need: The reference interview and online searching for undergraduates doing a social science assignment. Information Processing & Management, 32, 709-717.
21. Cole, C., & Kuhlthau, C. (2000). Information and information seeking of novice versus expert lawyers: How experts add value. The New Review of Information Behaviour Research, 1, 103-115.
22. Cole, C., & Leide, J. (2003). Using the user's mental model to guide integration of information space into information need. Journal of the American Society for Information Science and Technology, 54(1), 39-46.
23. Cole, C., Leide, J.E., Beheshti, J., Large, A., & Brooks, M. (2005). Investigating the ASK hypothesis in a real-life problem situation: A study of history and psychology undergraduates seeking information for a course essay. Journal of the American Society for Information Science and Technology, 56(14), 1544-1554.
24. Cole, C., Leide, J.E., Large, A., Beheshti, J., & Brooks, M. (2005). Putting it together online: Information need identification for the domain novice user. Journal of the American Society for Information Science and Technology, 56(7), 684-694.
25. Collegeboard.com Inc. (2007). Collegeboard Scoring Guide. Retrieved August 18, 2005, from http://www.collegeboard.com/student/testing/sat/about/sat/ essay_scoring.html
26. Crouch, C.J., Crouch, D.B., Chen, Q., & Holtz, S.J. (2002). Improving the retrieval effectiveness of very short queries. Information Processing & Management, 38(1), 1-36.
27. Crystal, A., & Ellington, B. (2004). Task analysis and human-computer interaction: Approaches, techniques, and levels of analysis. Proceedings of the 10th Americas Conference on Information Systems, New York (pp. 1-9). Retrieved from http://www.ils.unc.edu/Nacrystal/AMC1504_crystal_ellington_final.pdf
28. Culnan, M.J. (1983). Environmental scanning: The effects of task complexity and source accessibility on information gathering behavior. Decision Sciences, 14, 194-206.
29. Drabenstott, K.M. (2003). Do nondomain experts enlist the strategies of domain experts? Journal of the American Society for Information Science and Technology, 54(9), 836-854.
30. Ellis, D. (1996). The dilemma of measurement in information retrieval research. Journal of the American Society for Information Science, 47(1), 23-36.
31. Ellis, D., Cox, D., & Hall, K. (1993). A comparison of the information seeking patterns of researchers in the physical and social sciences. Journal of Documentation, 49(4), 356-369.
32. Entwistle, N., & Entwistle, D. (2003). Preparing for examinations: The interplay of memorizing and understanding, and the development of knowledge objects. Higher Education Research and Development, 22, 19-41.
33. Fidel, R. (1991a). Searchers' selection of search keys: I. Journal of the American Society for Information Science, 42(7), 490-500.
34. Fidel, R. (1991b). Searchers' selection of search keys: II. Journal of the American Society for Information Science, 42(7), 501-514.
35. Fidel, R. (1991c). Searchers' selection of search keys: III. Journal of the American Society for Information Science, 42(7), 515-527.
36. Fisher, K.E., Durrance, J.C., & Hinton, M.B. (2004). Information grounds and the use of need-based services by immigrants in Queens, New York: A context-based, outcome evaluation approach. Journal of the American Society for Information Science and Technology, 55(8), 754-766.
37. Fujihara, N., & Miura, A. (2003). Patterns of searching for information on the World Wide Web: A pilot study. Psychological Reports, 92, 1091-1096.
38. Gaddis, J.L. (2002). The landscape of history: How historians map the past. Oxford, England: Oxford University Press.
39. Hackman, J.R. (1969). Toward understanding the role of tasks in behavioral research. Acta Psychologica, 31, 97-128.
40. Hart, P.J., & Rice, R.E. (1991). Using information from external databases: Contextual relationships of use, access method, task, database type, organizational differences, and outcomes. Information Processing & Management, 27(5), 461-479.
41. Harter, S.P. (1986). Online information retrieval: Concepts, principles, and techniques. San Diego, CA: Academic Press.
42. Harter, S.P. (1992). Psychological relevance and information science. Journal of the American Society for Information Science, 43(9), 602-615.
43. Harter, S.P. (1996). Variations in relevance assessments and the measurement of retrieval effectiveness. Journal of the American Society for Information Science, 47(1), 37-49.
44. Harter, S.P., & Hert, C. (1997). Evaluation of information retrieval systems: Approaches, issues, and methods. Annual Review of Information Science and Technology, 32, 3-94.
45. Hearst, M.A. (1999). User interfaces and visualization. In B.-Y. Ricardo & R.-N. Berthier (Eds.), Modern information retrieval (pp. 257-323). New York: ACM Press.
46. Hellstern, M., Scott, G.M., & Garrison, S.M. (1998). The history student writer's manual. Upper Saddle River, NJ: Prentice Hall.
47. Hembrooke, H.A., Granka, L.A., Gay, G.K., & Liddy, E.D. (2005). The effects of expertise and feedback on search term selection and subsequent learning. Journal of the American Society for Information Science and Technology, 56(8), 861-871.
48. Hersh, W., Pentecost, J., & Hickam, D. (1996). A task-oriented approach to information retrieval evaluation. Journal of the American Society for Information Science, 47(1), 50-56.
49. Hildreth, CR. (2001). Accounting for users' inflated assessments of online catalogue search performance and usefulness: An experimental study. Information Research, 6(2). Retrieved April 18, 2006, from http://InformationR. net/ir/6-2/paper101.html
50. Holscher, C., & Strube, G. (2000). Web search behavior of Internet experts and newbies. International Journal of Computer and Telecommunications Networking, 33(1-6), 337-346.
51. Howard University. (2003). Essay styles. Howard University College of Arts and Sciences, Washington, DC. Retrieved July 8, 2003, from http://www.howard.edu/CollegeArtsSciences/Freshman%20Seminar/essay_styles.htm
52. Hsieh-Yee, I. (1993). Effects of search experience and subject knowledge on the search tactics of novice and experienced searchers. Journal of the American Society for Information Science, 45(3), 161-174.
53. Hyperhistory.net. (2005). Essay Scoring Guides (Rubrics). Retrieved August 18, 2005, from http://www.hyperhistory.net/apwh/exam/all3rubrics.htm
54. Ingwersen, P. (1996). Cognitive perspectives of information retrieval inter-action: Elements of a cognitive IR theory. Journal of Documentation, 52(1), 3-50. Jansen, B.J., & McNeese, M.D. (2005). Evaluating the effectiveness of and patterns of interactions with automated searching assistance. Journal of the American Society for Information Science and Technology, 56(14), 1480-1503.
55. Järvelin, K., & Ingwersen, P. (2004). Information seeking research needs extension towards tasks and technology. Information Research, 10(1), Paper No. 212. Available: http://informationR.net/ir/10-1/paper212.html
56. Kennedy, L., Cole, C., & Carter, S. (1999). The false focus in online searching: The particular case of undergraduates seeking information for course assignments in the humanities and social sciences. Reference & User Services Quarterly, 38, 267-273.
57. Kennedy, M.L., & Smith, H.M. (1994). Reading and writing in the academic community. Englewood Cliffs, NJ: Prentice Hall.
58. Kim, K.-S. (2001). Information-seeking on the Web: Effects of user and task variables. Library & Information Science Research, 23(3), 233-257.
59. Kirwan, B., & Ainsworth, L.K. (Eds.). (1992). A guide to task analysis. London: Taylor & Francis.
60. Kuhlthau, C.C. (1993). Seeking meaning: A process approach to library and information services. Norwood, NJ: Ablex.
61. Kuhlthau, CC. (1999). Accommodating the user's information search process: Challenges for information retrieval system designers. Bulletin of the American Society for Information Science, 25(3), 12-16.
62. Kuhlthau, CC (2004). Seeking meaning: A process approach to library and information services (2nd ed.). Westport, CT: Libraries Unlimited.
63. Leckie, G.J., Pettigrew, K.E., & Sylvain, C. (1996). Modelling the information seeking of professionals: A general model derived from research on engineers, health care professionals, and lawyers. Library Quarterly, 66(2), 161-193.
64. Leide, J., Large, A., Beheshti, J., Brooks, M., & Cole, C. (2003). Visualization schemes for domain novices exploring a topic space: The navigation classification scheme. Information Processing & Management, 39(6), 923-940.
65. Leplat, J. (1989). Relations between task and activity: Elements of elaborating a framework for error analysis. Ergonomics, 33, 1389-1402.
66. Line, M.B. (1971). The information uses and needs of social scientists: An overview of INFROSS. Aslib Proceedings, 23(8), 412-434.
67. Locke, E.A., Shaw, K.N., Saari, L.M., & Latham, G.P. (1981). Goal setting and task performance: 1969-1980. Psychological Bulletin, 90, 125-152.
68. Lucas, W., & Topi, H. (2002). Form and function: The impact of query term and operator usage on Web search results. Journal of the American Society for Information Science and Technology, 53(2), 95-108.
69. March, J., & Simon, H. (1967). Organizations (2nd ed.). New York: Wiley.
70. Marchionini, G. (1995). Information seeking in electronic environments. Cambridge, England: Cambridge University Press.
71. Markkula, M., Tico, M., Sepponen, B., Nirkkonen, K., & Sormunen, E. (2001). A test-collection for the evaluation of content-based image retrieval algorithms - A user and task-based approach. Information Retrieval, 4(3/4), 275-293.
72. McGuen, J.R., & Winkler, A.C. (1995). From idea to essay: A rhetoric, reader, and handbook (7th ed.). Toronto: Allyn & Bacon.
73. North Central Regional Technology in Education Consortium. Scoring Guide for Student Products. Retrieved April 10, 2007, from http://goal.learningpt.org/ spsg/GetProd.asp
74. O'Day, V.L., & Jeffries, R. (1993). Orienteering in an information landscape: How information seekers get from here to there. In INTERCHI'93 Proceedings of the Conference on Human Factors in Computing Systems, Amsterdam (pp. 438-445). New York: ACM Press.
75. Oddy, R.N., Liddy, E.D., Balakrishnan, B., Bishop, A., Elewononi, J., & Martin, E. (1992). Towards the use of situational information in information retrieval. Journal of Documentation, 48(2), 123-171.
76. Oregon Department of Education. (2004-2005). Writing Scoring Guide. Retrieved August 11, 2005, from http://www.ode.state.or.us/teachlearn/testing/ scoring/guides/2004-05/writingscoringguide0405.pdf
77. Patel, V., & Ramoni, M. (1997). Cognitive models of directional inference in expert medical reasoning. In P. Feltovich, K. Ford, & R. Hoffman (Eds.), Expertise in context: Human and machine (pp. 67-99). Menlo Park, CA: AAAI Press.
78. Reeb, B., & Gibbons, S. (2004). Students, librarians, and subject guides: Improving a poor rate of return. Libraries and the Academy, 4(1), 123-130.
79. Reid, J. (2000). A task-oriented non-interactive evaluation methodology for IR systems. Information Retrieval, 2(1), 115-129.
80. Rieh, S.Y. (2004). On the Web at home: Information seeking and Web searching in the home environment. Journal of the American Society for Information Science and Technology, 55(8), 743-753.
81. Robertson, H. (1995). The research essay: A guide to essays and papers (3rd ed.). Montreal: McGraw-Hill.
82. Ruthven, I. (2005), Integrating approaches to relevance. In A. Spink & C. Cole (Eds.), New directions in cognitive information retrieval (pp. 61-80). Dordrecht, The Netherlands: Springer.
83. Saracevic, T. (1996a). Interactive models in information retrieval (IR): A review and proposal. Proceedings of the 80th annual meeting of the American Society of Information Science, 33, 3-9.
84. Saracevic, T. (1996b). Relevance reconsidered '96. In P. Ingwersen & N.O. Pors (Eds.), Proceedings of the 2nd International Conference on Conceptions of Library and Information Science: Integration in Perspective (pp. 201-218). Copenhagen, Denmark: The Royal School of Librarianship.
85. Savolainen, R. (1995). Everyday life information seeking: Approaching information seeking in the context of way of life. Library and Information Science Research, 17, 259-294.
86. Schamber, L., Eisenberg, M.B., & Nilan, M.S. (1990). A reexamination of relevance: Toward a dynamic, situational definition. Information Processing & Management, 26, 755-776.
87. Spink, A., & Cole, C (2001). Everyday life information seeking research. Library and Information Science Research, 23(4), 301-304.
88. Spink, A., & Cole, C (2005). A multitasking framework for cognitive information retrieval. In A. Spink & C. Cole (Eds.), New directions in cognitive information retrieval (pp. 99-112). Dordrecht, The Netherlands: Springer.
89. Spink, A., & Jansen, B.J. (2004). Web search: Public searching of the Web. Dordrecht, The Netherlands: Kluwer.
90. Stieg, M.F. (1981). The information needs of historians. College & Research Libraries, 42, 549-560.
91. Stoan, S.K. (1984). Research and library skills: An analysis and interpretation. College & Research Libraries, 45, 99-109.
92. Sutton, S. (1994). The role of attorney mental models of law in case relevance determinations: An exploratory analysis. Journal of the American Society for Information Science, 45(3), 186-200.
93. Taylor, F. (1911). Scientific management. New York: Harper & Row.
94. Tiamiyu, M.A. (1992). The relationships between source use and work complexity, decision-maker discretion and activity duration in Nigerian government ministries. International Journal of Information Management, 12, 130-141.
95. Troll, D.A. (2002). How and why libraries are changing: What we know and what we need to know. Library and the Academy, 2(1), 99-123.
96. Tushman, M.L. (1978). Technical communication in R&D laboratories: The impact of project work characteristics. Academy of Management Journal, 21(4), 624-645.
97. University of North Carolina at Chapel Hill. (1998). Writing the paper. Handouts and links. Retrieved April 12, 2006, from http://www.unc.edu/depts/ wcweb/handouts/comparison_contrast.html
98. University of Toronto, Undergraduate Studies in History. The preparation of historical essays. Retrieved August 18, 2005, from http://www.chass.utoronto. ca/history/undergraduate/essays.html
99. Vakkari, P. (1999). Task complexity, problem structure and information actions integrating studies on information seeking and retrieval. Information Processing & Management, 35, 819-837.
100. Vakkari, P. (2000a). Cognition and changes of search terms and tactics during task performance: A longitudinal case study. Proceedings of the RIAO 2000 conference (pp. 894-907). Paris: C.I.D. 2000.
101. Vakkari, P. (2000b). Relevance and contributing information types of searched documents in task performance. Proceedings of the 23rd annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 2-9).
102. Vakkari, P. (2003). Task-based information searching. Annual Review of Information Science and Technology, 37, 413-464.
103. Vakkari, P., & Hakala, N. (2000). Changes in relevance criteria and problem stages in task performance. Journal of Documentation, 56(5), 540-562.
104. Vakkari, P., & Järvelin, K. (2005). Explanation in information seeking and retrieval. InA. Spink & C. Cole (Eds.), New directions in cognitive information retrieval (pp. 113-138). Dordrecht, The Netherlands: Springer.
105. Van de Ven, Á., & Ferry, D. (1980). Measuring and assessing organizations. New York: Wiley.
106. Wersig, G. (1973). Informationssoziologie: Hinweise zu einem informationswissenschaftlichen Teilbereich [Information sociology: References to an information science specialty]. Frankfurt aM: Athenäum Fischer.
107. Western International University. (2004). Writing Style Handbook. Retrieved August 1, 2005, from http://www.wintu.edu/pdf/writingstyleshandbook. pdf
108. Wildmuth, B.M. (2004). The effects of domain knowledge on search tactic formulation. Journal of the American Society for Information Science and Technology, 55(3), 246-258.
109. Wilson, T.D. (1997). Information behavior: An interdisciplinary perspective. Information Processing & Management, 33(4), 551-572.
110. Wilson, T.D. (1999). Models in information behaviour research. Journal of Documentation, 55, 249-270.
111. Wood, R.E., Mento, A.J., & Locke, E.A. (1987). Task complexity as a moderator of goal effects: A meta-analysis. Journal of Applied Psychology, 72(3), 416-425.
112. Yi, K., Beheshti, J., Cole, C., Leide, J.E., & Large, A. (2006). User search behavior of domain-specific IR systems: An analysis of the query logs from PsycINFO and ABC-Clio's Historical Abstracts/America: History and Life, Journal of the American Society for Information Science and Technology, 57(9), 1208-1220.