Using the information seeker to elicit construct models for search engine evaluation

Journal of the American Society for Information Science and Technology

Volumn 55, Issue 9, 2004, Pages 794-806

  Crudge, Sarah E ^a   Johnson, Frances C ^a  

^a MANCHESTER METROPOLITAN UNIVERSITY   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER AIDED DESIGN; COMPUTER PROGRAMMING LANGUAGES; COMPUTER SYSTEMS; INFORMATION RETRIEVAL; MATHEMATICAL MODELS; QUERY LANGUAGES; USER INTERFACES;

CAR MODELS; ELICIT CONSTRUCT MODELS; INFONRTICS; RETRIEVAL SYSTEMS;

SEARCH ENGINES;

EID: 3042636214     PISSN: 15322882     EISSN: None     Source Type: Journal    
DOI: 10.1002/asi.20023     Document Type: Article

References (54)

1. Applegate, R. (1993). Models of user satisfaction: Understanding false positives. Research Quarterly, 32(4), 525-539.
2. Bates, M.J. (1989). The design of browsing and berrypicking techniques for the online search interface. Online Review, 13(5), 407-424.
3. Bates, M.J. (1990). When should the person stop and the information search interface start? Information Processing and Management, 26(5), 575-591.
4. Beaulieu, M., Robertson, S.,&Rasmussen, E. (1996). Evaluating interac tive systems in TREC. Journal of the American Society for Information Science, 47(1), 85-94.
5. Belkin, N.J., Cabezas, A., Cool, C., Lin, S., Park, S., Rieh, S., et al. (1997). Rutgers' TREC-5 interactive track experience. In D. Harman (Ed.), TREC-5 proceedings of the Fifth Text Retrieval Conference (pp. 257-266). Washington, DC: Government Printing Office.
6. Belkin, N.J., Cool, C., Kelly, D., Lin, S.-J., Park, S.Y., Perez-Carballo, J.,&Sikora, C. (2001). Iterative exploration, design and evaluation of support for query reformulation in interactive information retrieval. Information Processing and Management, 37(3), 403-434.
7. Belkin, N.J., Cool, C., Stein, A.,&Thiel, U. (1995). Cases, scripts, and information-seeking strategies: On the design of interactive information retrieval systems. Expert Systems with Applications, 9(3), 379-395.
8. Burke, M. (2001). The use of repertory grids to develop a user-driven classi fication of a collection of digitized photographs. Proceedings of the 64th ASIST Annual Meeting (pp. 76-92). Medford, NJ: Information Today Inc.
9. Caputi, P.,&Reddy, P. (1999). A comparison of triadic and dyadic methods of personal construct elicitation. Journal of Constructivist Psy chology, 12(3), 253-264.
10. Chen, H.,&Dhar, V. (1991). Cognitive process as a basis for intelligent retrieval system design. Information Processing&Management, 27(5), 405-432.
11. Dillon, A. (1994). Designing usable electronic text: Ergonomic aspects of human information usage. London: Taylor and Francis.
12. Dillon, A.,&McKnight, C. (1990). Towards a classification of text types: A repertory grid approach. International Journal of Man-Machine Stud ies, 33(6), 623-636.
13. D'Souza, D., Fuller, M., De Kretser, O., Wilkinson, R.,&Wu, M. (2001). Melbourne TREC-9 Experiments. The Ninth Text REtrieval Conference (TREC-9). Retrieved March 15, 2002, from http://trec.nist.gov/pubs/trec9/ t9_proceedings.html
14. Dunn, W.N. (1986). The policy grid: A cognitive methodology for assess ing policy dynamics. In W.N. Dunn (Ed.), Policy analysis: Perspectives, concepts and methods (pp.355-375). Greenwich, CT: JAI Press.
15. Ellis, D. (1989). A behavioural approach to information retrieval system design. Journal of Documentation, 45, 171-212.
16. Fransella, F.,&Bannister, D. (1977). A manual for repertory grid tech nique. London: Academic Press.
17. Gaines, B.R.,&Shaw, M.L.G. (2001). WebGrid II. Retrieved September 2002, from http://www.repgrid.com/WebGrid/
18. Harter, S. P. (1984). Scientific inquiry: A model for online searching. Journal of the American Society for Information Science, 35(2), 110-117.
19. Hassenzahl, M.,&Trautmann, T. (2001, April). Analysis of Web sites with the repertory grid technique. Poster session presented at the CHI'2001 Conference on Human Factors in Computing Systems, Seattle, Wash ington, March 31-April 5, 2001. Retrieved from http:/www.tu-darmstadt.de/ fb/fb3/psy/soz/ veroeffentlichungen_mh/Chi01_hass_rgt.pdf
20. Hildreth, C.R. (2001). Accounting for users' inflated assessments of on-line catalogue search performance and usefulness: An experimental study. Information Research, 6(2). Retrieved November 18, 2001, from http://www.shef.ac.uk/~is/publications/infres/6-2/paper101.html
21. Hunter, M.G. (1997). The use of RepGrids to gather interview data about information systems analysts. Information Systems Journal, 7, 67-81.
22. Infonortics. (2003). Search engine meeting. Retrieved from http://www. infonortics.com/searchengines/
23. Johnson, F.C., Griffiths, J.R.,&Hartley, R.J. (2001). DEVISE: A frame work for the evaluation of Internet search engines (Library and Infor mation Commission Research Report No. 100). London: The Council for Museums, Archives and Libraries.
24. Johnson, P.C., Griffiths, J.R.,&Hartley, R.J. (2003). Task dimensions of user evaluations of information retrieval systems. Information Research, 8(4). Retrieved from http://informationr.net/ir/8-4/paper157.html
25. Kelly, G.A. (1991). The psychology of personal constructs. London: Routledge. (Original work published 1955).
26. Kuhlthau, C.C. (1991). Inside the search process: Information seeking from the users' perspective. Journal of the American Society for Information Science, 42(5), 361-371.
27. Kuhlthau, C.C. (1993). Seeking meaning: A process approach to library and information services. Norwood, NJ: Ablex.
28. Latta, G.F.,&Swigger, K. (1992). Validation of the repertory grid for use in modeling knowledge. Journal of the American Society for Informa tion Science, 43(2), 115-129.
29. McKnight, C. (2000). The personal construction of information space. Journal of the American Society for Information Science, 51(8), 730-733.
30. Moynihan, T. (1996). An inventory of personal constructs for information systems project risk researchers. Journal of Information Technology, 11, 359-371.
31. NUA. (2002). Net population to near 950 million by 2005. Retrieved March 29, 2002, from http://www.nua.ie/surveys/index.cgi?f=VS&art_id= 905357750&rel=true
32. O'Day, V.L.,&Jeffries, R. (1993). Orienteering in an information landscape: How information seekers get from here to there. Amsterdam, The Netherlands, April 24-29. Proceedings of ACM InterCHI '93, 438-445.
33. Park, S. (2000). Usability, user preferences, effectiveness, and user behav iours when searching individual and integrated full-text databases: Im plications for digital libraries. Journal of the American Society for Information Science, 51(5), 456-468.
34. Russell, I.E.,&Fatmi, H.A. (1994). A novel definition of expert knowl edge in expert systems. Proceedings of IEEE International Conference on Systems, Man and Cybernetics, 2207-2211.
35. Sandore, B. (1990). Online searching: What measure satisfaction? Library and Information Science Research, 12, 33-54.
36. Saracevic, T. (1996). Modelling interaction in information retrieval: A review and proposal. Proceedings of the 59th Annual ASIS Meeting (pp. 3-9). Medford, NJ: Information Today Inc.
37. Saracevic, T. (1997). Extension and application of the stratified model of information retrieval interaction. Proceedings of the Annual Meeting of the American Society for Information Science, 34, 3-9.
38. Saracevic, T.,&Kantor, P. (1988). A study of information seeking and retrieving. III. Searchers, searches and overlap. Journal of the American Society for Information Science, 39(3), 197-216.
39. Shaw, M.L.G. (1980). On becoming a personal scientist. London: Aca demic Press.
40. Shaw, M.L.G.,&Gaines, B.R. (1996). WebGrid: Knowledge elicitation and modeling on the Web. Proceedings of WebNet96. Retrieved July 1, 2002, from http://aace.virginia.edu/aace/conf/webnet/html/209/ 209.htm
41. Sherman, C. (2000). "Old economy" information retrieval clashes with "new economy" at the Fifth Annual Search Engine Conference. Retrieved August 8, 2003, from http://www.infotoday.com/ newsbreaks/nb000424-2. htm
42. Spink, A., Jansen, B.J.,&Ozmultu, H.C. (2000). Use of query reformu lation and relevance feedback by Excite users. Internet Research, 10(4), 317-328.
43. Spink, A., Jansen, B.J., Wolfram, D.,&Saracevic, T. (2002). From E-sex to E-commerce: Web search changes. IEEE Computer, 35(3), 107-109.
44. Stewart, V.,&Stewart, A. (1981). Business applications of repertory grid. London: McGraw-Hill.
45. Su, L.T. (1992). Evaluation measures for interactive information retrieval. Information Processing and Management, 28(4), 503-516.
46. Su, L.T. (1994). The relevance of recall and precision in user evaluation. Journal of the American Society for Information Science, 45(3), 207-217.
47. Su, L.T. (1998). Value of search results as a whole as the best single measure of information retrieval performance. Information Processing and Management, 34(5), 557-579.
48. Su, L.T., Chen, H.,&Dong, X. (1998). Evaluation of Web-based search engines from the end-user's perspective: A pilot study. Proceedings of the 61st Annual ASIS Meeting (pp. 348-361). Medford, NJ: Informa tion Today Inc.
49. Wang, H., Xie, M.,&Goh, T.N. (1999). Service quality of Internet search engines. Journal of Information Science, 25(6), 499-507.
50. Whyte, G.,&Bytheway, A. (1996). Factors affecting information systems' suc cess. International Journal of Service Industry Management, 7(1), 74-93.
51. Xie, H. (2000). Shifts of interactive intentions and information-seeking strategies in interactive information retrieval. Journal of the American Society for Information Science, 51(9), 841-857.
52. Xie, H. (2003). Supporting ease-of-use and user control: Desired features and structure of Web-based online IR systems. Information Processing and Management, 39(6), 899-922.
53. Zhang, P.,&Von Dran, G.M. (2000). Satisfiers and dissatisfiers: A two factor model for Website design and evaluation. Journal of the American Society for Information Science, 51(14), 1253-1268.
54. Zhang, X.,&Chignell, M. (2001). Assessment of the effects of user characteristics on mental models of information retrieval systems. Jour nal of the American Society for Information Science and Technology, 52(6), 445-459.