"THAT'S what I was looking for": Comparing user-rated relevance with search engine rankings

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 3585 LNCS, Issue , 2005, Pages 117-129

  Patil, Sameer ^a   Alpert, Sherman R ^b   Karat, John ^b   Wolf, Catherine ^b  

^a Ringgold Standard Institution Informatics   (United States)

^b IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; BENCHMARKING; COMPUTER SCIENCE; COMPUTER SOFTWARE; FEEDBACK; INFORMATION RETRIEVAL; USER INTERFACES; WEBSITES;

CORPORATE WEB SITES; RANKING ALGORITHMS; RETRIEVAL ALGORITHMS; SEARCH ENGINE RANKINGS;

SEARCH ENGINES;

EID: 33645963780     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/11555261_13     Document Type: Conference Paper

References (20)

1. Borlund, P. (2003) The Concept of Relevance in IR, Journal of The American Society for Information Science and Technology, 54 (10), pp. 913-925.
2. Bray, T. (2003) On Search: Precision and Recall, http://www.tbray.org/ ongoing/When/200x/2003/06/22/PandR
3. Della Mea, V. and Mizzaro, S. (2004) Measuring Retrieval Effectiveness: A New Proposal and a First Experiemental Validation, Journal of the American Society for Information Science and Technology, 55(6), pp. 530-543.
4. Dziadosz, S. and Chandrasekar, R. (2002) Do Thumbnails Previews Help Users Make Better Relevance Decisions about Web Search Results?, In Proc. SIGIR 2002.
5. Eisenberg, E. (1988) Measuring Relevance Judgments, Information Processing and Management, 24(4), pp. 373-389
6. Janes, J. W. (1994) Other People's Judgments: A Comparison of Users' and Others' Judgments of Document Relevance, Topicality, and Utility, Journal of the American Society for Information Science, 45 (3), pp.160-171.
7. Hersh, W. Turpin, A., Price, S. , Chan, B., Kramer, D., Sacherek, L., and, Oison, D. (2000) Do Batch and User Evaluations Give the Same Results?, In Proc. SIGIR 2000.
8. Leroy, G., Lally, A., and Chen, H. (2003) The Use of Dynamic Contexts to Improve Casual Internet Searching, ACM Transactions on Information Systems, 21 (3), pp. 229-253.
9. Liu, F., Yu, C., and Meng, W. (2002) Personalized Web Search by Mapping User Queries to Categories, In Proc. IKM 2002, pp. 558-565.
10. Karat, J. Wolf, C., Alpert, S. R., Velderman, P., and Patil, S. (2003) "Improving Search on IBM.com", IBM Research Technical Report.
11. Kyung-Sun, K. and Allen, B. (2002) Cognitive and Task Influence on Web Searching Behavior, Journal of the American Society for Information Science and Technology, 53(2), pp. 109-119.
12. Muramatsu, J. and Pratt, W. (2001) Transparent Queries: Investigating Users' Mental Models of Search Engines, In Proc. SIGIR 2001.
13. Nielsen, J. (2004) When Search Engines Become Answer Engines, Alertbox, August, 24, 2004, http://www.useit.com/alertbox/20040816.html
14. Pack, T., Dumais, S., and Logan, R. (2004) Wavelens: A New View onto Internet Search Results, In Proc. CHI 2004.
15. Spink, A. (2002) A User-Centered Approach to Evaluating Human Interaction with Web Search Engines: An Exploratory Study, Information Processing and Management: An International Journal, 38(3), pp.401-426.
16. Spink, A. and Greisdorf, H. (2001) Regions and Levels: Measuring and Mapping Users' Relevance Judgments, Journal of the American Society for Information Science and Technology, 52(2), pp. 161-173.
17. Spink, A., and Saracevic, T. (1997) Interaction in Information Retrieval: Selection and Effectiveness of Search Terms, Journal of the American Society for Information Science, 48(8), pp.741-761.
18. Spink, A., Wolfram, D., Jansen, M. B. J., and Saracevic, T. (2001) Searching the Web: The Public and Their Queries, Journal of the American Society for Information Science and Technology, 52(3), pp.226-234.
19. Wolf, C. G., Alpert, S. R., Vergo, J. G., Kozakov, L., and Doganata, Y (2004) Summarizing Technical Support Documents for Search: Expert and User Studies, IBM Systems Journal, 43(3), http://www.research.ibm.com/journal/sj/433/ wolf.pdf.
20. Yaltaghian, B., and Chignell, M. (2002) Re-ranking Search using Network Analysis: A Case Study with Google, In Proc. IBM Centre for Advanced Studies Conference.