Evaluation by comparing result sets in context

International Conference on Information and Knowledge Management, Proceedings

Volumn , Issue , 2006, Pages 94-101

  Thomas, Paul ^a   Hawking, David ^b  

^a AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

^b CSIRO   (Australia)

Author keywords

Embedded comparisons; Evaluation

Indexed keywords

DATA ACQUISITION; INTERFACES (COMPUTER); SEARCH ENGINES; WORLD WIDE WEB;

EMBEDDED COMPARISONS; INTERACTIVE RETRIEVAL; REAL SETTINGS; SEARCHER JUDGMENTS;

INFORMATION RETRIEVAL;

EID: 34547637035     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1183614.1183632     Document Type: Conference Paper

References (29)

1. A. Balmin, V. Hristidis, and Y. Papakonstantinou. ObjectRank: Authority-based keyword search in databases. In Proc. VLDB, 2004.
2. P. Borlund. The IIR evaluation model: A framework for evaluation of interactive information retrieval systems. Information Research, 8(3), 2003.
3. P. Borlund and P. Ingwersen. Measures of relative relevance and ranked half-life. In Proc. ACM SIGIR, 1998.
4. C. Buckley and E. M. Voorhees. Retrieval evaluation with incomplete information. In Proc. ACM SIGIR, 2004.
5. M. Claypool, P. Le, M. Waseda, and D. Brown. Implicit interest indicators. In Proc. Intelligent User Interfaces, 2001.
6. C. Cleverdon. The Cranfield tests on index language devices. In K. S. Jones and P. Willett, editors, Readings in Information Retrieval. Morgan Kaufmann, 1997.
7. S. Dumais, E. Cutrell, J. J. Cadiz, G. Jancke, R. Sarin, and D. C. Robbins. Stuff I've Seen: A system for personal information retrieval and re-use. In Proc. ACM SIGIR, 2003.
8. S. Fox. Evaluating implicit measures to improve the search experience. Talk presented at SIGIR Workshop on Implicit Measures of User Interests and Preferences, 2003.
9. M. Hancock-Beaulieu. Evaluating the impact of an online library catalogue on subject searching behaviour at the catalogue and at the shelves. Journal of Documentation, 46:318-338, 1990.
10. P. Hansen and K. Jarvelin. The information seeking and retrieval process at the Swedish Patent and Registration Office. In Proc. ACM SIGIR Workshop on Patent Retrieval, 2000.
11. D. Hawking and N. Craswell. Very large scale retrieval and web search. In Voorhees and Harman [28].
12. D. Hawking, N. Craswell, P. Bailey, and K. Griffiths. Measuring search engine quality. Information Retrieval, 4(1), 2001.
13. D. Hawking, C. Paris, R. Wilkinson, and M. Wu. Context in enterprise search and delivery. In Proc. IRiX Workshop, ACM SIGIR, 2005.
14. W. Hersh and P. Over. TREC-9 interactive track report. In Proc. TREC, 2001.
15. INitiative for the Evaluation of XML Retrieval. http://inex.is. informatik.uni-duisburg.de/.
16. T. Joachims. Evaluating retrieval performance using clickthrough data. In Proc. SIGIR Workshop on Mathematical/Formal Methods in IR, 2002.
17. T. Joachims, L. Granka, B. Pan, H. Hembrooke, and G. Gay. Accurately interpreting clickthrough data as implicit feedback. In Proc. ACM SIGIR, 2005.
18. Kartoo, S. A. http://www.kartoo.com/.
19. D. Kelly and N. J. Belkin. Display time as implicit feedback: Understanding task effects. In Proc. ACM SIGIR, 2004.
20. R. Nordli. "User revealment" - a comparison of initial queries and ensuing question development in online searching and in human reference interactions. In Proc. ACM SIGIR, 1999.
21. NTCIR (NII-NACSIS Test Collection for IR Systems) Project, http://research.nii.ac.jp/ntcir/.
22. C. Peters, M. Braschler, J. Gonzalo, and M. Kluck, editors. Second Workshop of the Cross-Language Evaluation Forum, CLEF 2001, volume 2406 of Lecture Notes in Computer Science. Springer-Verlag, 2002.
23. Y. Rasolofo, F. Abbaci, and J. Savoy. Approaches to collection selection and results merging for distributed information retrieval. In Proc. CIKM, 2001.
24. Y. Rasolofo, D. Hawking, and J. Savoy. Result merging strategies for a current news metasearcher. Information Processing and Management, 39(4), 2003.
25. E. Selberg and O. Etzioni. Multi-service search and comparison using the MetaCrawler. In Proc. WWW4, 1995.
26. X. Shen, B. Tan, and C. Zhai. Implicit user modeling for personalized search. In Proc. CIKM, 2005.
27. J. Teevan, C. Alvarado, M. S. Ackerman, and D. R. Karger. The perfect search engine is not enough: A study of orienteering behaviour in directed search. In Proc. Conf. Human Factors in Computing Systems, 2004.
28. E. M. Voorhees and D. K. Harman, editors. TREC: Experiment and Evaluation in Information Retrieval. MIT Press, 2005.
29. R. W. White, I. Ruthven, and J. M. Jose. A study of factors affecting the utility of implicit relevance feedback. In Proc. ACM SIGIR, 2005.