Relevance thresholds in system evaluations

ACM SIGIR 2008 - 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Proceedings

Volumn , Issue , 2008, Pages 693-694

  Scholer, Falk ^a   Turpin, Andrew ^a  

^a RMIT UNIVERSITY   (Australia)

Author keywords

Information retrieval evaluation; Search engines; User study

Indexed keywords

INFORMATION SERVICES; RESEARCH AND DEVELOPMENT MANAGEMENT; SEARCH ENGINES;

SYSTEM EVALUATIONS; USER STUDY;

INFORMATION RETRIEVAL;

EID: 57349138853     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1390334.1390455     Document Type: Conference Paper

References (9)

1. A. Al-Maskari, M. Sanderson, and P. Clough. The relationship between IR effectiveness measures and user satisfaction. SIGIR'07, p773-774.
2. J. Allan, B. Carterette, and J. Lewis. When will information retrieval be "good enough"? SIGIR '05, p433-440.
3. C. Clarke, N. Craswell, and I. Soboroff. Overview of the TREC 2004 terabyte track. Gaithersburg, MD, 2005.
4. G.A Gescheider. Psychophysics: method, theory and application. Lawrence Erlbaum Ass., New Jersey, 1985.
5. W. Hersh, A. Turpin, S. Price, B. Chan, D. Kraemer, L. Sacherek, and D. Olson. Do batch and user evaluations give the same results? SIGIR'00, p17-24.
6. D. Kelly, X. Fu, and C. Shah. Effects of rank and precision of search results on users' evaluations of system performance. TR-2007-02, U. of North Carolina, 2007.
7. A. Turpin and W. Hersh. Why batch and user evaluations do not give the same results. SIGIR'01, p225-231.
8. A. Turpin and F. Scholer. User performance versus precision measures... SIGIR'06, p11-18.
9. E. M. Voorhees and D. K. Harman. TREC : experiment and evaluation in information retrieval. MIT Press, 2005.