Incremental test collections

International Conference on Information and Knowledge Management, Proceedings

Volumn , Issue , 2005, Pages 680-687

  Carterette, Ben ^a   Allan, James ^a  

^a Biologically Inspired Neural and Dynamical Systems Laboratory   (United States)

Author keywords

Algorithms; Evaluation; Information retrieval; Test collections

Indexed keywords

KENDALL'S TAU RANK CORRELATION; RELEVANCE JUDGMENTS; SYSTEM EVALUATION; TEST COLLECTIONS;

ALGORITHMS; COSTS; INFORMATION ANALYSIS; KNOWLEDGE ACQUISITION;

INFORMATION RETRIEVAL;

EID: 33745768455     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1099554.1099723     Document Type: Conference Paper

References (14)

1. S. M. Beitzel, E. C. Jensen, A. Chowdhury, D. Grossman, and O. Frieder. Using manually-build web directories for automatic evaluation of known-item retrieval. In Proceedings of SIGIR '03, pages 373-374, 2003.
2. C. Buckley and E. M. Voorhees. Evaluating Evaluation Measure Stability. In Proceedings of SIGIR '00, pages 33-40, 2000.
3. C. Buckley and E. M. Voorhees. Retrieval evaluation with incomplete information. In Proceedings of SIGIR '04, pages 25-32, 2004.
4. G. V. Cormack, C. R. Palmer, and C. L. Clarke. Efficient Construction of Large Test Collections. In Proceedings of SIGIR '98, pages 282-289, 1998.
5. S. P. Harter. Variations in relevance assessments and the measurement of retrieval effectiveness. JASIS, 47(1):37-49, 1996.
6. M. Kendall. Rank Correlation Methods. Griffin, London, UK, fourth edition, 1970.
7. M. Sanderson and H. Joho. Forming test collections with no system pooling. In Proceedings of SIGIR '04, pages 33-40, 2004.
8. I. Soboroff, C. Nicholas, and P. Cahan. Ranking Retrieval Systems without Relevance Judgments. In Proceedings of SIGIR '01, pages 66-73, 2001.
9. K. Sparck Jones and C. J. van Rijsbergen. Information Retrieval Test Collections. Journal of Documentation, 32(1):59-75, 1976.
10. E. Voorhees. Variations in Relevance Judgments and the Measurement of Retrieval Effectiveness. In Proceedings of SIGIR '98, pages 315-323, 1998.
11. E. M. Voorhees. Evaluation by highly relevant documents. In Proceedings of SIGIR '01, pages 74-82, 2001.
12. E. M. Voorhees. The philosophy of information retrieval evaluation. In CLEF '01: Revised Papers from the Second Workshop of CLEF, pages 355-370, London, UK, 2002. Springer-Verlag.
13. E. M. Voorhees and D. Harman. Overview of the Eighth Text REtrieval Conference (TREC-8). In Proceedings of TREC-8, pages 1-24, 1999. NIST Special Publication 500-246.
14. J. Zobel. How Reliable are the Results of Large-Scale Information Retrieval Experiments? In Proceedings of SIGIR '98, pages 307-314, 1998.