Variations in Relevance Judgments and the Measurement of Retrieval Effectiveness

SIGIR 1998 - Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

Volumn , Issue , 1998, Pages 315-323

  Voorhees, Ellen M ^a  

^a NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DOCUMENT SETS; LABORATORY TOOLS; MEASUREMENTS OF; RELEVANCE ASSESSMENTS; RELEVANCE JUDGEMENT; RELEVANT DOCUMENTS; RESEARCH COMMUNITIES; RETRIEVAL EFFECTIVENESS; RETRIEVAL STRATEGIES; TEST COLLECTION;

INFORMATION RETRIEVAL;

EID: 85165029145     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/290941.291017     Document Type: Conference Paper

References (15)

1. Robert Burgin. Variations in relevance judgments and the evaluation of retrieval performance. Infor- mation Processing and Management, 28(5):619-627, 1992.
2. C. W. Cleverdon, J. Mills, and E. M. Keen. Factors determining the performance of indexing systems. Two volumes, Cranfield, England, 1968.
3. Cyril W. Cleverdon. The effect of variations in relevance assessments in comparative experimental tests of index languages. Technical Report Cranfield Library Report No. 3, Cranfield Institute of Technology, Cranfield, UK, October 1970.
4. Gordon V. Cormack, Charles L.A. Clarke, Christo-her R. Palmer, and Samuel S.L. To. Passage-based refinement (MultiText experiments for TREC-6). In Proceedings of the Sixth Text REtrieval Conference (TREC-6), pages 171-186, November 1997. Draft for conference notebook.
5. C. A. Cuadra and R. V. Katter. Opening the black box of relevance. Journal of Documentation, 23(4):291-303, 1967.
6. Donna K. Harman. The first Text REtrieval Conference (TREC-1), Rockville, MD, U.S.A, 4-6 November, 1992. Information Processing and Management, 29(4):411-414, 1993.
7. Stephen P. Harter. Variations in relevance assessments and the measurement of retrieval effectiveness. Journal of the American Society for Information Science, 47(l):37-49, 1996.
8. K.L. Kwok, L. Grunfeld, and J.H. Xu. TREC-6 en-glish and Chinese retrieval experiments using PIRCS. In Proceedings of the Sixth Text REtrieval Conference (TREC-6), pages 96-102, November 1997. Draft for conference notebook.
9. M.E. Lesk and G. Salton. Relevance assessments and retrieval system evaluation. Information Storage and Retrieval, 4:343-359, 1969.
10. G. Salton, editor. The SMART Retrieval System: Experiments in Automatic Document Processing. Prentice-Hall, Inc. Englewood Cliffs, New Jersey, 1971.
11. Linda Schamber. Relevance and information behavior. Annual Review of Information Science and Technology, 29:3-48, 1994.
12. Karen Spaxck Jones. Automatic indexing. Journal of Documentation, 30:393-432, 1974.
13. Karen Sparck Jones. Information Retrieval Experiment. Butterworths, London, 1981.
14. Alan Stuart. Kendall's tau. In Samuel Kotz and Norman L. Johnson, editors, Encyclopedia of Statistical Sciences, volume 4, pages 367-369. John Wiley & Sons, 1983.
15. M. Taube. A note on the pseudomathematics of relevance. American Documentation, 16(2):69-72, April 1965.