Information retrieval evaluation with partial relevance judgment

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 4042 LNCS, Issue , 2006, Pages 86-93

  Wu, Shengli ^a   McClean, Sally ^a  

^a UNIVERSITY OF ULSTER   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

DATABASE SYSTEMS; QUERY LANGUAGES; RELIABILITY; RESEARCH AND DEVELOPMENT MANAGEMENT; SENSITIVITY ANALYSIS;

INFORMATION RETRIEVAL EVALUATION; MEAN AVERAGE PRECISION; PARTIAL RELEVANCE JUDGMENT;

INFORMATION RETRIEVAL;

EID: 33746722678     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/11788911_7     Document Type: Conference Paper

References (18)

1. J. A. Aslam, E. Yilmaz, and V. Pavlu. The maximum entropy method for analysing retrieval measures. In Proceedings of ACM SIGIR'2005, pages 27-34, Salvador, Brazil, August 2005.
2. C. Buckley and E. M. Voorhees. Evaluating evaluation measure stability. In Proceedings of ACM SIGIR'2000, pages 33-40, Athens, Greece, July 2000.
3. C. Buckley and E. M. Voorhees. Retrieval evaluation with incomplete information. In Proceedings of ACM SIGIR'2004, pages 25-32, Sheffield, United Kingdom, July 2004.
4. R. L. Graham, D. E. Knuth, and O. Patashnik. Concrete, mathematics. Adclison-wesley publishing company, 1989.
5. K. Jävelin and J. Kekäläinen. Cumulated gain-baaed evaluation of IR techniques. ACM Transactions on Information Systems, 20(4):442-446, October 2002.
6. Y. Kagolovsky and J. R. Moehr. Current status of the evaluation of information retrieval. Journal of Medical Systems, 27(5):409-424, October 2003.
7. J. Kekäläinen, Binary and graded relevance in IR evaluations - comparison of the effects on ranking of IR systems. Information Processing & Management, 41(5):1019-1033, September 2005.
8. S. E. Robertson and M. M. Hancock-Beaulieu, On the evaluation of IR systems. Information Processing & Management, 28(4):457-466, July-August 1992.
9. M. Sanderson and J. Zobel. Information retrieval system evaluation: Effort, sensitivity, and reliability. In Proceedings of ACM SIGIR'8005, pages 162-169, Salvador, Brazil, August 2005.
10. W. M. Shaw, R. Burgin, and P. Howell. Performance standards and evaluations in IR test collections: Cluster-based retrieval models. Information Processing & Management, 33(1): 1-14, January 1997.
11. TREC. http://trec.nist.gov/.
12. C. J. van Rijsbergen. Information Retrieval Butterworths, 1979.
13. V. G. Voiskunskii. Evaluation of search results: A new approach. Journal of the American Society for Information Science, 48(2): 133-142, February 1997.
14. E. M. Voorhees. Variations in relevance judgments and the measurement of retrieval effectiveness. In Proceedings of ACM SIGIR'1998, pages 315-323, Melbourne, Australia, August 1998.
15. E. M. Voorhees. Variations in relevance judgments and the measurement of retrieval .effectiveness. Information Processing & Management, 36(5):697-716, September 2000.
16. E. M. Voorhees and C. Buckley. The effect of topic set size on retrieval experiment error. In Proceedings of ACM SIGIR'2002, pages 316-323, Tampere, Finland, August 2002.
17. S. Wu and S. McClean. Modelling rank-probability of relevance relationship in resultant document list for data fusion, submitted for publication.
18. J. Zobel. How reliable are the results of large-scale information retrieval experiments. In Proceedings of ACM SIGIR'1998, pages 307-314, Melbourne, Australia, August 1998.