Bootstrap-based comparisons of IR metrics for finding one relevant document

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 4182 LNCS, Issue , 2006, Pages 374-389

  Sakai, Tetsuya ^a  

^a TOSHIBA CORPORATION   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL METHODS; DATA ACQUISITION; DATA STRUCTURES; ONLINE SEARCHING; STATISTICAL TESTS;

BOOTSTRAP HYPOTHESIS TESTS; DATA SETS; DOCUMENTS; NORMALISED WEIGHTED RECIPROCAL RANK (NWRR); RECIPROCAL RANK (RR);

DATA MINING;

EID: 33751354079     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/11880592_29     Document Type: Conference Paper

References (18)

1. Buckley, C. and Voorhees, E. M.: Evaluating Evaluation Measure Stability. ACM SIGIR 2000 Proceedings (2000) 33-40
2. Buckley, C. and Voorhees, E. M.: Retrieval Evaluation with Incomplete Information. ACM SIGIR 2004 Proceedings (2004) 25-32
3. Efron, B. and Tibshirani, R. J.: An Introduction to the Bootstrap. Chapman & Hall/CRC (1993)
4. Eguchi, K. et al.: Overview of the Web Retrieval Task at the Third NTCIR Workshop. National Institute of Informatics Technical Report NII-2003-002E (2003)
5. Hawking, D. and Craswell, N.: The Very Large Collection and Web Tracks. In TREC: Experiment and Evaluation in Information Retrieval, MIT Press (2005) 199-231
6. Kando, N: Overview of the Fifth NTCIR Workshop. NTCIR-5 Proceedings (2005)
7. Kekäläinen, J.: Binary and Graded Relevance in IR Evaluations - Comparison of the Effects on Ranking of IR Systems. Information Processing and Management 41 (2005) 1019-1033.
8. Sakai, T.: The Reliability of Metrics based on Graded Relevance. AIRS 2005 Proceedings, LNCS 3689, Springer-Verlag (2005) 1-16
9. Sakai, T.: The Effect of Topic Sampling on Sensitivity Comparisons of Information Retrieval Metrics. NTCIR-5 Proceedings (2005) 505-512
10. Sakai, T.: On the Task of Finding One Highly Relevant Document with High Precision. Information Processing of Japan Transactions on Databases TOD-29 (2006)
11. Sakai, T.: Evaluating Evaluation Metrics based on the Bootstrap. ACM SIGIR 2006 Proceedings, to appear (2006)
12. Sakai, T.: Give Me Just One Highly Relevant Document: P-measure. ACM SIGIR 2006 Proceedings, to appear (2006)
13. Sakai, T.: A Further Note on Evaluation Metrics for the Task of Finding One Highly Relevant Document. Information Processing of Japan SIG Technical Reports FI-82 (2006) 69-76
14. Sakai, T.: On the Reliability of Information Retrieval Metrics based on Graded Relevance, Information Processing and Management, to appear (2006)
15. Sanderson, M. and Zobel, J.: Information Retrieval System Evaluation: Effort, Sensitivity, and Reliability. ACM SIGIR 2005 Proceedings (2005) 162-169
16. Soboroff, I.: On Evaluating Web Search with Very Few Relevant Documents. ACM SIGIR 2004 Proceedings (2004) 530-531
17. Voorhees, E. M. and Buckley, C.: The Effect of Topic Set Size on Retrieval Experiment Error. ACM SIGIR 2002 Proceedings (2002) 316-323
18. Voorhees, E. M.: Overview of the TREC 2004 Robust Retrieval Track. TREC 2004 Proceedings (2005)