Principles for robust evaluation infrastructure

International Conference on Information and Knowledge Management, Proceedings

Volumn , Issue , 2011, Pages 3-6

  Zobel, Justin ^a   Webber, William ^a   Sanderson, Mark ^b   Moffat, Alistair ^a  

^a UNIVERSITY OF MELBOURNE   (Australia)

^b RMIT UNIVERSITY   (Australia)

Author keywords

evaluation; retrieval experiment; system measurement

Indexed keywords

CRANFIELD; EVALUATION; EVALUATION APPROACH; EVALUATION INFRASTRUCTURE; KEY ELEMENTS; RETRIEVAL SYSTEMS; SYSTEM MEASUREMENT; SYSTEMATIC EVALUATION;

EXPERIMENTS; INFORMATION RETRIEVAL; INFORMATION RETRIEVAL SYSTEMS; KNOWLEDGE MANAGEMENT;

SEARCH ENGINES;

EID: 83255176093     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2064227.2064247     Document Type: Conference Paper

References (6)

1. T. G. Armstrong, A. Moffat, W. Webber, and J. Zobel, "Improvements that don't add up: Ad-hoc retrieval results since 1998", Proc. CIKM, pages 601-610, Hong Kong, China, 2009, dx.doi.org/10.1145/1645953.1646031.
2. K.A. Baggerly and K.R. Coombes, "Deriving chemosensitivity from cell lines: Forensic bioinformatics and reproducible research in high-throughput biology", Annals of Applied Statistics, 3(4), 2009, arxiv.org/pdf/1010. 1092.
3. C. Buckley and E. M. Voorhees, "Evaluating evaluation measure stability", Proc. SIGIR, pages 33-40, Athens, Greece, 2000, dx.doi.org/10.1145/345508.345543.
4. B. Carterette, V. Pavlu, E. Kanoulas, J. Aslam, and J. Allan, "Evaluation over thousands of queries", Proc. SIGIR, pages 651-658, Singapore, Singapore, 2008, dx.doi.org/10.1145/1390334.1390445.
5. L. Deng and X. Huang, "Challenges in adopting speech recognition", Comm. ACM, 47(1), 2004, pp. 69-75, dx.doi.org/10.1145/962081. 962108.
6. W. Webber, A. Moffat, and J. Zobel, "Statistical power in retrieval experimentation", Proc. CIKM, pages 571-580, Napa, USA, 2008, dx.doi.org/10.1145/1458082.1458158.