Evaluating non-deterministic retrieval systems

SIGIR 2014 - Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval

Volumn , Issue , 2014, Pages 911-914

  Jayasinghe, Gaya K ^a   Webber, William ^b   Sanderson, Mark ^a   Dharmasena, Lasitha S ^c   Culpepper, J Shane ^a  

^a RMIT UNIVERSITY   (Australia)

^b William Webber Consulting   (Australia)

^c DEAKIN UNIVERSITY   (Australia)

Author keywords

Effectiveness evaluation; Experimental design; Experimentation; Information retrieval; Measurement; Statistical analysis

Indexed keywords

ALGORITHMS; DESIGN OF EXPERIMENTS; MEASUREMENTS; RESEARCH; STATISTICAL METHODS;

DISTRIBUTED INFORMATION RETRIEVAL; EFFECTIVENESS EVALUATION; EXPERIMENTATION; MULTIVARIATE LINEAR MODEL; RANDOMIZED ALGORITHMS; RELIABLE METHODS; RETRIEVAL SYSTEMS;

INFORMATION RETRIEVAL;

EID: 84904575269     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2600428.2609472     Document Type: Conference Paper

References (6)

1. R. H. Baayen, D. J. Davidson, and D. M. Bates. Mixed-effects modeling with crossed random effects for subjects and items. Journal of memory and language, 59(4):390-412, 2008.
2. B. Carterette, E. Kanoulas, and E. Yilmaz. Simulating simple user behavior for system effectiveness evaluation. In CIKM, pages 611-620, 2011.
3. A. Kulkarni and J. Callan. Document allocation policies for selective searching of distributed indexes. In CIKM, pages 449-458, 2010.
4. D. Metzler and W. B. Croft. A markov random field model for term dependencies. In SIGIR, pages 472-479, 2005.
5. S. E. Robertson and E. Kanoulas. On per-topic variance in IR evaluation. In SIGIR, pages 891-900, 2012.
6. L. Si and J. Callan. Relevant document distribution estimation method for resource selection. In SIGIR, pages 298-305, 2003.