IR evaluation without a common set of topics

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 5766 LNCS, Issue , 2009, Pages 342-345

  Cattelan, Matteo ^a   Mizzaro, Stefano ^a  

^a UNIVERSITY OF UDINE   (Italy)

Author keywords

IR effectiveness; Topics; TREC

Indexed keywords

IR EFFECTIVENESS; SYSTEM EFFECTIVENESS; TOPICS; TREC;

INFORMATION RETRIEVAL; INFORMATION SERVICES;

SET THEORY;

EID: 70350610816     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-04417-5_35     Document Type: Conference Paper

References (11)

1. Alonso, O., Mizzaro, S.: Relevance criteria for e-commerce: A crowdsourcing-based experimental analysis. In: 32nd SIGIR (2009) (in press)
2. Alonso, O., Rose, D., Stewart, B.: Crowdsourcing for relevance evaluation. SIGIR Forum 42(2), 9-15 (2008)
3. Buckley, C., Voorhees, E.: Evaluating evaluation measure stability. In: 23rd SIGIR, pp. 33-40 (2000)
4. Guiver, J., Mizzaro, S., Robertson, S.: A few good topics: Experiments in topic set reduction for retrieval evaluation. In: ACM TOIS (2009) (in press)
5. Mizzaro, S.: The Good, the Bad, the Difficult, and the Easy: Something Wrong with Information Retrieval Evaluation? In: Macdonald, C., Ounis, I., Plachouras, V., Ruthven, I., White, R.W. (eds.) ECIR 2008. LNCS, vol. 4956, pp. 642-646. Springer, Heidelberg (2008)
6. Mizzaro, S., Robertson, S.: HITS hits TREC: exploring IR evaluation results with network analysis. In: 30th SIGIR, pp. 479-486 (2007)
7. Sanderson, M., Zobel, J.: Information retrieval system evaluation: effort, sensitivity, and reliability. In: 28th SIGIR, pp. 162-169 (2005)
8. Sparck Jones, K., van Rijsbergen, C.J.: Information retrieval test collections. Journal of Documentation 32, 59-75 (1976)
9. Voorhees, E., Buckley, C.: The effect of topic set size on retrieval experiment error. In: 25th SIGIR, pp. 316-323 (2002)
10. Webber, W., Moffat, A., Zobel, J.: Score standardization for inter-collection comparison of retrieval systems. In: 31st SIGIR, pp. 51-58 (2008)
11. Zobel, J.: How reliable are the results of large-scale information retrieval experiments? In: 21st SIGIR, pp. 307-314 (1998)