Easy tasks dominate information retrieval evaluation results

Datenbanksysteme in Business, Technologie und Web, BTW 2009 - 13th Fachtagung des GI-Fachbereichs "Datenbanken und Informationssysteme" (DBIS), Proceedings

Volumn , Issue , 2009, Pages 107-116

  Mandl, Thomas ^a  

^a UNIVERSITY OF HILDESHEIM   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

CRONBACH'S ALPHAS; CROSS-LANGUAGE EVALUATION FORUMS; EVALUATION EXPERIMENTS; FOUR-GROUP; MULTILINGUAL RETRIEVAL; NOVEL METHODOLOGY; POTENTIAL USERS; RELEVANT DOCUMENTS; RETRIEVAL EVALUATION; ROBUST MEASURES; SYSTEM RANKINGS; TEST RELIABILITY;

DATABASE SYSTEMS;

INFORMATION RETRIEVAL SYSTEMS;

EID: 84873902979     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (18)

1. Agirre, E.; Di Nunzio, G.; Ferro, N.; Mandl, T.; Peters, C.: CLEF 2008: Ad Hoc Track Overview. In: Evaluating Systems for Multilingual and Multimodal Access: 9th Workshop of the Cross-Language Evaluation Forum, CLEF 2008, Aarhus, Denmark, Revised Selected Papers. Berlin et al.: Springer [LNCS]. To appear.
2. Bodoff, D.; Li, P.: Test Theory for Assessing IR Test Collections. In: Proc Annual Intl. ACM Conf. on Research and Development in Information Retrieval (SIGIR) Amsterdam. 2007. 367-374.
3. Buckley, C.; Voorhees, E.: The Effect of Topic Set Size on Retrieval Experiment Error. In: Proc Annual Intl. ACM Conf. on Research and Development in Information Retrieval (SIGIR) Tampere, 2002. 316-323.
4. Buckley, C.; Voorhees, E. M.: Retrieval Evaluation with Incomplete Information. In: Proc Annual Intl ACM Conf. on Research and Development in Information Retrieval (SIGIR) 2004. 25-32
5. Buckley, C.; Voorhees, E.: Retrieval System Evaluation. In: TREC: Experiment and Evaluation in Information Retrieval. Cambridge & London: MIT Press. 2005. 53-75.
6. th Annual Intl. ACM Conf. on Research and Development in Information Retrieval (SIGIR) Seattle. 2006. 33-40.
7. Eguchi, K., Kando, N. and Kuriyama, K.: Sensitivity of IR Systems Evaluation to Topic Difficulty. In: Proc. of the Third International Conference on Language Resources and Evaluation (LREC 2002) 585-589.
8. th Annual Intl. Conf. on Research and development in information retrieval (SIGIR) 2004. 528-529.
9. Kwok, K.: An Attempt to Identify Weakest and Strongest Queries. In: SIGIR Workshop Predicting Query Difficulty. 2005. http://www.haifa.il.ibm.com/ sigir05-qp
10. Mandl, T.: Benutzerorientierte Bewertungsmaßstäbe für Information Retrieval Systeme: Der Robust Task bei CLEF 2006. In: Effektive Information Retrieval Verfahren in Theorie und Praxis: Proc d. Fünften Hildesheimer Evaluierungs- und Retrievalworkshop (HIER 2006) http://web1.bib.uni-hildesheim.de/edocs/2006/519937899/meta/
11. Mandl, T: Recent Developments in the Evaluation of Information Retrieval Systems: Moving Toward Diversity and Practical Applications. In: Informatica - An International Journal of Computing and Informatics vol. 32. 2008. pp. 27-38.
12. Mandl, T.; Womser-Hacker, C.; Ferro, N.; Di Nunzio, G.: How Robust are Multilingual Information Retrieval Systems? In: Proc. ACM SAC Symposium on Applied Computing. Fortaleza, Brazil. 2008. pp. 1132-1136.
13. Mizzaro, S.; Robertson, S.: HITS hits TREC - exploring IR evaluation results with network analysis. In: Proc. 30th Annual Intl. ACM Conf. on Research and Development in Information Retrieval (SIGIR) Amsterdam. 2007. 479-486
14. th Annual Intl. ACM Conf. on Research and Development in Information Retrieval (SIGIR) Amsterdam. 2007. 375-382.
15. th Workshop of the Cross-Language Evaluation Forum, CLEF 2006, Alicante, Spain, Revised Selected Papers. Berlin et al.: Springer [LNCS 4730] 2007. 21-34.
16. th ACM Intl. Conf. on Information and Knowledge Management (CIKM) Arlington, VA. 2006. pp. 872-877.
17. th Annual Intl. Conf. on Research and Development in Information Retrieval (SIGIR) Salvador, Brazil. 2005. 162-169.
18. Voorhees, E. The TREC robust retrieval track. ACM SIGIR Forum 39 (1) 2005. 11-20.