IR evaluation using multiple assessors per topic

ADCS 2007 - Proceedings of the Twelfth Australasian Document Computing Symposium

Volumn , Issue , 2007, Pages 9-16

  Trotman, Andrew ^a   Jenkinson, Dylan ^a  

^a UNIVERSITY OF OTAGO   (New Zealand)

Author keywords

Information Retrieval

Indexed keywords

DIFFERENT ORDER; RANK ORDER; SPEARMAN'S RANK CORRELATION COEFFICIENTS; TEST SETS;

INFORMATION RETRIEVAL;

LAKES;

EID: 84876705138     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Article

References (14)

1. Denoyer, L., & Gallinari, P. (2006). The Wikipedia XML corpus. In Proceedings of the INEX 2006 Workshop.
2. Malik, S., Trotman, A., Lalmas, M., & Fuhr, N. (2006). Overview of INEX 2006. In Proceedings of the INEX 2006 Workshop.
3. Ogilvie, P., & Lalmas, M. (2006). Investigating he exhaustivity dimension in content oriented XML element retrieval evaluation. In Proceedings of the 15th ACM Conference on Information and Knowledge Management (CIKM 2006).
4. Pehcevski, J., & Thom, J. A. (2005). HiXEval: Highlighting XML retrieval evaluation. In Proceedings of the INEX 2005 Workshop.
5. Pehcevski, J., & Thom, J. A. (2007). Evaluating focused retrieval tasks. In Proceedings of the SIGIR 2007 Workshop on Focused Retrieval, 33-40.
6. Piwowarski, B., Trotman, A., & Lalmas, M. (2007 (submitted)). Sound and complete relevance assessments for XML retrieval.
7. Sanderson, M., & Zobel, J. (2005). Information retrieval system evaluation: Effort, sensitivity, and reliability. In Proceedings of the 28th ACM SIGIR Conference on Information Retrieval, 162-169.
8. Spink, A., Wolfram, D., Jansen, B. J., & Saracevic, T. (2001). Searching the web: The public and their queries. Journal of the American Society for Information Science and Technology, 53(2):226-234.
9. Trotman, A. (2005). Wanted: Element retrieval users. In Proceedings of the INEX 2005 Workshop on Element Retrieval Methodology, Second Edition, 63-69.
10. Trotman, A., N.Pharo, & Lehtonen, M. (2006). XML-IR users and use cases. In Proceedings of the INEX 2006 Workshop.
11. Trotman, A., Pharo, N., & Jenkinson, D. (2007). Can we at least agree on something? In Proceedings of the SIGIR 2007 Workshop on Focused Retrieval, 49-56.
12. Voorhees, E. M. (2005). Overview of TREC 2005. In Proceedings of the 14th Text REtrieval Conference (TREC-14).
13. Voorhees, E. M., & Buckley, C. (2002). The effect of topic set size on retrieval experiment error. In Proceedings of the 25th ACM SIGIR Conference on Information Retrieval, 316-323.
14. Zobel, J. (1998). How reliable are the results of large-scale information retrieval experiments? In Proceedings of the 21st ACM SIGIR Conference on Information Retrieval, 307-314.