Investigating the exhaustivity dimension in content-oriented XML element retrieval evaluation

International Conference on Information and Knowledge Management, Proceedings

Volumn , Issue , 2006, Pages 84-93

  Ogilvie, Paul ^a   Lalmas, Mounia ^b  

^a Carnegie Mellon University   (United States)

^b QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

Author keywords

INEX; Relevance; Statistical tests; XML evaluation

Indexed keywords

STATISTICAL TESTS; TWO DIMENSIONAL; XML;

HUMAN ASSESSORS; INEX; XML EVALUATION;

INFORMATION RETRIEVAL;

EID: 34547626787     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1183614.1183631     Document Type: Conference Paper

References (16)

1. Y. Benjamini and D. Yekutieli. The control of the false discovery rate in multiple testing under dependency. The Annals of Statistics, 29(4):1165-1188, 2001.
2. D. Hull. Using statistical testing in the evaluation of retrieval experiments. SIGIR, 1993.
3. K. Järvelin and J. Kekäläinen. Cumulated Gain-based evaluation of IR techniques. TOIS, 20(4):422-446, 2002.
4. G. Kazai, M. Lalmas and A. P. de Vries. The Overlap Problem in Content-Oriented XML Retrieval Evaluation. SIGIR, 2004.
5. G. Kazai and M. Lalmas. INEX 2005 Evaluation Metrics. INEX 2005 Proceedings, 2006, In press.
6. S. Malik, M. Lalmas and N. Fuhr. Overview of INEX 2004. INEX 2004 Proceedings, 2005.
7. S. Malik, G. Kazai, M. Lalmas and N. Fuhr. Overview of INEX 2005. INEX 2005 Proceedings, 2006.
8. J. Pehcevski and J. A. Thorn. Hixeval: Highlighting XML retrieval evaluation. INEX 2005 Proceedings, 2006.
9. B. Piwowarski and M. Lalmas. Providing Consistent and Exhaustive Relevance Assessments for XML Retrieval Evaluation. CIKM, 2004.
10. B. Piwowarski, A. Trotman and M. Lalmas. Sound and Complete Relevance Assessments for XML Retrieval. Submitted, 2006.
11. V.V. Raghavan, P. Bollmann and O.S. Jung. A critical investigation of recall and precision as measures of retrieval system performance. TOIS 7(3):205-229, 1989.
12. M. Sanderson and J. Zobel. Information retrieval system evaluation: effort, sensitivity, and reliability. SIGIR, 2005.
13. J. Savoy. Statistical inference in retrieval effectiveness evaluation. IP&M 33(4):495-512, 1997.
14. A. Trotman. Wanted: Element retrieval users. INEX 2005 Workshop on Element Retrieval Methodology, 2005.
15. E. M. Voorhees and C. Buckley. The effect of topic set size on retrieval experiment error. SIGIR, 2002.
16. L. Wasserman. All of Statistics. Springer, 2004.