The overlap problem in content-oriented XML retrieval evaluation

Proceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

Volumn , Issue , 2004, Pages 72-79

  Kazai, Gabriella ^a   Lalmas, Mounia ^a   De Vries, Arien P ^b  

^a QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

^b CWI   (Netherlands)

Author keywords

Cumulated gain; Evaluation; INEX; Metrics; Overlap; Overpopulated recall base; XML retrieval

Indexed keywords

INFORMATION RETRIEVAL SYSTEMS; MATHEMATICAL MODELS; MATRIX ALGEBRA; PARAMETER ESTIMATION; PROBLEM SOLVING; VECTORS;

CUMULATED GAINS; INEX; OVERPOPULATED RECALL-BASE; XML RETRIEVAL;

XML;

EID: 8644287527     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1008992.1009008     Document Type: Conference Paper

References (11)

1. H. M. Blanken, T. Grabs, H.-J. Schek, R. Schenkel, and G. Weikum, editors. Intelligent Search on XML Data, Applications, Languages, Models, Implementations, and Benchmarks, volume 2818 of LNCS, Springer, 2003.
2. J. Clark and S. DeRose. XML Path Language (XPath) version 1.0. W3C Recommendation. http://www.w3.org/TR/xpath. Technical Report REC-xpath-19991116, WWW Consortium, Nov. 1999.
3. W. Cooper. Expected search length: A single measure of retrieval effectiveness based on the weak ordering action of retrieval systems. American Documentation, 19(1):30-41, 1968.
4. A. de Vries, G. Kazai, and M. Lalmas. Tolerance to Irrelevance: A user-effort oriented evaluation of retrieval systems without predefined retrieval unit. In Proceedings of the Recherche d'Informations Assistee par Ordinateur (RIAO 2004), Avignon, France, Apr. 2004.
5. N. Fuhr, M. Lalmas, and S. Malik, editors. Proceedings of the Second Workshop of the INitiative for the Evaluation of XML Retrieval (INEX). Dagstuhl, Germany, December 15-17, 2003, 2004. http://inex.is.informatik.uni-duisburg.de: 2003/proceedings.pdf.
6. N. Gövert, G. Kazai, N. Fuhr, and M. Lalmas. Evaluating the effectiveness of content-oriented XML retrieval. Technischer bericht, University of Dortmund, Computer Science 6, 2003.
7. K. Järvelin and J. Kekäläinen. Cumulated Gain-based evaluation of IR techniques. ACM Transactions on Information Systems (ACM TOIS), 20(4):422-446, 2002.
8. G. Kazai, M. Lalmas, N. Fuhr, and N. Gövert. A report on the first year of the INitiative for the Evaluation of XML retrieval (INEX'02), European research letter. Journal of the American Society for Information Science and Technology (JASIST), 55(6):551-556, Apr. 2004.
9. J. Kekäläinen and K. Järvelin. Using graded relevance assessments in IR evaluation. Journal of the American Society for Information Science and Technology, 53(13):1120-1129, 2002.
10. V. Raghavan, P. Bollmann, and G. Jung. A critical investigation of recall and precision. ACM Transactions on Information Systems, 7(3):205-229, 1989.
11. S. Wong and Y. Yao. On modeling information retrieval with probabilistic inference. ACM Transactions on Information Systems, 13(1):38-68, January 1995.