Sound and complete relevance assessment for XML retrieval

ACM Transactions on Information Systems

Volumn 27, Issue 1, 2008, Pages

  Piwowarski, Benjamin ^a,d   Trotman, Andrew ^b   Lalmas, Mounia ^c  

^a YAHOO RESEARCH   (United States)

^b UNIVERSITY OF OTAGO   (New Zealand)

^c QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

^d UNIVERSITY OF GLASGOW   (United Kingdom)

Author keywords

Evaluation; INEX; Passage retrieval; Relevance assessment; XML; XML retrieval

Indexed keywords

EVALUATION; INEX; PASSAGE RETRIEVAL; RELEVANCE ASSESSMENT; XML RETRIEVAL;

INFORMATION SERVICES; XML;

MARKUP LANGUAGES;

EID: 66949118546     PISSN: 10468188     EISSN: 15582868     Source Type: Journal    
DOI: 10.1145/1416950.1416951     Document Type: Article

References (29)

1. BAEZA-YATES, R., FUHR, N., AND MAAREK, Y. S. (EDS.). 2002. Proceedings of the ACM SIGIR Workshop on XML.
2. BLANKEN, H. M., GRABS, T., SCHEK, H.-J., SCHENKEL, R., AND WEIKUM, G. (EDS.). 2003. Intelligent Search on XML Data, Applications, Languages, Models, Implementations, and Benchmarks. Lecture Notes in Computer Science, vol.2818. Springer, Berlin, Germany.
3. CARMEL, D., MAAREK, Y., AND SOFFER, A. (EDS.). 2000. Proceedings of the ACM SIGIR Workshop on XML.
4. CLARKE, C. 2005. Range results in XML retrieval. In Proceedings of the INEX Workshop on Element Retrieval Methodology, 2nd ed. 4-5.
5. CLEVERDON, C. W., MILLS, J., AND KEEN, M. 1966. Factors determining the performance of indexing systems: Cranfield University, Cranfield, Bedforshine, U.K.
6. CORMACK, G. V., PALMER, C. R., TO, S. S. L., AND CLARICE, C. L. A. 1997. Passage-based refinement (multitext experiements for TREC-6). In Proceedings of the 6th Text REtrieval Conference (TREC-6), 171-186.
7. DENOYER, L. AND GALLINARI, P. 2006. The Wikipedia XML corpus. In Proceedings of the. INEX Workshop.
8. FUHR, N., KAMPS, J., LALMAS, M., MALIK, S., AND TROTMAN, A. 2007. Overview of the INEX 2007 ad hoc track. In Proceedings of INEX. 1-22.
9. HARMAN, D. 1992. Overview of the first text retrieval conference (TREC-I). In Proceedings of the 1st Text REtrieval Conference (TREC-1).
10. HARMAN, D. 1995. The TREC conferences. In Proceedings of the HIM International Conference.
11. KAMPS, J., KOOLEN, M.,LALMAS, M. 2007 Where to start reading a textual XML document? Proceedings of the 30th ACM SIGIR Conference on Information Retrieval. (Pubitemid 350165038)
12. KAZAI, G. AND LALMAS, M. 2005. INEX 2005 evaluation metrics. In Proceedings of INEX.
13. KAZAI, G., MASOOD, S., AND LALMAS, M. 2004. A study of the assessment of relevance for the INEX'02 test collection. In Proceedings of the 26th European Colloquium on Information Retrieval Research (ECIR 2004).
14. KEKÄLÄINEN, J., AND JÄRVELIN, K. 2002. Using graded relevance assessments in IR evaluation. J. Amer. Soc. Inform. Sei. Tech. 53, 13, 1120-1129. (Pubitemid 35359181)
15. LALMAS, M. AND TOMBROS, A. 2007. Evaluating XML retrieval effectiveness at INEX. SIGIR For. 41, 1, 40-57.
16. LUK, R. W. P., LEONG, H. V., DILLON, T. S., CHAN, A. T. S., CROFT, W. B., AND ALLAN, J. 2002. A survey in indexing and searching XML documents. J. Amer. Soc. Inform. Sci. Tech. 53, 6, 415-437. (Pubitemid 34300498)
17. MALIK, S., KAZAI, G., LALMAS, M., AND FUHR, N 2005. Overview of INEX 2005. In Proceedings of the INEX Workshop. 1-15.
18. OOILVIE, P. AND LALMAS, M. 2006. Investigating the exhaustivity dimension in content oriented XML element retrieval evaluation. In Proceedings of the 15th ACM Conference on Information and Knowledge Management (CIKM 2006). (Pubitemid 47203555)
19. PEHOEVSKI, J. AND THOM, J. A. 2005. HiXEval: Highlighting XML retrieval evaluation. In Proceedings of the INEX Workshop.
20. PEHCEVSKI, J., THOM, J. A. AND VERCOUSTRE, A.-M. 2005. Users and assessors in the context of INEX: Are relevance dimensions relevant? In Proceedings of the INEX Workshop on Element Retrieval Methodology, 2nd ed. 47-62.
21. PIWOWARSKI, B. AND LALMAS, M. 2004. Providing consistent and exhaustive relevance assessments for XML retrieval evaluation. In Proceedings of the 13th ACM Conference on Information and Knowledge Management. 361-370.
22. SARACEVIC, T. 1991. Individual differences in organizing, searching and retrieving information. Proc. Amer. Soc. Inform. Sci. 28, 82-86.
23. TOMBROS, A., LARSEN, B., AND MALIK, S. 2004. The interactive track at INEX 2004. In Proceedings of the INEX Workshop. 410-423. (Pubitemid 41314010)
24. TROTMAN, A. 2005. Wanted: Element retrieval users. In Proceedings of the INEX Workshop on Element Retrieval Methodology, 2nd ed. 63-69.
25. TROTMAN, A. AND GEVA, S. 2006. Passage retrieval and other XML-retrieval tasks. In Proceedings of the SIGIR Workshop on XML Element Retrieval Methodology. 43-50.
26. VOORHEES, E. M. 2000. Variations in relevance judgments and the measurement of retrieval effectiveness. Inform. Process. Manage. 36, 5, 697-716.
27. Vu, H.-T. AND GALLINARI, P. 2005. Using rankboost to compare retrieval systems. In Proceedings of the 14th Conference on Information and Knowledge Management (CIKM'05). (Pubitemid 44022178)
28. WOODLEY, A. AND GEVA, S. 2005. Fine tuning INEX. In Proceedings of the INEX Workshop on Element Retrieval Methodology, 2nd ed. 70-79.
29. ZOBEL, J. 1998. How reliable are the results of large-scale information retrieval experiments? In Proceedings of the 21st ACM SIGIR Conference on Information Retrieval. 307-314.