Evaluating the effectiveness of content-oriented XML retrieval methods

Information Retrieval

Volumn 9, Issue 6, 2006, Pages 699-722

  Gövert, Norbert ^a   Fuhr, Norbert ^b   Lalmas, Mounia ^c   Kazai, Gabriella ^c  

^a TU DORTMUND UNIVERSITY   (Germany)

^b UNIVERSITY OF DUISBURG ESSEN   (Germany)

^c QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

Author keywords

Effectiveness; Evaluation; Exhaustiveness and specificity; Metrics; XML retrieval

Indexed keywords

EID: 33749256966     PISSN: 13864564     EISSN: 15737659     Source Type: Journal    
DOI: 10.1007/s10791-006-9008-2     Document Type: Article

References (35)

1. Baeza-Yates, R., Fuhr, N., & Maarek, Y. S. (Eds.) (2002). Proceedings of the SIGIR 2002 Workshop on XML and Information Retrieval.
2. Baeza-Yates, R., Fuhr, N., Sacks-Davis, R., & Wilkinson, R. (Eds.) (2000). In Proceedings of the SIGIR 2000 Workshop on XML and Information Retrieval. http://www.halfa.il.ibm.com/sigir00-xml/index.html
3. Beaulieu, M., & Robertson, S. (1996). Evaluating interactive systems in TREC. Journal of the American Society for Information Science, 47(1), 85-94.
4. Chiaramella, Y., Mulhem, P., & Fourel, F. (1996). A model for multimedia information retrieval. Technical report, FERMI ESPRIT BRA 8134, University of Glasgow.
5. Clark, J., & DeRose, S. (1999). XML path language (XPath) version 1.0. Technical report, World Wide Web Consortium. http://www.w3.org/TR/xpath
6. Cleverdon, C. W., Mills, J., & Keen, E. M. (1966). Factors determining the performance of indexing systems, vol. 2: Test results. Technical report, Aslib Cranfield Research Project, Cranfield, England.
7. Cooper, W. S. (1968). Expected search length: A single measure of retrieval effectiveness based on weak ordering action of retrieval systems. Journal of the American Society for Information Science, 19, 30-4-1.
8. Cosijn, E., & Ingwersen, P. (2000). Dimensions of relevance. Information Processing and Management, 36(4), 533-550.
9. Craswell, N., & Hawking, D. (2004). Overview of the TREC 2004 Web Track. In Proceedings of the 13th Text Retrieval Conference (TREC-2004). Gaithersburg, Maryland, USA, N1ST. http://trec.nist.gov/pubs/trec13/papers/WEB. OVERVIEW.pdf.
10. Croft, W. B., Moffat, A., van Rijsbergen, C. J., Wilkinson, R., & Zobel, J. (Eds.) (1998). In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM.
11. Fuhr, N., Gövert, N., Kazai, G., & Lalmas, M. (Eds.) (2003). INitiative for the Evaluation of XML Retrieval (INEX). In Proceedings of the First INEX Workshop. Dagstuhl, Germany, Dec. 8-11, 2002. ERCIM Workshop Proceedings. Sophia Antipolis, France: ERCIM. http://www.ercim.org/publication/ ws-proceedings/INEX2002.pdf
12. Fuhr, N., & Lalmas, M. (2004). Report on the INEX 2003 Workshop. SIGIR Forum, 38(1).
13. Fuhr, N., Lalmas, M., & Malik, S. (Eds.) (2004a). INitiative for the Evaluation of XML Retrieval (INEX). In Proceedings of the Second INEX Workshop. Dagstuhl, Germany, Dec. 15-17, 2003. http://inex.is.informatik.uni-duisburg.de: 2003/proceedings.pdf.
14. Fuhr, N., Malik, S., & Lalmas, M. (2004b). Overview of the INitiative for the Evaluation of XML Retrieval (INEX) 2003. In Fuhr et al. (2004a), (pp. 1-11).
15. Harman, D. (1993). Overview of the First Text REtrieval Conference. In D. Harman (Ed.) The First Text REtrieval Conference (TREC-1), Gaithersburg, Md. 20899, National Institute of Standards and Technology Special Publication 500-207.
16. Jones, K. S., & van Rijsbergen, C. J. (1976). Information retrieval test collections. Journal of Documentation, 32(1), 59-75.
17. Kando, N., & Adachi, J. (2004). Report from the NTCIR workshop 3. SIGIR Forum, 38(1), 10-16.
18. Kazai, G. (2004). Report of the INEX 2003 Metrics working group. In Fuhr et al. (2004a) pp. 184-190.
19. Kazai, G., Lalmas, M., & de Vries, A. P. (2004). The overlap problem in content-oriented XML retrieval evaluation. In K. Järvelin, J. Allen, P. Bruza, & M. Sanderson (Eds.), Proceedings of the 27st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 72-79) New York, ACM.
20. Kekäläinen, J., & Järvelin, K. (2002). Using graded relevance sssessments in IR evaluation. Journal of the American Society for Information Science and Technology, 53(13).
21. Lancaster, F. W. ( 1968). Evaluation of the MEDLARS demand service. Report, National Library of Medicine, Bethesda, Maryland.
22. Peters, C., Braschler, M., Gonzalo, J., & Kluck, M. (Eds.) (2002). Evaluation of cross-language information retrieval systems (CLEF 2001). Vol. 2406 of Lecture Notes in Computer Science. Heidelberg et al., Springer.
23. Piwowarski, B., & Gallinari, P. (2004). Expected ratio of relevant units: A measure of structured information retrieval. In Fuhr et al. (2004a), pp. 158-166. http://inex.is.informatik.uni-duisburg.de:2003/proceedings.pdf
24. Piwowarski, B., & Lalmas, M. (2004). Ensuring consistent and exhaustive relevance assessments for XML retrieval evaluation. In L. Gravano (Ed.), Proceedings of the 13th International Conference on Information and Knowledge Management. New York, ACM.
25. Raghavan, V. V., Bollmann, P., & Jung, G. S. (1989). A critical investigation of recall and precision as measures of retrieval system performance. ACM Transactions on Information Systems, 7(3), 205-229.
26. Salton, G. (Ed.) (1971). The SMART retrieval system - Experiments in automatic document processing. Englewood, Cliffs, New Jersey: Prentice Hall.
27. Salton, G., & McGill, M. J. (1983). Introduction to modern information retrieval. New York: McGraw-Hill.
28. Saracevic, T. (1995). Evaluation of evaluation in information retrieval. In: E. A. Fox, P. Ingwersen, and R. Fidel (Eds.), Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. (pp. 138-146), New York, ACM. ISBN 0-89791-714-6.
29. Saracevic, T. (1996). Relevance reconsidered. In P. Ingwersen and N. O. Pors (Eds.), In Proceedings of the 2nd International Conference on Conceptions of Library and Information Science (CoLIS 2), Oct. 13-16, 1996, (pp. 201-218).
30. Tombros, A., Larsen, B., & Malik, S. (2005). The Interactive Track at INEX 2004. In N. Fuhr, M. Lalmas, S. Malik, & Z. Szlavik (Eds.), Advances in XML Information Retrieval: Third International Workshop of the Initiative, for the Evaluation of XML Retrieval, INEX 2004, Dagstuhl Castle, Germany, Dec. 6-8, 2004, Revised Selected Papers, Vol. 3493. Springer-Verlag GmbH, http://www.springer-online.com/3-540-26166-4.
31. trec_eval (2002). Evaluation techniques and measures. In Voorhees and Harman (2002), NIST.
32. Voorhees, E. M. (1998). Variations in relevance judgements and the measurement of retrieval effectiveness. In Croft et al. (1998), (pp. 315-323), ACM.
33. Voorhees, E. M., & Harman, D. K. (Eds.) (2002). The Tenth Text REtrieval Conference (TREC 2001). Gaithersburg, MD, USA: NIST.
34. Wong, S. K. M., & Yao, Y. Y. (1995). On modeling information retrieval with probabilistic inference. ACM Transactions on Information Systems, 13(1), 38-68.
35. Zobel, J. (1998). How reliable are the results of large-scale information retrieval experiments?. In Croft et al. (1998) (pp. 307-314), ACM.