Precision recall with user modeling (PRUM): Application to structured information retrieval

ACM Transactions on Information Systems

Volumn 25, Issue 1, 2007, Pages

  Piwowarski, B ^a   Gallinari, P ^b   Dupret, G ^a  

^a YAHOO RESEARCH   (United States)

^b UNIVERSITÉ PIERRE ET MARIE CURIE   (France)

Author keywords

Evaluation; Information retrieval; Measure; Precision recall; Web; XML

Indexed keywords

PROBABILISTIC LOGICS; USER INTERFACES; WORLD WIDE WEB; XML;

IR PARADIGMS; PRECISION-RECALL METRIC; STANDARD INFORMATION RETRIEVAL (IR) METRICS; WEB IR;

INFORMATION RETRIEVAL;

EID: 33847638550     PISSN: 10468188     EISSN: 15582868     Source Type: Journal    
DOI: 10.1145/1198296.1198297     Document Type: Article

References (23)

1. BAEZA-YATES, R. AND RIBEIRO- NETO, B. 1999. Modern Information Retrieval. Addison Wesley, New York, NY.
2. CLEVERDON, C. 1967. The cranfield tests on index language devices. In Proceedings Aslib. vol. 19, 173-192.
3. COOPER, W. S. 1968. Expected search length: A single measure of retrieval effectiveness based on the weak ordering action of retrieval systems. Amer. Documentat. 19, 30-41.
4. COOPER, W. S. 1973. On selecting a measure of retrieval effectiveness, part 1. J. Amer. Soci. Inform. Sci. 24, 87-100.
5. DUNLOP, M. D. 1997. Time, relevance and interaction modeling for information retrieval. In Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM Press, 206-213.
6. FUHH, N., LALMAS, M., AND MALIK, S., EDS. 2003. INEX Proceedings.
7. GÖVERT, N., KAZAI, G., FUHR, N., AND LALMAS, M. 2003. Evaluating the effectiveness of contentoriented XML retrieval. Tech. rep., Computer Science 6, University of Dortmund.
8. HECKERMAN, D. AND BREESE, J. S. 1994. A new look at causal independence. In Proceedings of the 10th Annual Conference on Uncertainty in Artificial Intelligence INEX Proceedings. (UAI'94). Morgan Kaufmann Publishers, San Francisco, CA, 286-292.
9. KAZAI, G. 2003. Report on the INEX 2003 metrics group. INEX Proceedings. 184-190.
10. KAZAI, G. AND LALMAS, M. 2005. Notes on what to measure in inex. In Proceedings of the INEX Workshop on Element Retrieval Methodology, A. Trotman, M. Lalmas, and N. Fuhr, eds. INEX Proceedings. University of Otago, Univerisity of Glasgow. INEX Proceedings.
11. KAZAI, G., LALMAS, M., AND PIWOWARSKI, B. 2003. Inex guidelines for topic development.
12. KAZAI, G., LALMAS, M., AND VRIES, A. P. 2004. The overlap problem in content-oriented XML retrieval evaluation. In Proceedings of the 27th Annual International Conference on Research and Development in Information Retrieval. Sheffield, UK. ACM Press, 72-79.
13. KEKÄLÄINEN, J. AND JÄRVELIN, K. 2002. Using graded relevance assessments in IR evaluation. J. Ameri. Soc. Inform. Sci. 53, 13, 1120-1129.
14. LALMAS, M. AND MOUTOGIANNI, E. 2000. A dempster-shafer indexing for the focussed retrieval of a hierarchically structured document space: Implementation and experiments on a web museum collection. In 6th RIAO Conference, Content-Based Multimedia Information Access. Paris, France.
15. LAWRENCE, P., BRIN, S., MOTWANI, R., AND WINOGRAD, T. 1998. The pagerank citation ranking: Bringing order to the Web. Tech. rep., Stanford Digital Library Technologies Project.
16. MEADOW, C. T., KRAFT, D. H., AND BOYCE, B. R. 1999. Text Information Retrieval Systems. Academic Press, Orlando, FL.
17. PIWOWARSKI, B. AND GALLINARI, P. 2003. Expected ratio of relevant units: A measure for structured information retrieval. INEX Proceedings.
18. QUINTANA, Y., KAMEL, M., AND MCGEACHY, R. 1993. Formal methods for evaluating information retrieval in hypertext systems. In Proceedings of the 11th Annual International Conference on Systems Documentation. Kitchener-Waterloo, Ontario, Canada. ACM Press, 259-272.
19. RAGHAVAN, V. V., JUNG, G. S., AND BOLLMANN, P. 1989. A critical investigation of recall and precision as measures of retrieval system performance. ACM Trans. Inform. Syst. 7, 3, 205-229.
20. SAPORTA, G. 1990. Probabilités, analyse des données et statistique. Editions Technip, Paris, France.
21. VAN RIJSBERGEN, C. J. 1979. Information Retrieval. Butterworths.
22. VOORHEES, E. M. 2003. Common evaluation measures. The 12th Text Retrieval Conference (TREC'03). Number SP 500-255. NIST, 1-13.
23. VRIES, A., KAZAI, G., AND LALMAS, M. 2004. Tolerance to irrelevance: A user-effort oriented evaluation of retrieval systems without predefined retrieval unit. In Proceedings of RIAO (Recherche d'Information Assistée par Ordinateur (Computer Assisted Information Retrieval)). Avignon, France.