Diagnostic evaluation of information retrieval models

ACM Transactions on Information Systems

Volumn 29, Issue 2, 2011, Pages

  Fang, Hui ^a   Tao, Tao ^b   Zhai, Chengxiang ^c  

^a UNIVERSITY OF DELAWARE   (United States)

^b MICROSOFT   (United States)

^c University of Illinois at Urbana Champaign ^*  (United States)

Author keywords

Constraints; Diagnostic evaluation; Formal models; Retrieval heuristics; TF IDF weighting

Indexed keywords

CONSTRAINTS; DIAGNOSTIC EVALUATION; FORMAL MODEL; RETRIEVAL HEURISTICS; TF-IDF WEIGHTING;

INFORMATION RETRIEVAL;

FUNCTION EVALUATION;

EID: 80051496446     PISSN: 10468188     EISSN: 15582868     Source Type: Journal    
DOI: 10.1145/1961209.1961210     Document Type: Article

References (45)

1. AMATI, G. AND RIJSBERGEN, C. J. V. 2002. Probabilistic models of information retrieval based on measuring the divergence from randomness. ACM Trans. Inf. Syst. 20, 4, 357-389.
2. CARTERETTE, B. AND ALLAN, J. 2005. Incremental test collections. In Proceedings of the 14th International Conference on Information and Knowledge Management (CIKM'05).
3. CARTERETTE, B., ALLAN, J., AND SITARAMAN, R. 2006. Minimal test collections for retrieval evaluation. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval.
4. CORMACK, G. V., PALMER, C. R., AND CLARKE, C. L. 1998. Efficient construction of large test collections. In Proceedings of the ACM-SIGIR Conference on Research and Development in Information Retrieval.
5. FANG, H. 2008. A re-examination of query expansion using lexical resources. In Proceedings of the 46th Annual Meetings of the Association for Computational Linguistics.
6. FANG, H., TAO, T., AND ZHAI, C. 2004. A formal study of information retrieval heuristics. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval.
7. FANG, H. AND ZHAI, C. 2005. An exploration of axiomatic approaches to information retrieval. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval.
8. FANG, H. AND ZHAI, C. 2006. Semantic term matching in axiomatic approaches to information retrieval. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval.
9. FUHR, N. 1992. Probabilistic models in information retrieval. Comput. J. 35, 3, 243-255.
10. FUHR, N. 2001. Language models and uncertain inference in information retrieval. In Proceedings of the Language Modeling and IR Workshop. 6-11.
11. HARMAN, D. AND BUCKLEY, C. 2004. Sigir 2004 workshop: Ria and where can ir go from here. SIGIR Forum 38, 2, 45-49.
12. HE, B. AND OUNIS, I. 2005. A study of the dirichlet priors for term frequency normalisation. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval.
13. HIEMSTRA, D. 2000. A probabilistic justification for using tf-idf term wieghting in information retrieval. Int. J. Digital Libraries, 131-139.
14. LAFFERTY, J. AND ZHAI, C. 2003. Probabilistic relevance models based on document and query generation. In Language Modeling and Information Retrieval, W. B. Croft and J. Lafferty Eds., Kluwer Academic Publishers.
15. LAVRENKO, V. AND CROFT, B. 2001. Relevance-Based language models. In Proceedings of the Annual ACM SIGIR Conference on Research and Development in Information Retrieval. 120-127. (Pubitemid 32951064)
16. LOPRESTI, D. AND ZHOU, J. 1996. Retrieval strategy for noisy text. In Proceedings of the Symposium on Document Analysis and Information Retrieval.
17. PONTE, J. AND CROFT, W. B. 1998. A language modeling approach to information retrieval. In Proceedings of the Annual ACM SIGIR Conference on Research and Development in Information Retrieval. 275-281.
18. ROBERTSON, S. AND SPARCK JONES, K. 1976. Relevance weighting of search terms. J. Amer. Soc. Inf. Sci. 27, 129-146.
19. ROBERTSON, S. AND WALKER, S. 1994. Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In Proceedings of the Annual ACM SIGIR Conference on Research and Development in Information Retrieval. 232-241.
20. ROBERTSON, S. AND WALKER, S. 1997. On relevance weights with little relevance information. In Proceedings of the Annual ACM SIGIR Conference on Research and Development in Information Retrieval. 16-24. (Pubitemid 127720300)
21. ROBERTSON, S. E., WALKER, S., JONES, S., M. HANCOCK-BEAULIEU, M., AND GATFORD, M. 1995. Okapi at TREC-3. In Proceedings of the 3rd Text REtrieval Conference (TREC-3). D. K. Harman Ed., 109-126.
22. SALTON, G. 1989. Automatic Text Processing: The Transformation, Analysis and Retrieval of Information by Computer. Addison-Wesley.
23. SALTON, G. AND BUCKLEY, C. 1988. Term-Weighting approaches in automatic text retrieval. Inf. Process. Manag. 24, 513-523.
24. SALTON, G. AND MCGILL, M. 1983. Introduction to Modern Information Retrieval. McGraw-Hill.
25. SALTON, G., YANG, C. S., AND YU, C. T. 1975. A theory of term importance in automatic text analysis. J. Amer. Soc. Inf. Sci. 26, 1, 33-44.
26. SANDERSON, M. AND JOHO, H. 2004. Forming test collections with no system pooling. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval.
27. SHI, S., WEN, J.-R., YU, Q., SONG, R., AND MA, W.-Y. 2005. Gravitation-based model for information retrieval. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. 488-495.
28. SINGHAL, A. 2001. Modern information retrieval: A brief overview. Bull. IEEE Comput. Soc. Techn. Committee Data Engin. 24, 4, 35-43.
29. SINGHAL, A., BUCKLEY, C., AND MITRA, M. 1996a. Pivoted document length normalization. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. 21-29.
30. SINGHAL, A., SALTON, G., AND BUCKLEY, C. 1996b. Length normalization in degraded text collections. In Proceedings of the Symposium on Document Analysis and Information Retrieval. 149-162.
31. SINGHAL, A., CHOI, J., HINDLE, D., LEWIS, D. D., AND PEREIRA, F. C. N. 1998. ATT at TREC-7. In Proceedings of the Text REtrieval Conference. 186-198.
32. SOBOROFF, I., NICHOLAS, C., AND CAHAN, P. 2001. Ranking retrieval systems without relevance judgements. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval.
33. SPARCK JONES, K. AND WILLETT, P., EDS. 1997. Readings in Information Retrieval. Morgan Kaufmann Publishers.
34. TAO, T. AND ZHAI, C. 2007. An exploration of proximity measures in information retrieval. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval.
35. TURTLE, H. AND CROFT, W. B. 1991. Evaluation of an inference network-based retrieval model. ACM Trans. Inf. Syst. 9, 3, 187-222.
36. VAN RIJBERGEN, C. J. 1977. A theoretical basis for theuse of co-occurrence data in information retrieval. J. Document., 106-119.
37. VAN RIJSBERGEN, C. J. 1986. A non-classical logic for information retrieval. Comput. J. 29, 6.
38. VOORHEES, E. M. 2007. Trec: Continuing information retrievals tradition of experimentation. Comm. ACM 50, 11, 51-54. (Pubitemid 350083512)
39. WONG, K.-F., SONG, D., BRUZA, P., AND CHENG, C.-H. 2001. Application of aboutness to functional benchmarking in information retrieval. ACM Trans. Infor. Syst. 19, 4, 337-370. (Pubitemid 33720988)
40. WONG, S. K. M. AND YAO, Y. Y. 1995. On modeling information retrieval with probabilistic inference. ACM Trans. Inf. Syst. 13, 1, 69-99.
41. ZHAI, C. AND LAFFERTY, J. 2001a. Model-based feedback in the language modeling approach to information retrieval. In Proceedings of the 10th International Conference on Information and Knowledge Management (CIKM'01). 403-410. (Pubitemid 34974657)
42. ZHAI, C. AND LAFFERTY, J. 2001b. A study of smoothing methods for language models applied to ad hoc information retrieval. In Proceedings of the Annual ACM SIGIR Conference on Research and Development in Information Retrieval. 334-342. (Pubitemid 32957519)
43. ZHOU, Y. AND CROFT, W. B. 2006. Ranking robustness: a novel framework to predict query performance. In Proceedings of the 15th International Conference on Information and Knowledge Management (CIKM'06). 567.
44. ZOBEL, J. 1998. How reliable are the results of large-scale information retrieval experiments? In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval.
45. ZOBEL, J. AND MOFFAT, A. 1998. Exploring the similarity space. SIGIR Forum 31, 1, 18-34. (Pubitemid 128619001)