Probabilistic Models of Information Retrieval Based on Measuring the Divergence from Randomness

ACM Transactions on Information Systems

Volumn 20, Issue 4, 2002, Pages 357-389

  Amati, Gianni ^a   Van Rijsbergen, Cornelis Joost ^b  

^a FONDAZIONE UGO BORDONI   (Italy)

^b UNIVERSITY OF GLASGOW   (United Kingdom)

Author keywords

Aftereffect model; Algorithms; binomial law; BM25; Bose Einstein statistics; document length normalization; eliteness; Experimentation; idf; information retrieval; Laplace; Poisson; probabilistic models; randomness; succession law; term frequency normalization; term weighting

Indexed keywords

AFTEREFFECT MODEL; BINOMIAL LAW; BOSE-EINSTEIN STATISTICS; DOCUMENT LENGTH NORMALIZATION; PROBABILISTIC MODELS; SUCCESSION LAW; TERM FREQUENCY NORMALIZATION; TERM WEIGHTING;

LAPLACE TRANSFORMS; MATHEMATICAL MODELS; POLYNOMIALS; PROBABILITY; QUERY LANGUAGES; RANDOM PROCESSES;

INFORMATION RETRIEVAL;

EID: 33750388702     PISSN: 10468188     EISSN: 15582868     Source Type: Journal    
DOI: 10.1145/582415.582416     Document Type: Article

References (38)

1. Allan, J., Callan, J. P., Croft, W. B., Ballesteros, L., Broglio, J., Xu, J., and Shu, H. 1996. IN-QUERY at TREC-5. In Proceedings of the Fifth Text REtrieval Conference (TREC-5). NIST Special Publication 500-238, Gaithersburg, Md., 119-132
2. Amati, G., Carpineto, C., and Romano, G. 2001. FUB at TREC 10 web track: A probabilistic framework for topic relevance term weighting. In Proceedings of the Tenth Text Retrieval Conference (TREC-10). NIST Special Publication 500-250, Gaithersburg, Md
3. Bookstein, A. and Swanson, D. 1974. Probabilistic models for automatic indexing. J. Am. Soc. Inf. Sci. 25, 312-318
4. Carpineto, C. and Romano, G. 2000. Trec-8 automatic ad-hoc experiments at fub. In Proceedings of the Eighth Text REtrieval Conference (TREC-8). NIST Special Publication 500-246, Gaithersburg, Md., 377-380
5. Cooper, W. and Maron, M. 1978. Foundations of probabilistic and utility-theoretic indexing. J. ACM 22, 5, 67-80
6. Cox, R. T. 1961. The Algebra of Probable Inference. Johns Hopkins Press, Baltimore, Md
7. Croft, W. and Harper, D. 1979. Using probabilistic models of document retrieval without relevance information. J. Doc. 35, 285-295
8. Damerau, F. 1965. An experiment in automatic indexing. Am. Doc. 16, 283-289
9. Feller, W. 1968. An Introduction to Probability Theory and Its Applications, Vol. I, third ed. Wiley, New York
10. Fuhr, N. 1989. Models for retrieval with probabilistic indexing. Inf. Process. Manage. 25, 1, 55-72
11. Good, I. J. 1968. The Estimation of Probabilities: An Essay on Modern Bayesian Methods, Vol. 30. MIT Press, Cambridge, Mass
12. Harman, D. 1993. Overview of the Second Text REtrieval Conference (TREC-2). In Proceedings of the TREC Conference. NIST Special publication 500-215, Gaithersburg, Md, 1-20
13. Harter, S. P. 1974. A probabilistic approach to automatic keyword indexing. PhD Thesis, Graduate Library, The University of Chicago, Thesis No. T25146
14. Harter, S. P. 1975a. A probabilistic approach to automatic keyword indexing. Part I: On the distribution of specialty words in a technical literature. J. ASIS 26, 197-216
15. Harter, S. P. 1975b. A probabilistic approach to automatic keyword indexing. Part II: An algorithm for probabilistic indexing. J. ASIS 26, 280-289
16. Hiemstra, D. and De Vries, A. 2000. Relating the new language models of information retrieval to the traditional retrieval models. Res. Rep. TR-CTIT-00-09, Centre for Telematics and Information Technology
17. Hintikka, J. 1970. On semantic information. In Information and Inference, J. Hintikka, and P. Suppes, Eds., Synthese Library. D. Reidel, Dordrecht, Holland, 3-27
18. Kwok, K. 1990. Experiments with component theory of probabilistic information retrieval based on single terms as document components. ACM Trans. Inf. Syst. 8, 4, 363-386
19. Lafferty, J. and Zhai, C. 2001. Document language models, query models, and risk minimization for information retrieval. In Proceedings of ACM SIGIR (New Orleans), ACM, New York, 111-119
20. Margulis, E. 1992. N-Poisson document modelling. In Proceedings of ACM-SIGIR 92 Conference (Denmark), ACM, New York, 177-189
21. Ponte, J. and Croft, B. 1998. A language modeling approach in information retrieval. In Proceedings of the 21st ACM SIGIR Conference on Research and Development in Information Retrieval, (Melbourne, Australia), B. Croft, A. Moffat, and C. van Rijsbergen, Eds., ACM, New York, 275-281
22. Popper, K. 1995. The Logic of Scientific Discovery (The bulk of the work was first published in Vienna in 1935; this reprint was first published by Hutchinson in 1959, new notes and footnotes in the present reprint). Routledge, London
23. Renyi, A. 1969. Foundations of Probability. Holden-Day, San Francisco
24. Robertson, S. 1986. On relevance weight estimation and query expansion. J. Doc. 42, 3, 288-297
25. Robertson, S. and Walker, S. 1994. Some simple approximations to the 2-Poisson model for probabilistic weighted retrieval. In Proceedings of the Seventeenth Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval (Dublin), SpringerVerlag, New York, 232-241
26. Robertson, S., Walker, S., Beaulieu, M., Gatford, M., and Payne, A. 1996. Okapi at Trec-4. In NIST Special Publication 500-2236: The Fourth Text REtrieval Conference (TREC-4), D. Harman, Ed., Department of Commerce, National Institute of Standards and Technology, Gaithersburg, Md., 182-191
27. Robertson, S. E. and Sparck-Jones, K. 1976. Relevance weighting of search terms. J. Am. Soc. Inf. Sci. 227, 129-146
28. Robertson, S. E., Van Rijsbergen, C. J., and Porter, M. 1981. Probabilistic models of indexing and searching. In Information Retrieval Research, S. E. Robertson, C. J. van Rijsbergen, and P. Williams, Eds., Butterworths, Oxford, UK, Chapter 4, 35-56
29. Salton, G. and Buckley, C. 1988. Term-weight approaches in automatic text retrieval. Inf. Process. Manage. 24, 5, 513-523
30. Singhal, A., Salton, G., Mitra, M., and Buckley, C. 1996. Document length normalization. Inf. Process. Manage. 32, 5, 619-633
31. Solomonoff, R. 1964a. A formal theory of inductive inference. Part I. Inf. Control 7, 1 (March), 1-22
32. Solomonoff, R. 1964b. A formal theory of inductive inference. Part II. Inf. Control 7, 2 (June), 224-254
33. Titterington, D. M., Smith, A. F. M., and Makov, U. E. 1985. Statistical Analysis of Finite Mixture Distributions. Wiley, Chichester
34. Turtle, H. and Croft, W. 1992. A comparison of text retrieval models. Comput. J. 35, 3 (June), 279-290
35. Van Rijsbergen, C. 1977. A theoretical basis for the use of co-occurrence data in information retrieval. J. Doc. 33, 106-119
36. Willis, D. 1970. Computational complexity and probability constructions. J. ACM 17, 2, 241-259
37. Witten, I. H., Moffat, A., and Bell, T. C. 1999. Managing Gigabytes, second ed. Morgan Kaufmann, San Francisco
38. Wong, S. and Yao, Y. 1995. On modeling information retrieval with probabilistic inference. ACM Trans. Inf. Syst. 16, 38-68