Generalized inverse document frequency

International Conference on Information and Knowledge Management, Proceedings

Volumn , Issue , 2008, Pages 399-408

  Metzler, Donald ^a  

^a YAHOO RESEARCH   (United States)

Author keywords

Estimation; Formal models; Inverse document frequency

Indexed keywords

AD-HOC RETRIEVAL TASKS; BAYESIAN PERSPECTIVE; EMPIRICAL EVALUATIONS; FORMAL MODELS; GENERALIZED INVERSE; INVERSE DOCUMENT FREQUENCY;

INFORMATION SERVICES; KNOWLEDGE MANAGEMENT;

FREQUENCY ESTIMATION;

EID: 70349257931     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1458082.1458137     Document Type: Conference Paper

References (24)

1. A. Aizawa. An information-theoretic perspective of TF-IDF measures. Information Processing and Management, 39(1):45-65, 2003.
2. C. Buckley, D. Dimmick, I. Soboroff, and E. Voorhees. Bias and the limits of pooling for large collections. Inf. Retr., 10(6):491-508, 2007.
3. W. S. Cooper. Some inconsistencies and misnomers in probabilistic information retrieval. In Proc. 14th Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, pages 57-61, New York, NY, USA, 1991. ACM.
4. W. B. Croft and D. Harper. Using probabilistic models of document retrieval without relevance information. Journal of Documentation, 35:285-295, 1979.
5. A. P. de Vries and T. Roelleke. Relevance information: a loss of entropy but a gain for IDF? In Proc 28th Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, pages 282-289, New York, NY, USA, 2005. ACM.
6. W. R. Greiff. A theory of term weighting based on exploratory data analysis. In Proc. 21st Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, pages 11-19, New York, NY, USA, 1998. ACM.
7. S. P. Harter. A probabilistic approach to automatic keyword indexing. Journal of the American Society for Information Science, 26:197-206 and 280-289, 1975.
8. B. He and I. Ounis. On setting the hyper-parameters of term frequency normalization for information retrieval. ACM Trans. Inf. Syst., 25(3):13, 2007.
9. K. S. Jones. A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation, 28:11-21, 1972.
10. J. Lafferty and C. Zhai. Probabilistic relevance models based on document and query generation. In W. B. Croft and J. Lafferty, editors, Language Modeling and Information Retrieval. 2003.
11. V. Lavrenko. A Generative Theory of Relevance. PhD thesis, University of Massachsetts Amherst, Amherst, MA, 2006.
12. L. Lee. IDF revisited: a simple new derivation within the Robertson-Spärck Jones probabilistic model. In Proc. 30th Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, pages 751-752, New York, NY, USA, 2007. ACM.
13. K. Papineni. Why inverse document frequency? In Proc 2nd Proc. North American Chapter of the Assn. for Computational Linguistics on Language Technologies, pages 1-8, Morristown, NJ, USA, 2001. Association for Computational Linguistics.
14. S. Robertson. The probability ranking principle in IR. Journal of Documentation, 33(4):294-304, 1977.
15. S. Robertson, S. Walker, S. Jones, M. M. Hancock-Beaulieu, and M. Gatford. Okapi at TREC-3. In Proc. 3rd Text REtrieval Conference, pages 109-126, 1994.
16. S. E. Robertson and K. S. Jones. Relevance weighting of search terms. Journal of the American Society for Information Science, 27(3):129-146, 1976.
17. S. E. Robertson, C. J. van Rijsbergen, and M. F. Porter. Probabilistic models of indexing and searching. In Proc. 3rd Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, pages 35-56, Kent, UK, UK, 1981. Butterworth & Co.
18. S. E. Robertson and S. Walker. Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In Proc. 17th Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, pages 232-241, New York, NY, USA, 1994. Springer-Verlag New York, Inc.
19. S. E. Robertson and S. Walker. On relevance weights with little relevance information. In Proc. 20th Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, pages 16-24, New York, NY, USA, 1997. ACM.
20. T. Roelleke. A frequency-based and a poisson-based definition of the probability of being informative. In Proc 26th Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, pages 227-234, New York, NY, USA, 2003. ACM.
21. A. Singhal, C. Buckley, and M. Mitra. Pivoted document length normalization. In Proc. 19th Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, pages 21-29, New York, NY, USA, 1996. ACM.
22. T. Strohman, D. Metzler, H. Turtle, and W. B. Croft. Indri: A language model-based serach engine for complex queries. In Proceedings of the International Conference on Intelligence Analysis, 2004.
23. Z. Xu and R. Akella. A new probabilistic retrieval model based on the dirichlet compound multinomial distribution. In Proc. 31st Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, page To appear., 2008.
24. C. Zhai and J. Lafferty. A study of smoothing methods for language models applied to information retrieval. ACM Trans. Inf. Syst., 22(2):179-214, 2004.