A risk minimization framework for information retrieval

Information Processing and Management

Volumn 42, Issue 1 SPEC. ISS, 2006, Pages 31-55

  Zhai, Chengxiang ^a   Lafferty, John ^b  

^a University of Illinois at Urbana Champaign   (United States)

^b Carnegie Mellon University   (United States)

Author keywords

Bayesian decision theory; Retrieval models; Risk minimization; Statistical language models

Indexed keywords

COMPUTER PROGRAMMING LANGUAGES; DECISION THEORY; PROBABILISTIC LOGICS; PROBLEM SOLVING; RISK ASSESSMENT; RISK MANAGEMENT; STATISTICAL METHODS;

BAYESIAN DECISION THEORY; RETRIEVAL MODELS; RISK MINIMIZATION; STATISTICAL LANGUAGE MODELS;

INFORMATION RETRIEVAL;

EID: 23644435310     PISSN: 03064573     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ipm.2004.11.003     Document Type: Article

References (66)

1. Allan, J., Gupta, R., & Khandelwal, V. (2001). Temporal summaries of news topics. In Proceedings of SIGIR 2001 (pp. 10-18).
2. Berger, A., & Lafferty, J. (1999). Information retrieval as statistical translation. In Proceedings of the 1999 ACM SIGIR conference on research and development in information retrieval (pp. 222-229).
3. D. Blei, A. Ng, and M. Jordan Latent Dirichlet allocation Journal of Machine Learning Research 3 2003 993 1022
4. P. Bollmann-Sdorra, and V. Raghavan On the delusiveness of adopting a common space for modeling IR objects-are queries documents? Journal of the American Society for Information Science 44 1993 579 587
5. A. Bookstein, and D. Swanson A decision theoretic foundation for indexing Journal for the American Society for Information Science 26 1975 45 50
6. Carbonell, J., & Goldstein, J. (1998). The use of MMR, diversity-based reranking for reordering documents and producing summaries. In Proceedings of SIGIR'1998.
7. Cooper, W. (1991). Some inconsistencies and misnomers in probabilistic IR. In Proceedings of SIGIR'91 (pp. 57-61).
8. Cooper, W. S. (1994). The formalism of probability theory in IR: A foundation for an encumbrance? In Proceedings of the 17th annual international ACM-SIGIR conference on research and development in information retrieval (pp. 242-247).
9. W.S. Cooper, and M.E. Maron Foundations of probabilistic and utility-theoretic indexing Journal of the ACM 25 1 1978 67 80
10. W.B. Croft Document representation in probabilistic models of information retrieval Journal of American Society for Information Science 1981 451 457
11. W.B. Croft, and D. Harper Using probabilistic models of document retrieval without relevance information Journal of Documentation 35 1979 285 295
12. S. Deerwester, S. Dumais, T. Landauer, G. Furnas, and R. Harshman Indexing by latent semantic analysis Journal of American Society for Information Science 41 1990 391 407
13. S. Dominich A unified mathematical definition of classical information retrieval Journal of the American Society for Information Science 51 6 2000 614 624
14. S. Dominich Mathematical foundations of Information Retrieval 2001 Kluwer Academic Publisher
15. Dominich, S. (2002). Paradox-free formal foundation of vector-space model. In Proceedings of the ACM SIGIR 2002 workshop on mathematical/formal methods in information retrieval (pp. 43-48).
16. Evans, D. A. & Zhai, C. (1996). Noun-phrase analysis in unrestricted text for information retrieval. In Proceedings of ACL 1996 (pp. 17-24).
17. Fox, E. (1983). Expending the boolean and vector space models of information retrieval with p-norm queries and multiple concept types. Phd. thesis, Cornell University.
18. N. Fuhr Probabilistic models in information retrieval The Computer Journal 35 3 1992 243 255
19. Fuhr, N. (2001). Language models and uncertain inference in information retrieval. In Proceedings of the language modeling and IR workshop. Extended abstract.
20. N. Fuhr, and C. Buckley A probabilistic learning approach for document indexing ACM Transactions on Information Systems 9 3 1991 223 248
21. R. Fung, and B.D. Favero Applying Bayesian networks to information retrieval Communications of the ACM 38 3 1995 42 48
22. Gey, F. (1994). Inferring probability of relevance using the method of logistic regression. In Proceedings of ACM SIGIR'94 (pp. 222-231).
23. S.P. Harter A probabilistic approach to automatic keyword indexing (part I & II) Journal of the American Society for Information Science 26 1975 197 206 (Part I), 280-289 (Part II)
24. Hiemstra, D., & Kraaij, W. (1998). Twenty-one at TREC-7: ad-hoc and cross-language track. In Proceedings of the seventh Text REtrieval Conference (TREC-7).
25. Kalt, T. (1996). A new probabilistic model of text classification and retrieval. Technical Report 78, CIIR, University of Massachusetts.
26. K.L. Kwok A network approach to probabilistic information retrieval ACM Transactions on Office Information System 13 1995 324 353
27. Lafferty, J., & Zhai, C. (2001). Document language models, query models, and risk minimization for information retrieval. In Proceedings of SIGIR'2001 (pp. 111-119).
28. Lafferty, J., & Zhai, C. (2003). Probabilistic relevance models based on document and query generation. In W. B. Croft, & J. Lafferty (Eds.), Language modeling and information retrieval. Kluwer Series on Information Retrieval.
29. Lavrenko, V., & Croft, B. (2001). Relevance-based language models. In Proceedings of SIGIR'2001 (pp. 120-127).
30. Lewis, D. D. (1992). Representation and learning in information retrieval. Technical Report 91-93, University of Massachusetts.
31. Lewis, D. D. (1998). Naive (bayes) at forty: the independence assumption in information retrieval. In European conference on machine learning (pp. 4-15).
32. M.E. Maron, and J.L. Kuhns On relevance, probabilistic indexing and information retrieval Journal of the ACM 7 1960 216 244
33. McCallum, A., & Nigam, K. (1998). A comparison of event models for naïve Bayes text classification. In: AAAI-1998 learning for text categorization workshop (pp. 41-48).
34. Miller, D. H., Leek, T., & Schwartz, R. (1999). A hidden Markov model information retrieval system. In Proceedings of the 1999 ACM SIGIR conference on research and development in information retrieval (pp. 214-221).
35. Mittendorf, E., & Schauble, P. (1994). Document and passage retrieval based on hidden Markov models. In: Proceedings of SIGIR'94 (pp. 318-327).
36. Over, P. (1998). TREC-6 interactive track report. In E. Voorhees & D. Harman (Eds.), The sixth Text REtrieval Conference (TREC-6) (pp. 73-82) nIST Special Publication 500-240.
37. Ponte, J., & Croft, W. B. (1998). A language modeling approach to information retrieval. In Proceedings of the ACM SIGIR (pp. 275-281).
38. Ribeiro, B. A. N., & Muntz, R. (1996). A belief network model for IR. In Proceedings of SIGIR'96 (pp. 253-260).
39. B. Ribeiro-Neto, I. Silva, and R. Muntz Bayesian network models for information retrieval F. Crestani G. Pasi Soft computing in information retrieval: Techniques and applications 2000 Springer-Verlag Berlin 259 291
40. S. Robertson, and K. Sparck Jones Relevance weighting of search terms Journal of the American Society for Information Science 27 1976 129 146
41. Robertson, S., & Walker, S. (1994). Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval. In Proceedings of SIGIR'94 (pp. 232-241).
42. Robertson, S., & Walker, S. (1997). On relevance weights with little relevance information. In Proceedings of SIGIR'97 (pp. 16-24).
43. S.E. Robertson The probability ranking principle in IR Journal of Documentation 33 4 1977 294 304
44. S.E. Robertson, M.E. Maron, and W.S. Cooper Probability of relevance: a unification of two competing models for information retrieval Information Technology-Research and Development 1 1982 1 21
45. S.E. Robertson, C.J.F. van Rijsbergen, and M. Porter Probabilistic models of indexing and searching Information retrieval research 1981 Butterworths London 35 56
46. G. Salton Automatic Text Processing: The Transformation, Analysis and Retrieval of Information by Computer 1989 Addison-Wesley
47. G. Salton, and C. Buckley Term-weighting approaches in automatic text retrieval Information Processing and Management 24 1988 513 523
48. G. Salton, and M. McGill Introduction to Modern Information Retrieval 1983 McGraw-Hill
49. G. Salton, A. Wong, and C.S. Yang A vector space model for automatic indexing Communications of the ACM 18 11 1975 613 620
50. G. Salton, C.S. Yang, and C.T. Yu A theory of term importance in automatic text analysis Journal of the American Society for Information Science 26 1 1975 33 44
51. Singhal, A., Buckley, C., Mitra, M. (1996). Pivoted document length normalization. In Proceedings of the 1996 ACM SIGIR conference on research and development in information retrieval (pp. 21-29).
52. Song, F., & Croft, B. (1999). A general language model for information retrieval. In Proceedings of the 1999 ACM SIGIR conference on research and development in information retrieval (pp. 279-280).
53. K. Sparck Jones, S. Walker, and S.E. Robertson A probabilistic model of information retrieval: development and comparative experiments-part 1 and part 2 Information Processing and Management 36 6 2000 779 808 and 809-840
54. Strzalkowski, T. (1997). NLP track at TREC-5. In D. Harman (Ed.) Proceedings of the fifth Text REtrieval Conference (TREC-5).
55. H. Turtle, and W.B. Croft Evaluation of an inference network-based retrieval model ACM Transactions on Information Systems 9 3 1991 187 222
56. C.J. van Rijbergen A theoretical basis for the use of co-occurrence data in information retrieval Journal of Documentation 1977 106 119
57. C.J. van Rijsbergen Information Retrieval 1979 Butterworths
58. C.J. van Rijsbergen A non-classical logic for information retrieval The Computer Journal 29 6 1986
59. S.K.M. Wong, P. Bollmann, and Y.Y. Yao Information retrieval based on axiomatic decision theory International Journal of General Systems 19 2 1991 101 117
60. S.K.M. Wong, and Y.Y. Yao A probability distribution model for information retrieval Information Processing and Management 25 1 1989 39 53
61. S.K.M. Wong, and Y.Y. Yao On modeling information retrieval with probabilistic inference ACM Transactions on Information Systems 13 1 1995 69 99
62. Zhai, C., (1997). Fast statistical parsing of noun phrases for document indexing. In 5th conference on applied natural language processing (ANLP-97) (pp. 312-319).
63. Zhai, C. (2002). Risk minimization and language modeling in text retrieval. Phd. thesis, Carnegie Mellon University.
64. Zhai, C., Cohen, W. W., & Lafferty, J. (2003). Beyond independent relevance: Methods and evaluation metrics for subtopic retrieval. In Proceedings of SIGIR 2003 (pp. 10-17).
65. Zhai, C., & Lafferty, J. (2001). Model-based feedback in the KL-divergence retrieval model. In Tenth international conference on information and knowledge management (CIKM 2001) (pp. 403-410).
66. Zhai, C. & Lafferty, J. (2002). Two-stage language models for information retrieval. In Proceedings of SIGIR'2002 (pp. 49-56).