Bias-variance analysis in estimating true query model for information retrieval

Information Processing and Management

Volumn 50, Issue 1, 2014, Pages 199-217

  Zhang, Peng ^a   Song, Dawei ^a,b   Wang, Jun ^c   Hou, Yuexian ^a  

^a TIANJIN UNIVERSITY   (China)

^b OPEN UNIVERSITY   (United Kingdom)

^c UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

Bias variance; Information retrieval; Query language model

Indexed keywords

ANALYSIS FRAMEWORKS; BIAS-VARIANCE; BIAS-VARIANCE TRADEOFFS; ESTIMATION QUALITY; EVALUATION STRATEGIES; FUNDAMENTAL THEORY; QUERY LANGUAGE MODEL; RETRIEVAL PERFORMANCE;

COMPUTATIONAL LINGUISTICS; ESTIMATION; QUERY LANGUAGES;

INFORMATION RETRIEVAL;

EID: 84885100338     PISSN: 03064573     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ipm.2013.08.004     Document Type: Article

References (45)

1. Abdul-Jaleel, N.; Allan, J.; Croft, W. B.; Diaz, F.; Larkey, L.; Li, X.; et al. (2004). Umass at trec 2004: Novelty and hard. In TREC '04.
2. G. Amati, C. Carpineto, G. Romano, and F.U. Bordoni Query difficulty, robustness and selective application of query expansion European conference on IR research 2004 Springer 127 137
3. D. Banks, P. Over, and N.-F. Zhang Blind men and elephants: Six approaches to TREC data Information Retrieval 1 1-2 1999 7 34
4. C.M. Bishop Pattern recognition and machine learning (information science and statistics) 2006 Springer-Verlag New York, Inc. Secaucus, NJ, USA
5. Brain, D.; & Webb, G. (1999). On the effect of data set size on bias and variance in classification learning. In Proceedings of the fourth Australian knowledge acquisition workshop (pp. 117-128). University of New South Wales.
6. Collins-Thompson, K. (2009a). Accounting for stability of retrieval algorithms using risk-reward curves. In Proceedings of the SIGIR 2009 workshop on the future of IR evaluation (pp. 27-28).
7. K. Collins-Thompson Reducing the risk of query expansion via robust constrained optimization CIKM '09 2009 ACM New York, NY, USA 837 846
8. Collins-Thompson, K.; & Callan, J. (2007). Estimation and use of uncertainty in pseudo-relevance feedback. In SIGIR '07 (pp. 303-310).
9. Dillon, J.V.; & Collins-Thompson, K. (2010). A unified optimization framework for robust pseudo-relevance feedback algorithms. In CIKM '10 (pp. 1069-1078).
10. R.O. Duda, P.E. Hart, and D.G. Stork Pattern classification second ed. 2001 Wiley-Interscience
11. Fuhr, N. (2001). Language models and uncertain inference in information retrieval. In Proceedings of the language modeling and IR workshop (pp. 6-11).
12. S. Geman, E. Bienenstock, and R. Doursat Neural networks and the bias/variance dilemma Neural Computation 4 1992 1 58
13. Ghahramani, Z.; Ghahramani, Z.; & Chul Kim, H. (2003). Bayesian classifier combination.
14. K. Hofmann, S. Whiteson, and M. de Rijke Balancing exploration and exploitation in listwise and pairwise online learning to rank for information retrieval Information Retrieval Journal 2012
15. Karimzadehgan, M.; & Zhai, C. (2010). Exploration-exploitation tradeoff in interactive relevance feedback. In CIKM '10 (pp. 1397-1400).
16. M. Karimzadehgan, and C. Zhai A learning approach to optimizing exploration-exploitation tradeoff in relevance feedback Journal of Machine Learning Research 2012
17. Lafferty, J.D.; & Zhai, C. (2001). Document language models, query models, and risk minimization for information retrieval. In SIGIR '01 (pp. 111-119).
18. J.D. Lafferty, and C. Zhai Probabilistic relevance models based on document and query generation Language modeling and information retrieval 2003 Kluwer Academic Publishers 1 10
19. Lavrenko, V.; & Croft, W.B. (2001). Relevance-based language models. In SIGIR '01 (pp. 120-127).
20. Lebanon, G. (2010). Bias, variance, and MSE of estimators.
21. X. Li A new robust relevance model in the language model framework Information Processing & Management 44 3 2008 991 1007 (Pubitemid 351375309)
22. Lipka, N.; & Stein, B. (2011). Robust models in information retrieval. In 8th International workshop on text-based information retrieval (TIR).
23. Lv, Y.; & Zhai, C. (2009). Adaptive relevance feedback in information retrieval. In CIKM '09 (pp. 255-264).
24. Lv, Y.; Zhai, C.; & Chen, W. (2011). A boosting approach to improving pseudo-relevance feedback. In SIGIR '11 (pp. 165-174).
25. M.E. Maron, and J.L. Kuhns On relevance, probabilistic indexing and information retrieval Journal of ACM 7 1960 216 244
26. Ogilvie, P.; & Callan, J. (2002). Experiments using the lemur toolkit. In TREC '02 (pp. 103-108).
27. C. Perlich, F.J. Provost, and J.S. Simonoff Tree induction vs. logistic regression: a learning-curve analysis Journal of Machine Learning Research 4 2003 211 255
28. Ponte, J.M.; & Croft, W.B. (1998). A language modeling approach to information retrieval. In SIGIR '98 (pp. 275-281).
29. van Rijsbergen, C.J. (1997). Readings in information retrieval, A non-classical logic for information retrieval (pp. 268-272).
30. Robertson, S.E.; & Kanoulas, E. (2012). On per-topic variance in IR evaluation. In SIGIR '12 (pp. 891-900).
31. S.E. Robertson, and H. Zaragoza The probabilistic relevance framework: Bm25 and beyond Foundations and Trends in Information Retrieval 3 4 2009 333 389
32. K. Spärck Jones, S. Robertson, D. Hiemstra, and H. Zaragoza Language modelling and relevance Language modeling and information retrieval 2003 Kluwer Academic Publishers p. 57-31
33. Tao, T.; & Zhai, C. (2006). Regularized estimation of mixture models for robust pseudo-relevance feedback. In SIGIR '06 (pp. 162-169).
34. G. Valentini, T.G. Dietterich, and N. Cristianini Bias-variance analysis of support vector machines for the development of SVM-based ensemble methods Journal of Machine Learning Research 5 2004 725 775
35. Wang, J. (2009). Mean-variance analysis: A new document ranking theory in information retrieval. In ECIR (pp. 4-16).
36. Wang, J.; & Zhu, J. (2009). Portfolio theory of information retrieval. In SIGIR '09 (pp. 115-122).
37. Zhai, C. (2007). A brief review of information retrieval models. Technical report, Dept. of Computer Science, UIUC.
38. Zhai, C.; & Lafferty, J.D. (2001). A study of smoothing methods for language models applied to ad hoc information retrieval. In SIGIR '01 (pp. 334-342).
39. Zhang, P.; Hou, Y.; & Song, D. (2009). Approximating true relevance distribution from a mixture model based on irrelevance data. In SIGIR '09 (pp. 107-114).
40. Zhang, P.; Song, D.; Zhao, X.; & Hou, Y. (2010). A study of document weight smoothness in pseudo relevance feedback. In AIRS '10 (pp. 527-538).
41. Zhang, P.; Song, D.; Wang, J.; Zhao, X.; & Hou, Y. (2011). On modeling rank-independent risk in estimating probability of relevance. In AIRS '11 (pp. 13-24).
42. P. Zhang, D. Song, J. Wang, and Y. Hou Bias-variance decomposition of IR evaluation SIGIR '13 2013 ACM 1021 1024
43. Zhu, J.; Wang, J.; Cox, I.J.; & Taylor, M.J. (2009). Risky business: Modeling and exploiting uncertainty in information retrieval. In SIGIR '09 (pp. 99-106).
44. Zighelnic, L.; & Kurland, O. (2008). Query-drift prevention for robust query expansion. In SIGIR '08 (pp. 825-826).
45. Zucchini, W.; Berzel, A.; & Nenadic, O. (2005). Applied smoothing techniques. In Lecture notes. Institute for Statistics and Econometrics, University of Gottingen.