Cost-sensitive machine learning for information retrieval

Cost-Sensitive Machine Learning

Volumn , Issue , 2011, Pages 265-295

  Szummer, Martin ^a   Radlinski, Filip ^b  

^a MICROSOFT RESEARCH   (United Kingdom)

^b MICROSOFT   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 83055165625     PISSN: None     EISSN: None     Source Type: Book    
DOI: None     Document Type: Chapter

References (59)

1. James Allan, Javed A. Aslamy, Ben Carterette, Virgil Pavlu, and Evangelos Kanoulas. Million query track 2008 overview. In Proceedings of Text REtrieval Conference, 2008.
2. Massih Reza Amini, Tuong Vinh Truong, and Cyril Goutte. A boosting algorithm for learning bipartite ranking functions with partially labeled data. In Proceedings of the ACM Conference on Research and Develop- ment in Information Retrieval (SIGIR), pages 99-106, 2008.
3. Peter Bailey, Nick Craswell, Ian Soboroff, Paul Thomas, Arjen P. de Vries, and Emine Yilmaz. Relevance assessment: Are judges exchangeable and does it matter? In Proceedings of the ACM Conference on Research and Development in Information Retrieval (SIGIR), pages 667-674, 2008.
4. Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. Manifold regularization:A geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, (7):2399-2434, 2006.
5. Justin Boyan, Dayne Freitag, and Thorsten Joachims. A machine learning architecture for optimizing web search engines. In AAAI Workshop on Internet Based Information Systems, 1996.
6. R. A. Bradley and M. E. Terry. The rank analysis of incomplete block designs. 1. The method of paired comparisons. Biometrika, 39:324-345, 1952.
7. Chris Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. Hullender. Learning to rank using gradient descent. In Proceedings of the International Conference on Machine Learning (ICML), pages 89-96, 2005.
8. Christopher J. C. Burges, Robert Ragno, and Quoc Viet Le. Learning to rank with nonsmooth cost functions. In Proceedings of the Interna- tional Conference on Advances in Neural Information Processing Systems (NIPS), pages 193-200, 2006.
9. Jamie Carbonell and Jade Goldstein. The use of MMR, diversity-based reranking for reordering documents and producing summaries. In Pro- ceedings of the ACM Conference on Research and Development in Infor- mation Retrieval (SIGIR), pages 335-336, 1998.
10. Ben Carterette, James Allan, and Ramesh Sitaraman. Minimal test collections for retrieval evaluation. In Proceedings of the ACM Conference on Research and Development in Information Retrieval (SIGIR), pages 268-275, New York, NY, 2006.
11. Ben Carterette, Paul Bennett, David Chickering, and Susan Dumais. Here or there: Preference judgments for relevance. In Proceedings of the European Conference on Information Retrieval (ECIR), pages 16-27, 2008.
12. Ben Carterette and Paul N. Bennett. Evaluation measures for preference judgments. In Proceedings of the ACM Conference on Research and Development in Information Retrieval (SIGIR), pages 685-686, 2008.
13. Nicolo Cesa-Bianchi and Gabor Lugosi. Prediction, Learning, and Games. Cambridge University Press, 2006.
14. Olivier Chapelle, Yi Chang, and Tie-Yan Liu, editors. Yahoo! Learning to Rank Challenge, 2010. http://learningtorankchallenge.yahoo.com/.
15. Harr Chen and David R. Karger. Less is more: Probabilistic models for retrieving fewer relevant documents. In Proceedings of the ACM Con- ference on Research and Development in Information Retrieval (SIGIR), pages 429-436, 2006.
16. Cheng Zhai, William W. Cohen, and John Lafferty. Beyond Independent Relevance: Methods and Evaluation Metrics for Subtopic Retrieval. In Proceedings of the ACM Conference on Research and Development in Information Retrieval (SIGIR), pages 10-17, 2003.
17. Wei Chu and Zoubin Ghahramani. Extensions of Gaussian processes for ranking: Semi-supervised and active learning. In NIPS Workshop on Learning to Rank, 2005.
18. Charles L. A. Clarke, Maheedhar Kolla, Gordon V. Cormack, Olga Vechtomova, Azin Ashkan, Stefan Büttcher, and Ian MacKinnon. Novelty and diversity in information retrieval evaluation. In Proceedings of the ACM Conference on Research and Development in Information Retrieval (SIGIR), pages 659-666, 2008.
19. Charles L. A. Clarke, Maheedhar Kolla, and Olga Vechtomova. An effectiveness measure for ambiguous and underspecified queries. In Pro- ceedings of the International Conference on the Theory of Information Retrieval (ICTIR), pages 188-199, 2009.
20. William W. Cohen, Robert E. Shapire, and Yoram Singer. Learning to order things. Journal of Artificial Intelligence Research, 10:243-270, 1999.
21. I.J. Cox, M.L. Miller, T.P. Minka, T.V. Papathomas, and P.N. Yianilos. The Bayesian image retrieval system, PicHunter: Theory, implementation, and psychophysical experiments. IEEE Transactions on Image Processing, 9(1):20-37, January 2000.
22. P. Diaconis and R. Graham. Spearman’s Footrule as a Measure of Disarray. Journal of the Royal Statistical Society, Series B, 39(2):262-268, 1977.
23. Pinar Donmez, Krysta M. Svore, and Christopher J. C. Burges. On the local optimality of Lambdarank. In Proceedings of the ACM Conference on Research and Development in Information Retrieval (SIGIR), pages 460-467, 2009.
24. Georges Dupret and Ciya Liao. A model to estimate intrinsic document relevance from the clickthrough logs of a web search engine. In WSDM '10: Proceedings of the Third ACM International Conference on Web Search and Data Mining, pages 181-190, New York, NY, 2010.
25. Ronald Fagin, Ravi Kumar, and D. Sivakumar. Comparing top k lists. SIAM Journal of Discrete Math, 17(1):134-160, 2003.
26. Filip Radlinski, Robert Kleinberg, and Thorsten Joachims. Learning diverse rankings with multi-armed bandits. In Proceedings of the Interna- tional Conference on Machine Learning (ICML), pages 784-791, 2008.
27. Y. Freund, R. Iyer, R.E. Schapire, and Y. Singer. An effcient boosting algorithm for combining preferences. Journal of Machine Learning Research, 4:933-969, 2003.
28. Mark E. Glickman. Parameter estimation in large dynamic paired comparison experiments. Applied Statistics, 48:377-394, 1999.
29. Kalervo Järvelin and Jaana Kekäläinen. IR evaluation methods for retrieving highly relevant documents. In Proceedings of the ACM Confer- ence on Research and Development in Information Retrieval (SIGIR), pages 41-48, 2000.
30. Thorsten Joachims. Optimizing search engines using clickthrough data. In International Conference on Knowledge Discovery and Data Mining (KDD), pages 133-142, 2002.
31. Thorsten Joachims. Optimizing Search Engines Using Clickthrough Data. In Proceedings of the ACM International Conference on Knowledge Dis- covery and Data Mining (KDD), pages 132-142, 2002.
32. Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, Filip Radlinski, and Geri Gay. Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search. ACM Transactions on Information Science (TOIS), 25(2), 2007. Article 7.
33. Jun Wang and Jianhan Zhu. Portfolio Theory of Information Retrieval. In Proceedings of the ACM Conference on Research and Development in Information Retrieval (SIGIR), pages 115-122, 2009.
34. Diane Kelly and Jaime Teevan. Implicit Feedback for Inferring User Preference: A Bibliography. SIGIR Forum, 32(2):18-28, 2003.
35. Charles Kemp and Kotagiri Ramamohanarao. Long-Term Learning for Web Search Engines. In Proceedings of the European Conference on Prin- ciples and Pratice of Knowledge Discovery in Databases (PKDD), pages 263-274, 2002.
36. M. Kendall and J. D. Gibbons. Rank Correlation Methods. Edward Arnold, London, 1990.
37. Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. In- troduction to Information Retrieval. Cambridge University Press, 2008.
38. John Marden. Analyzing and Modeling Rank Data. Chapman & Hall, 1995.
39. Matthew Streeter and Daniel Golovin. An Online Algorithm for Maximizing Submodular Functions. Technical Report CMU-CS-07-171, School of Computer Science, Carnegie Mellon University, December 2007.
40. Tom Minka and Stephen Robertson. Selection bias in the LETOR datasets, SIGIR 2008 Workshop on Learning to Rank for Information Retrieval, 2008.
41. Stefano Mizzaro. Relevance: The whole history. Journal of the American Society for Information Science, 48:810-832, 1997.
42. Foster Provost and Tom Fawcett. Analysis and visualization of classifier performance: Comparison under imprecise class and cost distributions. In Proceedings of the ACM International Conference on Knowledge Dis- covery and Data Mining (KDD), pages 43-48, 1997.
43. Tao Qin, Tie-Yan Liu, Jun Xu, and Hang Li. LETOR: A benchmark collection for research on learning to rank for information retrieval. In- formation Retrieval Journal, 13(4):346-374, 2010.
44. Filip Radlinski and Thorsten Joachims. Minimally invasive randomization for collecting unbiased preferences from clickthrough logs. In Pro- ceedings of the National Conference on Artificial Intelligence (AAAI), pages 1406-1412, 2006.
45. Filip Radlinski and Thorsten Joachims. Active exploration for learning rankings from clickthrough data. In Proceedings of the ACM International Conference on Knowledge Discovery and Data Mining (KDD), pages 570-579, 2007.
46. S. E. Robertson. The probability ranking principle in IR. Journal of Documentation, 33:294-304, 1977.
47. Mark Sanderson. Test collection based evaluation of information retrieval systems. Foundations and Trends in Information Retrieval, 4(4):247-375, 2010.
48. Matthias Seeger. Learning with labeled and unlabeled data. Unpublished. http://www.dai.ed.ac.uk/homes/seeger/, February 2001.
49. Agarwal Shivani. Ranking on graph data. In Proceedings of the Interna- tional Conference on Machine Learning (ICML), pages 25-32, 2006.
50. Aleksandrs Slivkins, Filip Radlinski, and Sreenivas Gollapudi. Learning optimally diverse rankings over large document collections. In Proceedings of the International Conference on Machine Learning (ICML), pages 983-990, 2010.
51. Martin Szummer and Emine Yilmaz. Semi-supervised ranking. In preparation.
52. Michael Taylor, John Guiver, Stephen Robertson, and Tom Minka. Softrank:Optimizing non-smooth rank metrics. In Proceedings of ACM In- ternational Conference on Web Search and Data Mining (WSDM), pages 77-86, 2008.
53. L.J.P. van der Maaten and G.E. Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, 9:2579-2605, 2008.
54. C. J. van Rijsbergen. Information Retrieval. utterworth & Co, 2nd edition, 1979.
55. Yisong Yue and Thorsten Joachims. Predicting diverse subsets using structural SVMs. In Proceedings of the International Conference on Ma- chine Learning (ICML), pages 1224-1231, 2008.
56. Yisong Yue, Rajan Patel, and Hein Roehrig. Beyond position bias: Examining result attractiveness as a source of presentation bias in clickthrough data. In WWW '10: Proceedings of the 19th International Conference on World Wide Web, pages 1011-1018, New York, NY, 2010.
57. ChengXiang Zhai and John Lafferty. A risk minimization framework for information retrieval. Information Processing & Management, 42(1):31-55, 2006.
58. Feimin Zhong, Dong Wang, Gang Wang, Weizhu Chen, Yuchen Zhang, Zheng Chen, and Haixun Wang. Incorporating post-click behaviors into a click model. In SIGIR '10: Proceeding of the 33rd International ACM SI-GIR Conference on Research and Development in Information Retrieval, pages 355-362, New York, NY, 2010.
59. Onno Zoeter, Michael Taylor, Ed Snelson, John Guiver, Nick Craswell, and Martin Szummer. A decision theoretic framework for ranking using implicit feedback. In SIGIR 2008 Workshop on Learning to Rank for Information Retrieval, 2008.