The K-armed dueling bandits problem

Journal of Computer and System Sciences

Volumn 78, Issue 5, 2012, Pages 1538-1556

  Yue, Yisong ^a   Broder, Josef ^b   Kleinberg, Robert ^c   Joachims, Thorsten ^c  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b Department of Obstetrics Gynecology   (United States)

^c School of Operations Research and Information Engineering   (United States)

Author keywords

Multi armed bandits; Online learning; Preference elicitation

Indexed keywords

INFORMATION THEORY;

CONVENTIONAL APPROACH; MULTI ARMED BANDIT; ONLINE LEARNING; PAIR-WISE COMPARISON; PARTIAL INFORMATION; PERCEIVED QUALITY; PREFERENCE ELICITATION; PRODUCT ATTRACTIVENESS;

E-LEARNING;

EID: 84861586270     PISSN: 00220000     EISSN: 10902724     Source Type: Journal    
DOI: 10.1016/j.jcss.2011.12.028     Document Type: Conference Paper

References (29)

1. Jean-Yves Audibert, Sébastien Bubeck, Minimax policies for adversarial and stochastic bandits, in: Conference on Learning Theory (COLT), 2009.
2. Peter Auer, Nicolò Cesa-Bianchi, and Paul Fischer Finite-time analysis of the multiarmed bandit problem Mach. Learn. 47 2 2002 235 256
3. Peter Auer, Nicolò Cesa-Bianchi, Yoav Freund, and Robert Schapire The nonstochastic multiarmed bandit problem SIAM J. Comput. 32 1 2002 48 77
4. Micah Adler, Peter Gemmell, Mor Harchol-Balter, Richard Karp, Claire Kenyon, Selection in the presence of noise: The design of playoff systems, in: ACM-SIAM Symposium on Discrete Algorithms (SODA), 1994.
5. Nir Ailon, Mehryar Mohri, An efficient reduction of ranking to classification, in: Conference on Learning Theory (COLT), 2008.
6. Peter Auer Using confidence bounds for exploitation-exploration trade J. Mach. Learn. Res. 3 2003 397 422
7. Maria-Florina Balcan, Nikhil Bansal, Alina Beygelzimer, Don Coppersmith, John Langford, Gregory Sorkin, Robust reductions from ranking to classification, in: Conference on Learning Theory (COLT), 2007.
8. Michael Ben-Or, Avinatan Hassidim, The bayesian learner is optimal for noisy binary search (and pretty good for quantum as well), in: IEEE Symposium on Foundations of Computer Science (FOCS), 2008.
9. Nicolò Cesa-Bianchi, Gábor Lugosi, and Gilles Stoltz Regret minimization under partial monitoring Math. Oper. Res. 31 3 2006 562 580
10. William Cohen, Robert Schapire, and Yoram Singer Learning to order things J. Artificial Intelligence Res. 10 1999 243 270
11. Thomas M. Cover, and Joy A. Thomas Elements of Information Theory 1999 J. Wiley
12. Eyal Even-Dar, Shie Mannor, and Yishay Mansour Action elimination and stopping conditions for the multi-armed bandit and reinforcement learning problems J. Mach. Learn. Res. 7 2006 1079 1105
13. Yoav Freund, Raj Iyer, Robert Schapire, and Yoram Singer An efficient boosting algorithm for combining preferences J. Mach. Learn. Res. 4 2003 933 969
14. Uriel Feige, Prabhakar Raghavan, David Peleg, and Eli Upfal Computing with noisy information SIAM J. Comput. 23 5 1994
15. Ralf Herbrich, Thore Graepel, Klaus Obermayer, Support vector learning for ordinal regression, in: International Conference on Artificial Neural Networks (ICANN), 1999.
16. Wassily Hoeffding Probability inequalities for sums of bounded random variables J. Amer. Statist. Assoc. 58 1963 13 30
17. Thorsten Joachims, A support vector method for multivariate performance measures, in: International Conference on Machine Learning (ICML), 2005.
18. Richard M. Karp, Robert Kleinberg, Noisy binary search and its applications, in: ACM-SIAM Symposium on Discrete Algorithms (SODA), 2007.
19. Robert Kleinberg, Alexandru Niculescu-Mizil, Yogeshwer Sharma, Regret bounds for sleeping experts and bandits, in: Conference on Learning Theory (COLT), 2008.
20. T.L. Lai, and Herbert Robbins Asymptotically efficient adaptive allocation rules Adv. in Appl. Math. 6 1985 4 22
21. Phil Long, Rocco Servedio, Boosting the area under the ROC curve, in: Proceedings of Neural Information Processing Systems (NIPS), 2007.
22. John Langford, Tong Zhang, The epoch-greedy algorithm for contextual multi-armed bandits, in: Proceedings of Neural Information Processing Systems (NIPS), 2007.
23. Rajeev Motwani, and Prabhakar Raghavan Randomized Algorithms 1995 Cambridge University Press
24. Shie Mannor, and John N. Tsitsiklis The sample complexity of exploration in the multi-armed bandit problem J. Mach. Learn. Res. 5 2004 623 648
25. Sandeep Pandey, Deepak Agarwal, Deepayan Chakrabarti, Vanja Josifovski, Bandits for taxonomies: A model-based approach, in: SIAM Conference on Data Mining (SDM), 2007.
26. Filip Radlinski, Madhu Kurup, Thorsten Joachims, How does clickthrough data reflect retrieval quality?, in: ACM Conference on Information and Knowledge Management (CIKM), 2008.
27. Herbert Robbins Some aspects of the sequential design of experiments Bull. Amer. Math. Soc. 58 1952 527 535
28. Yisong Yue, Josef Broder, Robert Kleinberg, Thorsten Joachims, The k-armed dueling bandits problem, in: Conference on Learning Theory (COLT), 2009.
29. Yisong Yue, Thorsten Joachims, Interactively optimizing information retrieval systems as a dueling bandits problem, in: International Conference on Machine Learning (ICML), 2009.