Online-to-confidence-set conversions and application to sparse stochastic bandits

Journal of Machine Learning Research

Volumn 22, Issue , 2012, Pages 1-9

  Abbasi Yadkori, Yasin ^a   Pál, Dávid ^b   Szepesvári, Csaba ^a  

^a GOOGLE INC   (United States)

^b UNIVERSITY OF ALBERTA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE; LEARNING ALGORITHMS; STOCHASTIC SYSTEMS;

EXPONENTIATED GRADIENT ALGORITHMS; LINEAR COMBINATIONS; LINEAR PREDICTION; LINEAR PREDICTORS; NOVEL TECHNIQUES; ON-LINE ALGORITHMS; ONLINE LEARNING ALGORITHMS; ONLINE PREDICTION;

E-LEARNING;

EID: 84908661477     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (36)

1. Yasin Abbasi-Yadkori, Andras Antos, and Csaba Szepesvari. Forced-exploration based algorithms for playing in stochastic linear bandits. In COLT Workshop on On-line Learning with Limited Feedback, 2009.
2. Yasin Abbasi-Yadkori, David Pal, and Csaba Szepesvaari. Improved algorithms for linear stochastic bandits. In Advances in Neural Information Processing Systems, 2011a.
3. Yasin Abbasi-Yadkori, Daavid Paal, and Csaba Szepesvari. Online least squares estimation with self-normalized processes: An application to bandit problems. Arxiv preprint http://arxiv.org/abs/1102.2670, 2011b.
4. Andraas Antos and Csaba Szepesvaari. Personal Communication, 2009.
5. Jean Yves Audibert, Remi Munos, and Csaba Szepesvari. Exploration-exploitation trade-off using variance estimates in multi-armed bandits. In Theoretical Computer Science-2008, 2008.
6. Jean-Yves Audibert, Raemi Munos, and Csaba Szepesvari. Exploration-exploitation tradeoff using variance estimates in multi-armed bandits. Theoretical Computer Science, 410(19):1876-1902, 2009.
7. Peter Auer. Using confidence bounds for exploitation-exploration trade-offs. Journal of Machine Learning Research, 3:397-422, 2002.
8. Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. Finite time analysis of the multiarmed bandit problem. Machine Learning, 47(2-3):235-256, 2002a.
9. Peter Auer, Nicolo Cesa-Bianchi, and Claudio Gentile. Adaptive and self-confident on-line learning algorithms. Journal of Computer and System Sciences, 64(1):48-75, 2002b.
10. Peter L. Bartlett and Ambuj Tewari. REGAL: A reg-ularization based algorithm for reinforcement learning in weakly communicating MDPs. In Proceedings of the 25th Annual Conference on Uncertainty in Artificial Intelligence, 2009.
11. Emmanuel J. Candoes. Compressive sampling. In Proceedings of the International Congress of Mathematicians, volume 3, pages 1433-1452, 2006.
12. Alexandra Carpentier and Remi Munos. Bandit theory meets compressed sensing for high dimensional stochastic linear bandit. In Proceedings of fifteenth international conference on Artificial Intelligence and Statistics, 2012.
13. Nicoloo Cesa-Bianchi and Gaabor Lugosi. Prediction, Learning, and Games. Cambridge University Press, New York, NY, USA, 2006.
14. Nicoloo Cesa-Bianchi, Alex Conconi, and Claudio Gentile. On the generalization ability of on-line learning algorithms. IEEE Transactions on Information Theory, 50(9):2050-2057, 2004.
15. Arnak S. Dalalyan and Alexandre B. Tsybakov. Aggregation by exponential weighting and sharp oracle inequalities. In Proceedings of the 20th Annual Conference on Learning Theory, pages 97-111, 2007.
16. Varsha Dani, Thomas P. Hayes, and Sham M. Kakade. Stochastic linear optimization under bandit feed-back. In Rocco Servedio and Tong Zhang, editors, Proceedings of the 21st Annual Conference on Learning Theory (COLT 2008), pages 355-366, 2008.
17. Ofer Dekel and Yoram Singer. Data-driven online to batch conversions. NIPS 2005, 18:267, 2006.
18. Eyal Even-Dar, Shie Mannor, and Yishay Mansour. PAC bounds for multi-armed bandit and Markov decision processes. In Fifteenth Annual Conference on Computational Learning Theory (COLT), pages 255-270, 2002.
19. David A. Freedman. On tail probabilities for martingales. The Annals of Probability, 3(1):100-118, 1975.
20. Claudio Gentile and Nick Littlestone. The robustness of the p-norm algorithms. In Proceedings of the twelfth annual conference on Computational learning theory, pages 1-11, New York, NY, USA, 1999. ACM.
21. Sebastien Gerchinovitz. Sparsity regret bounds for individual sequences in online linear regression. In Proceedings of the 24st Annual Conference on Learning Theory (COLT 2011), 2011.
22. Adam J. Grove, Nick Littlestone, and Dale Schuur-mans. General convergence results for linear discriminant updates. In Machine Learning, pages 171-183. ACM Press, 1997.
23. Thomas Jaksch, Ronald Ortner, and Peter Auer. Near-optimal regret bounds for reinforcement learning. Journal of Machine Learning Research, 11:1563-1600, 2010.
24. Jyrki Kivinen and Manfred K. Warmuth. Exponentiated gradient versus gradient descent for linear predictors. Information and Computation, 132(1):1-63, January 1997.
25. Tze Leung Lai, Herbert Robbins, and Ching Zong Wei. Strong consistency of least squares estimates in multiple regression. Proceedings of the National Academy of Sciences, 75(7):3034-3036, 1979.
26. John Langford. Machine learning reductions, 2011. http://hunch.net/~jl/projects/reductions/reductions.html.
27. Lihong Li, Wei Chu, John Langford, and Robert E. Schapire. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th International Conference on World Wide Web (WWW 2010), pages 661-670. ACM, 2010.
28. Nick Littlestone. Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Machine Learning, 2(4):285-318, 1988.
29. Nicolas Littlestone. From on-line to batch learning. In Annual Workshop on Computational Learning Theory: Proceedings of the second annual workshop on Computational learning theory (COLT 1989). Association for Computing Machinery, Inc, One Astor Plaza, 1515 Broadway, New York, NY, 10036-5701, USA, 1989.
30. Shie Mannor and John N. Tsitsiklis. The sample complexity of exploration in the multi-armed bandit problem. Journal of Machine Learning Research, 5:623-648, 2004.
31. Volodymyr Mnih, Csaba Szepesvari, and Jean-Yves Audibert. Empirical bernstein stopping. In Andrew McCallum and Sam Roweis, editors, Proceedings of the 25th Annual International Conference on Machine Learning (ICML 2008), pages 672-679, 2008.
32. Paat Rusmevichientong and John N. Tsitsiklis. Linearly parameterized bandits. Mathematics of Operations Research, 35(2):395-411, 2010.
33. Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodolical), 58(1):267-288, 1996.
34. Vladimir Vovk. Competitive on-line statistics. International Statistical Review, 69:213-248, 2001.
35. Thomas J. Walsh, Istvaan Szita, Carlso Diuk, and Michael L. Littman. Exploring compact reinforcement-learning representations with linear regression. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (UAI 2009), pages 591-598. AUAI Press, 2009.
36. Larry Wasserman. All of Nonparametric Statistics. Springer, 1998.