Online bandit learning against an adaptive adversary: From regret to policy regret

Proceedings of the 29th International Conference on Machine Learning, ICML 2012

Volumn 2, Issue , 2012, Pages 1503-1510

  Arora, Raman ^a   Dekel, Ofer ^b   Tewari, Ambuj ^c  

^a TOYOTA TECHNOLOGICAL INSTITUTE AT CHICAGO   (United States)

^b MICROSOFT RESEARCH   (United States)

^c University of Texas at Austin   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE ADVERSARY; FORMAL DEFINITION; GAME-THEORETIC; INTERNAL REGRET; ON-LINE ALGORITHMS; ONLINE LEARNING ALGORITHMS; STANDARD DEFINITIONS; SUBLINEAR; UNBOUNDED MEMORY;

GAME THEORY; LEARNING ALGORITHMS;

LEARNING SYSTEMS;

EID: 84867129684     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. Abernethy, J., Hazan, E., and Rakhlin, A. Competing in the dark: An efficient algorithm for bandit linear optimization. In COLT, pp. 263-274, 2008.
2. Auer, P., Cesa-Bianchi, N., Freund, Y., and Schapire, R. The nonstochastic multiarmed bandit problem. SIAM Journal on Computing, 32(1):48-77, 2002.
3. Awerbuch, B. and Kleinberg, R. D. Adaptive routing with end-to-end feedback: distributed learning and geometric approaches. In STOC, pp. 45-53, 2004.
4. Bartlett, P. L., Dani, V., Hayes, T. P., Kakade, S., Rakhlin, A., and Tewari, A. High-probability regret bounds for bandit online linear optimization. In COLT, pp. 335-342, 2008.
5. Blum, A. and Mansour, Y. From external to internal regret. JMLR, 8:1307-1324, 2007.
6. Borodin, A. and El-Yaniv, R. Online computation and competitive analysis. Cambridge University Press, 1998.
7. Cesa-Bianchi, N. and Lugosi, G. Prediction, learning, and games. Cambridge University Press, 2006.
8. Dani, V. and Hayes, T. P. Robbing the bandit: Less regret in online geometric optimization against an adaptive adversary. In SODA, 2006.
9. de Farias, D. P. and Megiddo, N. Combining expert advice in reactive environments. Journal of the ACM, 53(5): 762-799, 2006.
10. Dekel, O., Gilad-Bachrach, R., Shamir, Ohad, and Xiao, Lin. Optimal distributed online prediction. In ICML, 2011.
11. Even-Dar, E., Kakade, S. M., and Mansour, Y. Online Markov decision processes. Math. of Operations Research, 34(3):726-736, 2009.
12. Flaxman, A. D., Kalai, A. Tauman, and McMahan, B. H. Online convex optimization in the bandit setting: gradient descent without a gradient. In SODA, pp. 385-394, 2005.
13. Hazan, E. and Kale, S. Online submodular minimization. In Advances in Neural Information Processing Systems (NIPS), 2009.
14. Hazan, E., Kalai, A., Kale, S., and Agarwal, A. Logarithmic regret algorithms for online convex optimization. In COLT, 2006.
15. Kleinberg, R. Nearly tight bounds for the continuumarmed bandit problem. In NIPS, pp. 697-704, 2004.
16. Maillard, O. and Munos, R. Adaptive bandits: Towards the best history-dependent strategy. In AISTATS, 2010.
17. McMahan, H. B. and Blum, A. Online geometric optimization in the bandit setting against an adaptive adversary. In COLT, 2004.
18. Merhav, N., Ordentlich, E., Seroussi, C., and Weinberger, M.J. Sequential strategies for loss functions with memory. IEEE IT, 48(7):1947-1958, 2002.
19. Nesterov, Y. E. and Nemirovsky, A. S. Interior point polynomial algorithms in convex programming. SIAM, 1994.
20. Neu, G., György, A., Szepesvári, C., and Antos, A. Online Markov decision processes under bandit feedback. In NIPS, pp. 1804-1812, 2010.
21. Robbins, H. Some aspects of the sequential design of experiments. Bulletin of the AMS, 58:527-535, 1952.
22. Ryabko, D. and Hutter, M. On the possibility of learning in reactive environments with arbitrary dependence. Theor. Comput. Sci., 405(3):274-284, 2008.
23. Szepesvári, C. Algorithms for Reinforcement Learning. Synth. Lectures in A.I. and Machine Learning. Morgan & Claypool Publishers, 2010.
24. Yu, J. Y., Mannor, S., and Shimkin, N. Markov decision processes with arbitrary reward processes. Math. of Operations Research, 34(3):737-757, 2009.
25. Zinkevich, M. Online convex programming and generalized infinitesimal gradient ascent. In ICML, 2003.