An asymptotically optimal policy for finite support models in the multiarmed bandit problem

Machine Learning

Volumn 85, Issue 3, 2011, Pages 361-391

  Honda, Junya ^a   Takemura, Akimichi ^a  

^a UNIVERSITY OF TOKYO   (Japan)

Author keywords

Bandit problems; Convex optimization; Finite time regret; MED policy

Indexed keywords

ASYMPTOTIC BOUNDS; ASYMPTOTICALLY OPTIMAL; BANDIT PROBLEMS; CONVEX OPTIMIZATION TECHNIQUES; EXPLORATION AND EXPLOITATION; FINITE SUPPORTS; FINITE TIME; FINITE-TIME REGRET; MULTI-ARMED BANDIT PROBLEM; MULTIPLE ARMS; UPPER AND LOWER BOUNDS; UPPER BOUND;

CONVEX OPTIMIZATION; REINFORCEMENT LEARNING;

OPTIMIZATION;

EID: 83155180573     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-011-5257-4     Document Type: Article

References (23)

1. Agrawal, R. (1995a). The continuum-armed bandit problem. SIAM Journal on Control and Optimization, 33, 1926-1951.
2. Agrawal, R. (1995b). Sample mean based index policies with o (log n) regret for the multi-armed bandit problem. Advances in Applied Probability, 27, 1054-1078.
3. Audibert, J.-Y., & Bubeck, S. (2009). Minimax policies for adversarial and stochastic bandits. In Proceedings of COLT 2009. Montreal: Omnipress.
4. Auer, P., Cesa-Bianchi, N., & Fischer, P. (2002a). Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47, 235-256. (Pubitemid 34126111)
5. Auer, P., Cesa-Bianchi, N., Freund, Y., & Schapire, R. E. (2002b). The nonstochastic multiarmed bandit problem. SIAM Journal on Computing, 32, 48-77.
6. Boyd, S., & Vandenberghe, L. (2004). Convex optimization. Cambridge: Cambridge University Press.
7. Burnetas, A. N., & Katehakis, M. N. (1996). Optimal adaptive policies for sequential allocation problems. Advances in Applied Mathematics, 17, 122-142. (Pubitemid 126160351)
8. Cover, T. M., & Thomas, J. A. (2006). Elements of information theory (2nd edn.). New York: Wiley-Interscience.
9. Even-Dar, E., Mannor, S., & Mansour, Y. (2002). Pac bounds for multi-armed bandit and Markov decision processes. In Proceedings of COLT 2002 (pp. 255-270). London: Springer.
10. Fiacco, A. V. (1983). Introduction to sensitivity and stability analysis in nonlinear programming. New York: Academic Press.
11. Gittins, J. C. (1989). Multi-armed bandit allocation indices. Wiley-Interscience Series in Systems and Optimization. Chichester: Wiley.
12. Honda, J., & Takemura, A. (2010). An asymptotically optimal bandit algorithm for bounded support models. In Proceedings of COLT 2010, Haifa, Israel (pp. 67-79).
13. Ishikida, T., & Varaiya, P. (1994). Multi-armed bandit problem revisited. Journal of Optimization Theory and Applications, 83, 113-154.
14. Katehakis, M. N., & Veinott, A. F. Jr. (1987). The multi-armed bandit problem: decomposition and computation. Mathematics of Operations Research, 12, 262-268.
15. Kleinberg, R. (2005). Nearly tight bounds for the continuum-armed bandit problem. In Proceedings of NIPS 2005 (pp. 697-704). New York: MIT Press.
16. Lai, T. L., & Robbins, H. (1985). Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics, 6, 4-22.
17. Meuleau, N., & Bourgine, P. (1999). Exploration of multi-state environments: Local measures and backpropagation of uncertainty. Machine Learning, 35, 117-154.
18. Pollard, D. (1984). Convergence of stochastic processes. Springer Series in Statistics. New York: Springer.
19. Robbins, H. (1952). Some aspects of the sequential design of experiments. Bulletin of the American Mathematical Society, 58, 527-535.
20. Strens, M. (2000). A Bayesian framework for reinforcement learning. In Proceedings of ICML 2000 (pp. 943-950). San Francisco: Kaufmann.
21. Vermorel, J., & Mohri, M. (2005). Multi-armed bandit algorithms and empirical evaluation. In Proceedings of ECML 2005, Porto, Portugal (pp. 437-448). Berlin: Springer.
22. Wyatt, J. (1997). Exploration and inference in learning from reinforcement. Doctoral dissertation, Department of Artificial Intelligence, University of Edinburgh.
23. Yakowitz, S., & Lowe, W. (1991). Nonparametric bandit methods. Annals of Operation Research, 28, 297-312.