Using confidence bounds for exploitation-exploration trade-offs

Journal of Machine Learning Research

Volumn 3, Issue 3, 2003, Pages 397-422

  Auer, Peter ^a  

^a GRAZ UNIVERSITY OF TECHNOLOGY   (Austria)

Author keywords

Bandit Problem; Exploitation Exploration; Linear Value Function; Online Learning; Reinforcement Learning

Indexed keywords

ALGORITHMS; ONLINE SYSTEMS; RANDOM PROCESSES; STATISTICS;

CONFIDENCE BOUNDS;

LEARNING SYSTEMS;

EID: 0041966002     PISSN: 15324435     EISSN: None     Source Type: Journal    
DOI: 10.1162/153244303321897663     Document Type: Conference Paper

References (19)

1. N. Abe and P. M. Long. Associative reinforcement learning using linear probabilistic concepts. In Proc. 16th International Conf. on Machine Learning, pages 3-11. Morgan Kaufmann, San Francisco, CA, 1999.
2. N. Abe and A. Nakamura. Learning to optimally schedule internet banner advertisements. In Proc. 16th International Conf. on Machine Learning, pages 12-21. Morgan Kaufmann, San Francisco, CA, 1999.
3. R. Agrawal. Sample mean based index policies with O(log n) regret for the multi-armed bandit problem. Advances in Applied Probability, 27:1054-1078, 1995.
4. N. Alon and J. H. Spencer. The Probabilistic Method. Wiley, New York, 1992.
5. P. Auer. Using upper confidence bounds for online learning. In Proceedings of the 41th Annual Symposium on Foundations of Computer Science, pages 270-293. IEEE Computer Society, 2000.
6. P. Auer, N. Cesa-Bianchi, Y. Freund, and R. E. Schapire. Gambling in a rigged casino: The adversarial multi-armed bandit problem. In Proceedings of the 36th Annual Symposium on Foundations of Computer Science, pages 322-331. IEEE Computer Society Press, Los Alamitos, CA, 1995.
7. P. Auer, N. Cesa-Bianchi, Y. Freund, and R. E. Schapire. Gambling in a rigged casino: The adversarial multi-armed bandit problem. NeuroCOLT2 Technical Report NC2-TR-1998-025, Royal Holloway, University of London, 1998. Accessible via http at www.neurocolt.org.
8. P. Auer, N. Cesa-Bianchi, Y. Freund, and R. E. Schapire. Gambling in a rigged casino: The adversarial multi-armed bandit problem. Journal version, 2000.
9. P. Auer and M. K. Warmuth. Tracking the best disjunction. Machine Learning, 32:127-150, 1998. A preliminary version has appeared in Proceedings of the 36th Annual Symposium on Foundations of Computer Science.
10. K. Azuma. Weighted sums of certain dependent random variables. Tohoku Math. J., 3:357-367, 1967.
11. D. A. Berry and B. Fristedt. Bandit Problems. Chapman and Hall, 1985.
12. M. Herbster and M. K. Warmuth. Tracking the best expert. Machine Learning, 32:151-178, 1998.
13. W. Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58:13-30, 1963.
14. L. P. Kaelbling. Associative reinforcement learning: A generate and test algorithm. Machine Learning, 15:299-319, 1994a.
15. L. P. Kaelbling. Associative reinforcement learning: Functions in k-DNF. Machine Learning, 15:279-298, 1994b.
16. T. L. Lai and H. Robbins. Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics, 6:4-22, 1985.
17. N. Littlestone and M. K. Warmuth. The weighted majority algorithm. Information and Computation, 108(2):212-261, 1994.
18. H. Robbins. Some aspects of the sequential design of experiments. Bulletin American Mathematical Society, 55:527-535, 1952.
19. R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, 1998.