Action elimination and stopping conditions for the multi-armed bandit and reinforcement learning problems

Journal of Machine Learning Research

Volumn 7, Issue , 2006, Pages 1079-1105

  Even Bar, Eyal ^a   Mannor, Shie ^b   Mansour, Yishay ^c  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

^b MCGILL UNIVERSITY   (Canada)

^c TEL AVIV UNIVERSITY   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

MANNOR; MULTI-ARMED BANDIT; REINFORCEMENT LEARNING ALGORITHMS; REINFORCEMENT LEARNING PROBLEMS; TSITSIKLIS;

COMPUTER SIMULATION; LEARNING ALGORITHMS; MATHEMATICAL MODELS; PROBABILITY; PROBLEM SOLVING; ROBUSTNESS (CONTROL SYSTEMS); SPEED CONTROL; STATISTICAL METHODS;

LEARNING SYSTEMS;

EID: 33745295134     PISSN: 15337928     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (30)

1. W. A. Aiello, Y. Mansour, S. Rajagopolan, and A. Rosen. Competitive queue policies for differentiated services. In INFOCOM, 2000.
2. (To appear in J. of Algorithms).
3. D. Angluin and L. G. Valiant. Fast probabilistic algorithms for Hamiltonian circuits and matchings. Journal of Computer and System Sciences, 18:155-193, 1979.
4. P. Auer, N. Cesa-Bianchi, Y. Freund, and R. E. Schapire. Gambling in a rigged casino: The adversarial multi-armed bandit problem. In Proc. 36th Annual Symposium on Foundations of Computer Science, pages 322-331. IEEE Computer Society Press, 1995.
5. P. Auer, N. Cesa-Bianchi, Y. Freund, and R. E. Schapire. The non-stochastic multi-armed bandit problem. SIAM J. on Computing, 32(1):48-77, 2002.
6. D. A. Berry and B. Fristedt. Bandit Problems. Chapman and Hall, 1985.
7. D. P. Bertsekas and J. N. Tsitsiklis. Neuro-Dynamic Programming. Athena Scientific, 1995.
8. D. P. Bertsekas and J. N. Tsitsiklis. Neuro-Dynamic Programming. Athena Scientific, Belmont, MA, 1996.
9. V. S. Borkar and S.P Meyn. The O.D.E. method for convergence of stochastic approximation and reinforcement learning. SIAM J. Control Optim., 38(2):447-469, 2000.
10. E. Even-Dar and Y. Mansour. Learning rates for Q-learning. Journal of Machine Learning Research, 5:1-25, 2003.
11. (A preliminary version appeared in the Fourteenth Annual Conference on Computation Learning Theory (2001), 589-604.).
12. W. Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301): 13-30, 1963.
13. R. Howard. Dynamic programming and Markov decision processes. MIT press, 1960.
14. S. Kakade and J. Langford. Approximately optimal approximate reinforcement learning. In Proceedings of the Nineteenth International Conference on Machine Learning, pages 267-274. Morgan Kaufmann, 2002.
15. M. Kearns and S. Singh. Near-optimal reinforcement learning in polynomial time. Machine Learning, 49(2-3):209-232, 2002.
16. (A preliminary version appeared in ICML (1998), 260-268.).
17. M. Kearns and S. P. Singh. Finite-sample convergence rates for Q-learning and indirect algorithms. In Neural Information Processing Systems 10, pages 996-1002, 1998.
18. A. Kesselman, Z. Lotker, Y. Mansour, B. Patt-Shamir, B. Schieber, and M. Sviridenko. Buffer over-flow management in QoS switches. SIAM J. on Computing, 33(3):563-583, 2004.
19. (A preliminary version appeared in ACM Symposium on Theory of Computing (2001), 520-529.).
20. T. L. Lai and H. Robbins. Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics, 6:4-22, 1985.
21. J. MacQueen. A modified dynamic programming method for Markov decision problems. J. Math. Anal. Appl., 14:38-43, 1966.
22. S. Mannor and J. N. Tsitsiklis. The sample complexity of exploration in the multi-armed bandit problem. Journal of Machine Learning Research, 5:623-648, 2004.
23. (A preliminary version appeared in the Sixteenth Annual Conference on Computation Learning Theory (2003), 418-432.).
24. D. Ormoneit and S. Sen. Kernel-based reinforcement learning. Machine Learning, 49(2-3): 161-178, 2002.
25. M. Puterman. Markov Decision Processes. Wiley-Interscience, 1994.
26. H. Robbins. Some aspects of sequential design of experiments. Bull. Amer. Math. Soc., 55:527-535, 1952.
27. S. P. Singh and R. C. Yee. An upper bound on the loss from approximate optimal-value functions. Machine Learning, 16(3):227-233, 1994.
28. R. Sutton and A. Barto. Reinforcement Learning. 1998.
29. Cs. Szepesvri and R. Munos. Finite time bounds for sampling based fitted value iteration. In Proceedings of the 22nd International Conference on Machine Learning (ICML), page 881886, 2005.
30. C. Watkins. Learning from Delayed Rewards. PhD thesis, Cambridge University, 1989.