Local bandit approximation for optimal learning problems

Advances in Neural Information Processing Systems

Volumn , Issue , 1997, Pages 1019-1025

  Duff, Michael O ^a   Barto, Andrew G ^a  

^a Biologically Inspired Neural and Dynamical Systems Laboratory   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

MARKOV PROCESSES;

ADAPTIVE CONTROL; BANDIT PROCESS; BAYES-OPTIMAL; GITTINS INDEX; LEARNING PROBLEM; LEARNING STRATEGY; MARKOV DECISION PROCESSES;

OPTIMIZATION;

EID: 16244388049     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (15)

1. Bar-Shalom, Y. & Tse, E. (1976) Caution, probing and the value of information in the control of uncertain systems, Ann. Econ. Soc. Meas. 5:323-337.
2. R. Bellman & R. Kalaba, (1959) On adaptive control processes. IRE Trans., 4:1-9.
3. Bokar, V. & Varaiya, P.P. (1979) Adaptive control of Markov chains I: finite parameter set. IEEE Trans. Auto. Control 24:953-958.
4. Cozzolino, J.M., Gonzalez-Zubieta, R., & Miller, R.L. (1965) Markov decision processes with uncertain transition probabilities. Tech. Rpt. 11, Operations Research Center, MIT.
5. Dayan, P. & Sejnowski, T. (1996) Exploration Bonuses and Dual Control. Machine Learning (in press).
6. Duff, M.O. (1995) Q-learning for bandit problems, in Machine Learning: Proceedings of the Twelfth International Conference on Machine Learning: pp. 209-217.
7. Duff, M.O. (1997) Approximate computational methods for optimal learning and dual control. Technical Report, Deptartment of Computer Science, Univ. of Massachusetts, Amherst.
8. Feldbaum, A. (1965) Optimal Control Systems, Academic Press.
9. Gittins, J.C. & Jones, D. (1979) Bandit processes and dynamic allocation indices (with discussion). J. R. Statist. Soc. 5 41:148-177.
10. Katehakis, M.H. & Veinott, A.F. (1987) The multi-armed bandit problem: decomposition and computation Math. OR 12: 262-268.
11. MacQueen, J. (1966). A modified dynamic programming method for Markov decision problems, J. Math. Anal. Appl., 14:38-43.
12. Robinsion, D.R. (1981) Algorithms for evaluating the dynamic allocation index. Research Report No. 80/DRR/4, Manchester-Sheffield School of Probability and Statistics.
13. Tsitsiklis, J. (1993) A short proof of the Gittins index theorem. Proc. 32nd Conf. Dec. and Control: 389-390.
14. Varaiya, P.P., Walrand, J.C, & Buyukkoc, C. (1985) Extensions of the multiarmed bandit problem: the discounted case. IEEE Trans. Auto. Control 30(5):426-439.
15. Watkins, C. (1989) Learning from Delayed Rewards Ph.D. Thesis, Cambidge University.