New algorithms of the Q-learning type

Automatica

Volumn 44, Issue 4, 2008, Pages 1111-1119

  Bhatnagar, Shalabh ^a   Babu, K Mohan ^b  

^a INDIAN INSTITUTE OF SCIENCE   (India)

^b Motorola India Pvt Ltd   (India)

Author keywords

Markov decision processes; Q learning; Reinforcement learning; SPSA; Two timescale stochastic approximation

Indexed keywords

APPROXIMATION ALGORITHMS; LEARNING ALGORITHMS; MARKOV PROCESSES; ROUTING ALGORITHMS; TELECOMMUNICATION NETWORKS;

MARKOV DECISION PROCESSES; Q-LEARNING; TWO TIMESCALE STOCHASTIC APPROXIMATION;

REINFORCEMENT LEARNING;

EID: 41049095293     PISSN: 00051098     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.automatica.2007.09.009     Document Type: Article

References (10)

1. Bertsekas D.P., and Tsitsiklis J.N. Neuro-dynamic programming (1996), Athena Scientific, Belmont, MA
2. Bhatnagar S., Fu M.C., Marcus S.I., and Wang I.-J. Two-timescale simultaneous perturbation stochastic approximation using deterministic perturbation sequences. ACM Transactions on Modelling and Computer Simulation 13 2 (2003) 180-209
3. Borkar V.S. Stochastic approximation with two time scales. Systems Control Letters 29 (1997) 291-294
4. Boyan J.A., and Littman M.L. Packet routing in dynamically changing networks: A reinforcement learning approach. Advances in Neural Information Processing Systems 6 (1994) 671-678
5. Konda V.R., and Borkar V.S. Actor-critic like learning algorithms for Markov decision processes. SIAM Journal on Control and Optimization 38 1 (1999) 94-123
6. Marbach P., Mihatsch O., and Tsitsiklis J.N. Call admission control and routing in integrated services networks using neuro-dynamic programming. IEEE Journal on Selected Areas in Communication 18 (2000) 197-208
7. Spall J.C. Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Transactions on Automatic Control 37 (1992) 332-341
8. Spall J.C. A one-measurement form of simultaneous perturbation stochastic approximation. Automatica 33 (1997) 109-112
9. Tsitsiklis J.N. Asynchronous stochastic approximation and Q-learning. Machine Learning 16 (1994) 185-202
10. Watkins J.C.H., and Dayan P. Q-learning. Machine Learning 8 (1992) 279-292