An actor-critic algorithm for constrained Markov decision processes

Systems and Control Letters

Volumn 54, Issue 3, 2005, Pages 207-213

  Borkar, V S ^a  

^a TATA INSTITUTE OF FUNDAMENTAL RESEARCH   (India)

Author keywords

Actor critic algorithms; Constrained Markov decision processes; Envelope theorem; Reinforcement learning; Stochastic approximation

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; DECISION THEORY; DYNAMIC PROGRAMMING; LEARNING SYSTEMS; THEOREM PROVING;

ACTOR-CRITIC ALGORITHMS; CONSTRAINED MARKOV DECISION PROCESSES; ENVELOPE THEOREM; REINFORCEMENT LEARNING; STOCHASTIC APPROXIMATION;

MARKOV PROCESSES;

EID: 13244278201     PISSN: 01676911     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.sysconle.2004.08.007     Document Type: Article

References (17)

1. E. Altman Constrained Markov Decision Processes 1999 Chapman & Hall/CRC Press Boca Raton, FL
2. E. Altman Applications of Markov decision processes in communication networks E.A. Feinberg A. Shwartz Handbook of Markov Decision Processes 2001 Kluwer Academic Publishers Dordrecht 489 536
3. M. Bardi, and I. Capuzzo-Dolcetta Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations 1997 Birkhauser Boston
4. A. Benveniste, M. Metivier, and P. Priouret Adaptive Algorithms and Stochastic Approximations 1990 Springer Berlin
5. D.P. Bertsekas, and J.N. Tsitsiklis Neuro-Dynamic Programming 1996 Athena Scientific Belmont, MA
6. V.S. Borkar Stochastic approximation with two time scales Systems Control Lett. 29 1997 291 294
7. V.S. Borkar Convex analytic methods in Markov decision processes E.A. Feinberg A. Shwartz Handbook of Markov Decision Processes 2001 Kluwer Academic Publishers Dordrecht 347 375
8. V.R. Konda, and V.S. Borkar Actor-critic-type learning algorithms for Markov decision processes SIAM J. Control Optim. 38 1999 94 123
9. V.R. Konda, and J.N. Tsitsiklis On actor-critic algorithms SIAM J. Control Optim. 42 2003 1143 1166
10. S. Mannor, and N. Shimkin A geometric approach to multi-criterion reinforcement learning J. Mach. Learn. Res. 5 2004 325 360
11. A. Mas-Colell, M.D. Whinston, and J.R. Green Microeconomic Theory 1995 Oxford University Press New York
12. P. Milgrom, and I. Segal Envelope theorems for arbitrary choice sets Econometrica 70 2002 583 601
13. A.B. Piunovskiy Optimal Control of Random Sequences in Problems with Constraints 1997 Kluwer Academic Publishers Dordrecht
14. M. Puterman Markov Decision Processes 1994 Wiley New York
15. J.N. Tsitsiklis, and B. Van Roy An analysis of temporal-difference learning with function approximation IEEE Trans. Automat. Control 42 1997 674 690
16. B. Van Roy Neuro-dynamic programming overview and recent trends E.A. Feinberg A. Shwartz Handbook of Markov Decision Processes 2001 Kluwer Academic Publishers Dordrecht 431 459
17. F.J. Vazquez Abad, V. Krishnamurthy, Self learning control of constrained Markov decision processes - a gradient approach, Les Cahiers du GERAD, 2003.