Gradient-based policy iteration: An example

Proceedings of the IEEE Conference on Decision and Control

Volumn 3, Issue , 2002, Pages 3367-3371

  Cao, Xi Ren ^a   Fang, Hai Tao ^b  

^a HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

^b INSTITUTE OF SYSTEMS SCIENCE   (China)

Author keywords

Discrete event dynamic systems; Markov decision processes; Perturbation analysis; Poisson equations; Potentials; Q learning; W factors

Indexed keywords

GRADIENT METHODS; ITERATIVE METHODS; MARKOV PROCESSES; OPTIMIZATION; PERTURBATION TECHNIQUES; POISSON EQUATION; SENSITIVITY ANALYSIS;

DISCRETE EVENT DYNAMIC SYSTEMS; REINFORCEMENT LEARNING;

DISCRETE TIME CONTROL SYSTEMS;

EID: 0036992818     PISSN: 01912216     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (12)

1. D. P. Bertsekas, and J. N. Tsitsiklis, Neuro-Dynamic Programming, Athena Scientific, Belmont, Massachusetts, 1996.
2. X.R. Cao, The relations among potentials, perturbation analysis, and Markov decision processes, Discrete Event Dynamic Systems: Theory and Applications, Vol. 8, 71-87, 1998.
3. X.R. Cao, A unified approach to Markov decision problems and performance sensitivity analysis, Automatica, 36, 771-774, 2000.
4. X. R. Cao, From perturbation analysis to Markov decision processes and reinforcement learning, manuscript, 2002.
5. X. R. Cao and H. F. Chen, Potentials, perturbation realization, and sensitivity analysis of Markov processes, IEEE Transactions on Automat. Control, Vol. 42, 1382-1393, 1997.
6. D.R. Cox, A use of complex probabilities in the theory of stochastic processes, Proceedings Cambridge Philosophical Society, 51, 313-319, 1955.
7. C. Cassandras and S. Lafortune, Introduction to Discrete Event Dynamic Systems, Kluwer Academic Publishers, Boston, 1999.
8. Y. C. Ho and X. R. Cao, Perturbation Analysis of Discrete-Event Dynamic Systems, Kluwer Academic Publisher, Boston, 1991.
9. S.P. Meyn, The policy iteration algorithm for average reward Markov decision processes with general state space, IEEE Trans. Automat. Control, 42, 1663-1680, 1997.
10. M.L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, New York, 1994.
11. D. Revuz, Markov Chains, Elsevier, Amesterdam, 1984.
12. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, MIT Press, Cambridge, MA, 1998.