Unified NDP method based on TD(0) learning for both average and discounted Markov decision processes

Kongzhi Lilun Yu Yingyong/Control Theory and Applications

Volumn 23, Issue 2, 2006, Pages 292-296

  Tang, Hao ^a   Zhou, Lei ^a   Yuan, Ji Bin ^a  

^a HEFEI UNIVERSITY OF TECHNOLOGY   (China)

Author keywords

Markov decision processes; Neuro dynamic programming; Performance potentials; TD(0) learning

Indexed keywords

LEARNING OPTIMIZATION PROBLEMS; MARKOV DECISION PROCESSES (MDPS); NEURO DYNAMIC PROGRAMMING (NDP); NEURO POLICY ITERATION ALGORITHM; PERFORMANCE POTENTIALS; REINFORCEMENT LEARNING (RL); TD(0) LEARNING; TEMPORAL DIFFERENCE;

DYNAMIC PROGRAMMING; ESTIMATION; EVALUATION; ITERATIVE METHODS; NEURAL NETWORKS; OPTIMIZATION; PERFORMANCE;

MARKOV PROCESSES;

EID: 33745951445     PISSN: 10008152     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (12)

1. CAO X R, CHEN H F. Perturbation realization, potentials and sensitivity analysis of Markov processes[J]. IEEE Trans on Automatic Control, 1997, 42(10): 1382-1393.
2. CAO X R. The relations among potentials, perturbation analysis, and Markov decision processes[J]. Discrete Event Dynamic Systems: Theory and Applications, 1998, 8(1): 71-78.
3. PUTERMAN M L. Markov Decision Processes: Discrete Stochastic Dynamic Programming[M]. New York: Wiley, 1994.
4. BERTSAKEAS D P, TSITSIKLIS J N. Neuro-Dynamic Programming[M]. Belmont, MA: Athena Scientific, 1996.
5. TANG H, Xi H S, YIN B Q. Performance optimization of continuous-time Markov control processes based on performance potentials[J]. Int J of Systems Science, 2003, 34(1): 63-71.
6. CAO X R. Single sample path-based optimization of Markov chains[J]. J of Optimization Theory and Applications, 1999, 100(3): 527-548.
7. CAO X R. From Perturbation analysis to Markov decision processes and reinforcement learning[J]. Discrete Event Dynamic Systems: Theory and Applications, 2003, 13(1/2): 9-39.
8. SUTTON R S, BARTO A G. Reinforcement Learning: An Introduction[M]. Cambridge, MA: MIT Press, 1998.
9. TANG H, YUAN J B, LU Y, et al. Performance potential-based neuro-dynamic programming for SMDPs[J]. Acta Automatic Sinica, 2005, 31(4): 642-645.
10. TANG Hao, XI Hongsheng, YIN Baoqun. On-line optimization algorithm for Markov control processes based on a single sample path[J]. Control Theory and Application, 2002, 19(6): 863-871.
11. CAO X R. Semi-Markov decision problems and performance sensitvity analysis[J]. IEEE Trans on Automatic Control, 2003, 48(5): 758-769.
12. YIN Baoqun, XI Hongsheng, ZHOU Yaping. Queueing System Performance Analysis and Markov Control Processes[M]. Hefei: Press of University of Science and Technology of China, 2004.