Event-based optimization of Markov systems

IEEE Transactions on Automatic Control

Volumn 53, Issue 4, 2008, Pages 1076-1082

  Cao, Xi Ren ^a   Zhang, Junyu ^b  

^a HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

^b SUN YAT SEN UNIVERSITY   (China)

Author keywords

Markov decision processes (MDPs); Performance potentials; Perturbation analysis (PA); Policy gradients; Policy iteration

Indexed keywords

DECISION THEORY; OPTIMIZATION; PERTURBATION TECHNIQUES; SENSITIVITY ANALYSIS;

EVENT-BASED OPTIMIZATION; PERFORMANCE POTENTIALS; POLICY GRADIENTS; POLICY ITERATION;

MARKOV PROCESSES;

EID: 44849134414     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2008.919557     Document Type: Article

References (9)

1. A. Barto and S. Mahadevan, "Recent advances in hierarchical reinforcement learning, special issue on reinforcement learning," Discrete Event Dyn. Syst.: Theory Appl., vol. 13, pp. 41-77, 2003.
2. J. Baxter and P. L. Bartlett, "Infinite-horizon policy-gradient estimation,", J. Artif. Intell. Res., vol. 15, pp. 319-350, 2001.
3. X. R. Cao, "Basic ideas for event-based optimization of Markov systems,: Discrete Event Dyn. Syst.: Theory Appl., vol. 15, pp. 169-197, 2005.
4. X. R. Cao and H. F. Chen, "Perturbation realization, potentials and sensitivity analysis of processes," IEEE Trans. Autom. Control, vol. 42, no. 10, pp. 1382-1393, Oct. 1997.
5. X. R. Cao and J. Y. Zhang, "The nth-order bias optimality for multi-chain Markuv decision processes," IEEE Trans. Autnom. Control vol. 53, no. 2, pp. 496-508, Mar. 2008.
6. E. Çinlar, Introduction to Stochastic Processes. Upper Saddle River, NJ: Prentice-Hall, 1995.
7. W. L. Cooper, S. G. Henderson, and M. E. Lewis, "Convergence of simulation-based policy iteration," Probab. Eng. Inf. Sci., vol. 17, pp. 213 234, 2003.
8. H. T. Fang and X. R. Cao, "Potential-based on-line policy iteration algorithms for Markov decision processes," IEEE Trans. Autom. Control, vol. 49, no. 4, pp. 493-505, Apr. 2004.
9. P. Marbach and T. N. Tsitsiklis, "Simulation-based optimization of Markov reward processes," IEEE Trans. Autom. Control, vol. 46, no. 2. pp. 191-209, Feb. 2001.