Recursive learning automata approach to Markov decision processes

IEEE Transactions on Automatic Control

Volumn 52, Issue 7, 2007, Pages 1349-1355

  Chang, Hyeong Soo ^a   Fu, Michael C ^b   Hu, Jiaqiao ^c   Marcus, Steven I ^d  

^a Sogang University   (South Korea)

^b UNIVERSITY OF MARYLAND   (United States)

^c Stony Brook University   (United States)

^d UNIVERSITY OF MARYLAND   (United States)

Author keywords

Learning automata; Markov decision process (MDP); Sampling

Indexed keywords

FINITE ELEMENT METHOD; MARKOV PROCESSES; OPTIMIZATION; RANDOM PROCESSES; RECURSIVE FUNCTIONS;

FINITE-TIME ANALYSIS; MARKOV DECISION PROCESSES; RECURSIVE AUTOMATA SAMPLING ALGORITHM (RASA);

LEARNING ALGORITHMS;

EID: 34547108579     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2007.900859     Document Type: Article

References (18)

1. T. P. I. Ahamed, P. S. N. Rao, and P. S. Sastry, "A reinforcement learning approach to automatic generation control," Electric Power Syst. Res., vol. 63, pp. 9-26, 2002.
2. J. Baxter and P. L. Bartlett, "Infinite-horizon policy-gradient estimation," J. Artif. Intell. Res., vol. 15, pp. 319-350, 2001.
3. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, 1996.
4. H. S. Chang, M. C. Fu, J. Hu, and S. I. Marcus, "An adaptive sampling algorithm for solving Markov decision processes," Operat. Res., vol. 53, no. 1, pp. 126-139, 2005.
5. H. S. Chang, M. C. Fu, J. Hu, and S. I. Marcus, Simulation-Based Algorithms for Markov Decision Processes. London, U.K.: Springer-Verlag, 2007.
6. H. S. Chang, M. C. Fu, J. Hu, and S. I. Marcus, "An asymptotically efficient simulation-based algorithm for finite horizon stochastic dynamic programming," IEEE Trans. Autom. Control, vol. 52, no. 1, pp. 89-94, Jan. 2007.
7. O. Hernácndez-Lerma, Adaptive Markov Control Processes. New York: Springer-Verlag, 1989.
8. R. Jain and P. Varaiya, "Simulation-based uniform value function estimates of Markov decision processes," SIAM J. Control Optim., vol. 45, no. 5, pp. 1633-1656, 2006.
9. A. S. Poznyak, K. Najim, and E. Gomez-Ramirez, Self-Learning Control of Finite Markov Chains. New York: Marcel Dekker, 2000.
10. M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming. New York: Wiley, 1994.
11. K. Rajaraman and P. S. Sastry, "Finite time analysis of the pursuit algorithm for learning automata," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 26, no. 4, pp. 590-598, Aug. 1996.
12. G. Santharam and P. S. Sastry, "A reinforcement learning neural network for adaptive control of Markov chains," IEEE Trans. Syst, Man, Cybern. A, Syst. Humans, vol. 27, no. 5, pp. 588-600, Sep. 1997.
13. P. S. Sastry, "Systems of learning automata-estimator algorithms and applications," Ph.D. dissertation, Dept. Electr. Eng., Indian Inst. Sci., Bangalore, India, Jun. 1985.
14. P. S. Sastry and M. A. L. Thathachar, Networks of Learning Automata: Techniques for Online Stochastic Optimization. New York: Springer-Verlag, 2003.
15. A. N. Shiryaev, Probability, 2nd ed. New York: Springer-Verlag, 1995.
16. R. Sutton and A. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998.
17. M. A. L. Thathachar and P. S. Sastry, "Varieties of learning automata: An overview," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 32, no. 6, pp. 711-722, Dec. 2002.
18. R. M. Wheeler Jr. and K. S. Narendra, "Decentralized learning in finite Markov chains," IEEE Trans. Autom. Control, vol. AC-31, no. 6, pp. 519-526, Jun. 1986.