Evolutionary policy iteration for solving Markov decision processes

IEEE Transactions on Automatic Control

Volumn 50, Issue 11, 2005, Pages 1804-1808

  Chang, Hyeong Soo ^a   Lee, Hong Gi ^b   Fu, Michael C ^c   Marcus, Steven I ^d  

^a Sogang University   (South Korea)

^b Chung Ang University   (South Korea)

^c UNIVERSITY OF MARYLAND   (United States)

^d UNIVERSITY OF MARYLAND   (United States)

Author keywords

(Distributed) policy iteration; Evolutionary algorithm; Genetic algorithm; Markov decision process; Parallelization

Indexed keywords

CONVERGENCE OF NUMERICAL METHODS; EVOLUTIONARY ALGORITHMS; GENETIC ALGORITHMS; ITERATIVE METHODS; MARKOV PROCESSES; OPTIMIZATION; PROBABILITY;

EVOLUTIONARY POLICY ITERATION; MARKOV DECISION PROCESS; PARALLELIZATION;

DECISION THEORY;

EID: 28644446278     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2005.858644     Document Type: Article

References (13)

1. D.P. Bertsekas and J. N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods. Englewood Cliffs, NJ: Prentice-Hall, 1989.
2. H. S. Chang, R. Givan, and E. K. P. Chong, "Parallel rollout for on-line solution of partially observable Markov decision processes,": Discrete Event Dyna. Syst.: Theory Appl., vol. 15, no. 3, pp. 309-341, 2004.
3. H. Chin and A. Jafari, "Genetic algorithm methods for solving the best stationary policy of finite Markov decision processes," in Proc. 30th Southeastern Symp. System Theory, 1998, pp. 538-543.
4. K. A. De Jong, "An analysis of the behavior of a class of genetic adaptive systems," Ph.D. dissertation, Univ. Michigan, Ann Arbor, MI, 1975.
5. R. Givan, S. Leach. and T. Dean, "Bounded Markov decision processes," Artif. Intell., vol. 122, pp. 71-109, 2000.
6. D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning. Reading, MA: Addison-Wesley, 1989.
7. A. Jalali and M. J. Ferguson, "On distributed dynamic programming," IEEE Trans. Autom. Control, vol. 37, no. 5, pp. 685-689, May 1992.
8. A. Z.-Z. Lin, J. Bean, and C. White, III, "A hybrid genetic/ optimization algorithm for finite horizon partially observed Markov decision processes" Dept. Ind. Oper. Eng., Univ. Michigan, Ann Arbor, MI, Tech. Rep. 98-25, 1998.
9. M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming. New York: Wiley, 1994.
10. C. R. Reeves, "Genetic algorithms for the operations researcher," INFORMS J. Comput., vol. 9, no. 3, pp. 231-250, 1997.
11. M. Srinivas and L. M. Patnaik, "Genetic algorithms: A survey," IEEE Comput., vol. 27, no. 6, pp. 17-26, 1994.
12. R. J. Williams and L. C. Baird, "Analysis of some incremental variants of policy iteration: First steps toward understanding actor-critic learning systems,", Tech. Rep. NU-CCS-93-11, 1993.
13. R. J. Williams and L. C. Baird, "A mathematical analysis of actor-critic architectures for learning optimal controls through incremental dynamic programming," in Proc. 6th Yale Workshop on Adaptive and Learning Systems. New Haven, CT, Aug. 15-17, 1990.