Policy search with cross-entropy optimization of basis functions

2009 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL 2009 - Proceedings

Volumn , Issue , 2009, Pages 153-160

  Buşoniu, Lucian ^a   Ernst, Damien ^c   De Schutter, Bart ^a,b   Babuska, Robert ^a  

^a DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

^b DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

^c UNIVERSITY OF LIÈGE   (Belgium)

Author keywords

[No Author keywords available]

Indexed keywords

BASIS FUNCTIONS; CLOSED-LOOP; COMPUTATIONAL COSTS; CROSS ENTROPY; CROSS-ENTROPY METHOD; INITIAL STATE; LARGE CLASS; MARKOV DECISION PROCESSES; NOVEL ALGORITHM; PARAMETERIZATIONS; POLICY SEARCH; SIMULATION EXPERIMENTS;

ALGORITHMS; KNOWLEDGE BASED SYSTEMS; MARKOV PROCESSES; OPTIMIZATION; PARAMETERIZATION; REINFORCEMENT; REINFORCEMENT LEARNING; SYSTEMS ENGINEERING; UNMANNED AERIAL VEHICLES (UAV);

DYNAMIC PROGRAMMING;

EID: 67650502101     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ADPRL.2009.4927539     Document Type: Conference Paper

References (17)

1. D.P. Bertsekas, Dynamic Programming and Optimal Control, 3rd ed. Athena Scientific, 2007, Vol. 2.
2. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. MIT Press, 1998.
3. R. Munos and A. Moore, "Variable-resolution discretization in optimal control," Machine Learning, Vol. 49, no. 2-3, pp. 291-323, 2002.
4. M. G. Lagoudakis and R. Parr, "Least-squares policy iteration," Journal of Machine Learning Research, Vol. 4, pp. 1107-1149, 2003.
5. D. Ernst, P. Geurts, and L. Wehenkel, "Tree-based batch mode reinforcement learning," Journal of Machine Learning Research, Vol. 6, pp. 503-556, 2005. (Pubitemid 40958851)
6. S. Mahadevan and M. Maggioni, "Proto-value functions: A Laplacian framework for learning representation and control in Markov decision processes," Journal of Machine Learning Research, Vol. 8, pp. 2169-2231, 2007. (Pubitemid 350046199)
7. R. S. Sutton, D. A. McAllester, S. P. Singh, and Y. Mansour, "Policy gradient methods for reinforcement learning with function approximation," in Advances in Neural Information Processing Systems 12, S. A. Solla, T. K. Leen, and K.-R. Müller, Eds. MIT Press, 2000, pp. 1057-1063.
8. P. Marbach and J. N. Tsitsiklis, "Approximate gradient methods in policy-space optimization of Markov reward processes," Discrete Event Dynamic Systems: Theory and Applications, Vol. 13, pp. 111-148, 2003.
9. R. Munos, "Policy gradient in continuous time," Journal of Machine Learning Research, Vol. 7, pp. 771-791, 2006.
10. S. Mannor, R. Y. Rubinstein, and Y. Gat, "The cross-entropy method for fast policy search," in Proceedings 20th International Conference on Machine Learning (ICML-03), Washington, US, 21-24 August 2003, pp. 512-519.
11. H. S. Chang, M. C. Fu, J. Hu, and S. I. Marcus, Simulation-Based Algorithms for Markov Decision Processes. Springer, 2007.
12. S. Whiteson and P. Stone, "Evolutionary function approximation for reinforcement learning," Journal of Machine Learning Research, Vol. 7, pp. 877-917, 2006. (Pubitemid 43736560)
13. R. Y. Rubinstein and D. P. Kroese, The Cross Entropy Method. A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation, and Machine Learning, ser. Information Science and Statistics, M. Jordan, J. Kleinberg, B. Scholkopf, F. Kelly, and I. Witten, Eds. Springer, 2004.
14. V. R. Konda and J. N. Tsitsiklis, "On actor-critic algorithms," SIAM Journal on Control and Optimization, Vol. 42, no. 4, pp. 1143-1166, 2003.
15. A. Costa, O. D. Jones, and D. Kroese, "Convergence properties of the cross-entropy method for discrete optimization," Operations Research Letters, Vol. 35, pp. 573-580, 2007. (Pubitemid 47198343)
16. L. Buşoniu, D. Ernst, B. De Schutter, and R. Babuška, "Continuousstate reinforcement learning with fuzzy approximation," in Adaptive Agents and Multi-Agent Systems III, ser. Lecture Notes in Computer Science, K. Tuyls, A. Nowé, Z. Guessoum, and D. Kudenko, Eds. Springer, 2008, Vol. 4865, pp. 27-43.
17. J. Randløv and P. Alstrøm, "Learning to drive a bicycle using reinforcement learning and shaping," in Proceedings 15th International Conference on Machine Learning (ICML-98), Madison, US, 24-27 July 1998, pp. 463-471.