Improving Gaussian process value function approximation in policy gradient algorithms

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 6792 LNCS, Issue PART 2, 2011, Pages 221-228

  Jakab, Hunor ^a,b   Csató, Lehel ^a,b  

^a BABE BOLYAI UNIVERSITY   (Romania)

^b EÖTVÖS LORÁND UNIVERSITY   (Hungary)

Author keywords

control problems; Gaussian processes; policy gradient methods; Reinforcement learning; value function estimation

Indexed keywords

BASIS VECTOR; CONTINUOUS DOMAIN; CONTINUOUS STATE-ACTION SPACES; CONTROL PROBLEMS; DISTANCE-BASED; DYNAMIC SYSTEM CONTROL; GAUSSIAN PROCESS REGRESSION; GAUSSIAN PROCESSES; GRADIENT BASED; GRADIENT ESTIMATES; GRADIENT VARIANCE; KULLBACK-LEIBLER DISTANCE; POLICY GRADIENT; POLICY GRADIENT METHODS; POLICY SEARCH; TIME-DEPENDENT FACTORS; VALUE FUNCTION APPROXIMATION; VALUE FUNCTIONS; VALUE-BASED; WILLIAMS;

APPROXIMATION THEORY; CONVERGENCE OF NUMERICAL METHODS; ESTIMATION; GAUSSIAN DISTRIBUTION; GAUSSIAN NOISE (ELECTRONIC); GRADIENT METHODS; LEARNING ALGORITHMS; NEURAL NETWORKS; PROCESS CONTROL; REINFORCEMENT LEARNING;

APPROXIMATION ALGORITHMS;

EID: 79959344344     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-21738-8_29     Document Type: Conference Paper

References (15)

1. Baird, L., Moore, A.: Gradient descent for general reinforcement learning. In: Kearns, M.S., Solla, S.A., Cohn, D.A. (eds.) NIPS 1998. Advances in Neural Information Processing Systems, vol. 11, pp. 968-974. MIT Press, Cambridge (1998)
2. Csató, L.: Gaussian Processes - Iterative Sparse Approximation. PhD thesis, Neural Computing Research Group (2002)
3. Csató, L., Opper, M.: Sparse representation for Gaussian process models. In: Leen, T.K., Dietterich, T.G., Tresp, V. (eds.) NIPS, vol. 13, pp. 444-450. MIT Press, Cambridge (2001)
4. Deisenroth, M.P., Rasmussen, C.E., Peters, J.: Gaussian process dynamic programming. Neurocomputing 72(7-9), 1508-1524 (2009)
5. Engel, Y., Mannor, S., Meir, R.: Reinforcement learning with Gaussian processes. In: Proceedings of the 22nd International Conference on Machine learning, pp. 201-208, New York (2005)
6. Fan, Y., Xu, J., Shelton, C.R.: Importance sampling for continuous time Bayesian networks. Journal of Machine Learning Research 11, 2115-2140 (2010)
7. Ghavamzadeh, M., Engel, Y.: Bayesian policy gradient algorithms. In: Schölkopf, B., Platt, J., Hoffman, T. (eds.) NIPS 2007, Advances in Neural Information Processing Systems, vol. 19, pp. 457-464. MIT Press, Cambridge (2007)
8. Jakab, H.S., Csató, L.: Using Gaussian processes for variance reduction in policy gradient algorithms. In: 8th International Conference on Applied Informatics, Eger, pp. 55-63 (2010)
9. Peters, J., Schaal, S.: Reinforcement learning of motor skills with policy gradients. Neural Networks 21(4), 682-697 (2008)
10. Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, New York (1994)
11. Rasmussen, C.E., Kuss, M.: Gaussian processes in reinforcement learning. In: Saul, L.K., Thrun, S., Schlkopf, B. (eds.) NIPS 2003, Advances in Neural Information Processing Systems, pp. 751-759. MIT Press, Cambridge (2004)
12. Rasmussen, C.E., Williams, C.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)
13. Sugiyama, M., Hachiya, H., Towell, C., Vijayakumar, S.: Geodesic gaussian kernels for value function approximation. Auton. Robots 25, 287-304 (2008)
14. Sutton, R.S., McAllester, D.A., Singh, S.P., Mansour, Y.: Policy gradient methods for reinforcement learning with function approximation. In: Solla, S.A., Leen, T.K., Müller, K.R. (eds.) NIPS 1999, Advances in Neural Information Processing Systems, pp. 1057-1063. MIT Press, Cambridge (1999)
15. Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning 8, 229-256 (1992)