Policy gradient in continuous time published

Journal of Machine Learning Research

Volumn 7, Issue , 2006, Pages 771-791

  Munos, Rémi ^a  

^a CENTRE DE MATHÉMATIQUES APPLIQUÉES   (France)

Author keywords

Gradient estimate; Likelihood ratio method; Optimization; Pathwise derivation

Indexed keywords

APPROXIMATION THEORY; DECISION MAKING; OPTIMIZATION; PARAMETER ESTIMATION; PROBLEM SOLVING; SEARCH ENGINES;

GRADIENT ESTIMATE; LIKELIHOOD RATIO METHOD; PATHWISE DERIVATION; REINFORCEMENT LEARNING;

OPTIMAL CONTROL SYSTEMS;

EID: 33646399442     PISSN: 15337928     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (19)

1. J. Baxter and P. L. Bartlett. Infinite-horizon gradient-based policy search. Journal of Artificial Intelligence Research, 15:319-350, 2001.
2. A. Bensoussan. Perturbation methods in optimal control. Wiley/Gauthier-Villars Series in Modern Applied Mathematics. John Wiley & Sons Ltd., Chichester, 1988. Translated from the French by C. Tomson.
3. A. Bogdanov. Optimal control of a double inverted pendulum on a cart. Technical report CSE-04-006, CSEE, OGI School of Science and Engineering, OHSU, 2004.
4. P. W. Glynn. Likelihood ratio gradient estimation: an overview. In A. Thesen, H. Grant, and W. D. Kelton, editors, Proceedings of the 1987 Winter Simulation Conference, pages 366-375, 1987.
5. E. Gobet and R. Munos. Sensitivity analysis using Itô-Malliavin calculus and martingales, application to stochastic optimal control. SIAM journal on Control and Optimization, 43(5): 1676-1713, 2005.
6. G. H. Golub and C. F. Van Loan. Matrix Computations, 3rd ed. Baltimore, MD: Johns Hopkins, 1996.
7. R. E. Kalman, P. L. Falb, and M. A. Arbib. Topics in Mathematical System Theory. New York: McGraw Hill, 1969.
8. P. E. Kloeden and E. Platen. Numerical Solutions of Stochastic Differential Equations. Springer-Verlag, 1995.
9. H. J. Kushner and G. Yin. Stochastic Approximation Algorithms and Applications. Springer-Verlag, Berlin and New York, 1997.
10. S. M. LaValle. Planning Algorithms. Cambridge University Press, 2006.
11. M. Ledoux. The concentration of measure phenomenon. American Mathematical Society, Providence, RI, 2001.
12. P. Marbach and J. N. Tsitsiklis. Approximate gradient methods in policy-space optimization of Markov reward processes. Journal of Discrete Event Dynamical Systems, 13:111-148, 2003.
13. B. T. Polyak. Introduction to Optimization. Optimization Software Inc., New York, 1987.
14. M. I. Reiman and A. Weiss. Sensitivity analysis via likelihood ratios. In J. Wilson, J. Henriksen, and S. Roberts, editors, Proceedings of the 1986 Winter Simulation Conference, pages 285-289, 1986.
15. R. S. Sutton and A. G. Barto. Reinforcement learning: An introduction. Bradford Book, 1998.
16. R. S. Sutton, D. McAllester, S. Singh, and Y. Mansour. Policy gradient methods for reinforcement learning with function approximation. Neural Information Processing Systems. MIT Press, pages 1057-1063, 2000.
17. M. Talagrand. A new look at independence. Annals of Probability, 24:1-34, 1996,
18. R. J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8:229-256, 1992.
19. J. Yang and H. J. Kushner. A Monte Carlo method for sensitivity analysis and parametric optimization of nonlinear stochastic systems. SIAM J. Control Optim., 29(5): 1216-1249, 1991.