A unifying perspective of parametric policy search methods for Markov Decision Processes

Advances in Neural Information Processing Systems

Volumn 4, Issue , 2012, Pages 2717-2725

  Furmston, Thomas ^a   Barber, David ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

EXPECTATION-MAXIMISATION; MARKOV DECISION PROCESSES; NATURAL GRADIENT; NEWTON'S METHODS; OPTIMISATION METHOD; OPTIMISATIONS; PARAMETER SPACES; ROBUSTNESS PROPERTIES;

MARKOV PROCESSES; NEWTON-RAPHSON METHOD; OPTIMIZATION;

ALGORITHMS;

EID: 84877731836     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (31)

1. S. Amari. Natural Gradient Works Efficiently in Learning. Neural Computation, 10:251-276, 1998.
2. M. Azar, V. Gómez, and H. Kappen. Dynamic policy programming with function approximation. Journal of Machine Learning Research - Proceedings Track, 15:119-127, 2011.
3. J. Bagnell and J. Schneider. Covariant Policy Search. IJCAI, 18:1019-1024, 2003.
4. J. Baxter and P. Bartlett. Infinite Horizon Policy Gradient Estimation. Journal of Artificial Intelligence Research, 15:319-350, 2001.
5. D. P. Bertsekas. Dynamic Programming and Optimal Control. Athena Scientific, second edition, 2000.
6. D. P. Bertsekas. Approximate Policy Iteration: A Survey and Some New Methods. Research report, Massachusetts Institute of Technology, 2010.
7. D. P. Bertsekas and S. Ioffe. Temporal Differences-Based Policy Iteration and Applications in Neuro- Dynamic Programming. Research report, Massachusetts Institute of Technology, 1997.
8. S. Bhatnagar, R. Sutton, M. Ghavamzadeh, and L. Mark. Natural Actor-Critic Algorithms. Automatica, 45:2471-2482, 2009.
9. S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
10. P. Dayan and G. E. Hinton. Using Expectation-Maximization for Reinforcement Learning. Neural Computation, 9:271-278, 1997.
11. A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society. Series B (Methodological), 39(1):1-38, 1977.
12. C. Fahey. Tetris AI, Computers Play Tetris http://colinfahey.com/tetris/ tetris-en.html, 2003.
13. T. Furmston and D. Barber. Efficient Inference for Markov Control Problems. UAI, 29:221-229, 2011.
14. P.W. Glynn. Likelihood Ratio Gradient Estimation for Stochastic Systems. Communications of the ACM, 33:97-84, 1990.
15. E. Greensmith, P. Bartlett, and J. Baxter. Variance Reduction Techniques For Gradient Based Estimates in Reinforcement Learning. Journal of Machine Learning Research, 5:1471-1530, 2004.
16. S. Kakade. A Natural Policy Gradient. NIPS, 14:1531-1538, 2002.
17. H. Khalil. Nonlinear Systems. Prentice Hall, 2001.
18. J. Kober and J. Peters. Policy Search for Motor Primitives in Robotics. Machine Learning, 84(1-2):171-203, 2011.
19. L. Kocsis and C. Szepesvári. Bandit Based Monte-Carlo Planning. European Conference on Machine Learning (ECML), 17:282-293, 2006.
20. V. R. Konda and J. N. Tsitsiklis. On Actor-Critic Algorithms. SIAM J. Control Optim., 42(4):1143-1166, 2003.
21. P. Marbach and J. Tsitsiklis. Simulation-Based Optimisation of Markov Reward Processes. IEEE Transactions on Automatic Control, 46(2):191-209, 2001.
22. N. Meuleau, L. Peshkin, K. Kim, and L. Kaelbling. Learning Finite-State Controllers for Partially Observable Environments. UAI, 15:427-436, 1999.
23. J. Nocedal and S. Wright. Numerical Optimisation. Springer, 2006.
24. J. Peters and S. Schaal. Natural Actor-Critic. Neurocomputing, 71(7-9):1180-1190, 2008.
25. K. Rawlik, Toussaint. M, and S. Vijayakumar. On Stochastic Optimal Control and Reinforcement Learning by Approximate Inference. International Conference on Robotics Science and Systems, 2012.
26. S. Richter, D. Aberdeen, and J. Yu. Natural Actor-Critic for Road Traffic Optimisation. NIPS, 19:1169-1176, 2007.
27. R. Sutton, D. McAllester, S. Singh, and Y. Mansour. Policy Gradient Methods for Reinforcement Learning with Function Approximation. NIPS, 13:1057-1063, 2000.
28. M. Toussaint, S. Harmeling, and A. Storkey. Probabilistic Inference for Solving (PO)MDPs. Research Report EDI-INF-RR-0934, University of Edinburgh, School of Informatics, 2006.
29. N. Vlassis, M. Toussaint, G. Kontes, and S. Piperidis. Learning Model-Free Robot Control by a Monte Carlo EM Algorithm. Autonomous Robots, 27(2):123-130, 2009.
30. L. Weaver and N. Tao. The Optimal Reward Baseline for Gradient Based Reinforcement Learning. UAI, 17(29):538-545, 2001.
31. R. Williams. Simple Statistical Gradient Following Algorithms for Connectionist Reinforcement Learning. Machine Learning, 8:229-256, 1992.