Safe policy iteration

30th International Conference on Machine Learning, ICML 2013

Volumn , Issue PART 2, 2013, Pages 1344-1352

  Pirotta, Matteo ^a   Restelli, Marcello ^a   Pecorino, Alessio ^a   Calandriello, Daniele ^a  

^a POLITECNICO DI MILANO   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; LEARNING SYSTEMS;

CARD GAMES; GREEDY POLICY; LOWER BOUNDS; POLICY EVALUATION; POLICY ITERATION; STATE-OF-THE-ART APPROACH;

ITERATIVE METHODS;

EID: 84897496610     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (18)

1. Azar, M. Gheshlaghi, Gómez, V., and Kappen, H. J. Dynamic policy programming. Journal of Machine Learning Research, 13(Nov):3207-3245, 2012.
2. Bertsekas, D.P. Approximate policy iteration: A survey and some new methods. Journal of Control Theory and Applications, 9(3):310-335, 2011.
3. Dutech, A., Edmunds, T., Kok, J., Lagoudakis, M., Littman, M., Riedmiller, M., Russell, B., Scherrer, B., Sutton, R., Timmer, S., et al. Reinforcement learning benchmarks and bake-offs ii. In Workshop at advances in neural information processing systems conference. Citeseer, 2005.
4. Gabillon, V., Lazaric, A., Ghavamzadeh, M., and Scherrer, B. Classication-based policy iteration with a critic. In Proceedings of ICML, pp. 1049-1056, 2011.
5. Haviv, Moshe and Heyden, Ludo Van Der. Perturbation bounds for the stationary probabilities of a finite markov chain. Advances in Applied Probability, 16(4):pp. 804-818, 1984. ISSN 00018678. URL http://www.jstor.org/ stable/1427341.
6. Howard, R.A. Dynamic programming and Markov processes. 1960.
7. Kakade, S.M. A natural policy gradient. NIPS, 14: 1531-1538, 2001.
8. Kakade, S.M. On the sample complexity of reinforcement learning. PhD thesis, PhD thesis, University College London, 2003.
9. Kakade, S.M. and Langford, J. Approximately optimal approximate reinforcement learning. In Proceedings of ICML, pp. 267-274, 2002.
10. Koller, Daphne and Parr, Ronald. Policy Iteration for Factored MDPs. In Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, pp. 326-334, San Francisco, CA, USA, 2000. Morgan Kaufmann Publishers Inc. ISBN 1-55860-709-9.
11. Lagoudakis, M.G. and Parr, R. Least-squares policy iteration. Journal of Machine Learning Research, 4: 1107-1149, 2003.
12. Lazaric, A., Ghavamzadeh, M., and Munos, R. Analysis of a classication-based policy iteration algorithm. In Proceedings of ICML, pp. 607-614, 2010.
13. Munos, R. Error bounds for approximate value iteration. In Proceedings of AAAI, volume 20, pp. 1006, 2005.
14. Perkins, T.J. and Precup, D. A convergent form of approximate policy iteration. NIPS, 15:1595-1602, 2002.
15. Peters, J., Vijayakumar, S., and Schaal, S. Natural actor-critic. In Proceedings of ECML, volume 3720, pp. 280-291. Springer, 2005.
16. Sutton, R.S., McAllester, D., Singh, S., and Mansour, Y. Policy gradient methods for reinforcement learning with function approximation. In NIPS, volume 12, pp. 1057-1063. MIT Press, 2000.
17. Wagner, P. A reinterpretation of the policy oscillation phenomenon in approximate policy iteration. In NIPS, 2011.
18. Ye, Y. The simplex and policy-iteration methods are strongly polynomial for the markov decision problem with a fixed discount rate. Mathematics of Operations Research, 36(4):593-603, 2011.