Dynamic policy programming with function approximation

Journal of Machine Learning Research

Volumn 15, Issue , 2011, Pages 119-127

  Azar, Mohammad Gheshlaghi ^a   Gómez, Vicenç ^a   Kappen, Hilbert J ^a  

^a RADBOUD UNIVERSITY NIJMEGEN   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATE DYNAMIC PROGRAMMING; ASYMPTOTIC BOUNDS; DYNAMIC POLICY; FUNCTION APPROXIMATION; FUNCTION APPROXIMATION TECHNIQUES; INFINITE HORIZONS; LOSS BOUNDS; MARKOV DECISION PROBLEM; OPTIMAL POLICIES;

ARTIFICIAL INTELLIGENCE; REINFORCEMENT LEARNING;

DYNAMIC PROGRAMMING;

EID: 84862300689     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (22)

1. Antos, A., Munos, R., and Szepesvári, C. (2008). Fitted q-iteration in continuous action-space mdps. In Proceedings of the 21st Annual Conference on Neural Information Processing Systems.
2. Bartlett, P. L. (2003). An introduction to reinforcement learning theory: Value function methods. Lecture Notes in Artificial Intelligence, 2600/2003:184-202.
3. Baxter, J. and Bartlett, P. L. (2001). Infinite-horizon policy-gradient estimation. Journal of Artificial Intelligence Research, 15:319-350.
4. Bertsekas, D. P. (2007). Dynamic Programming and Optimal Control, volume II. Athena Scientific, Belmount, Massachusetts, third edition.
5. Bertsekas, D. P. and Tsitsiklis, J. N. (1996). Neuro- Dynamic Programming. Athena Scientific, Belmont, Massachusetts.
6. Bhatnagar, S., Sutton, R. S., Ghavamzadeh, M., and Lee, M. (2009). Natural actor-critic algorithms. Automatica, 45(11):2471-2482.
7. Boyan, J. A. and Moore, A. W. (1995). Generalization in reinforcement learning: Safely approximating the value function. In Advances in Neural Information Processing Systems, pages 369-376.
8. Farias, D. P. and Roy, B. V. (2000). On the existence of fixed points for approximate value iteration and temporal-difference learning. Journal of Optimization Theory and Applications, 105(3):589-608.
9. Kakade, S. (2001). Natural policy gradient. In Advances in Neural Information Processing Systems 14, pages 1531-1538, Vancouver, British Columbia, Canada.
10. Kappen, H. J. (2005). Path integrals and symmetry breaking for optimal control theory. Statistical Mechanics, 2005(11):P11011.
11. MacKay, D. J. C. (2003). Information Theory, Inference, and Learning Algorithms. Cambridge University Press, Cambridge, United Kingdom, first edition.
12. Munos, R. (2005). Error bounds for approximate value iteration. In Proceedings of the 20th National Conference on Artificial Intelligence, volume II, pages 1006-1011, Pittsburgh, Pennsylvania.
13. Munos, R. and Szepesvári, C. (2008). Finite-time bounds for fitted value iteration. Journal of Machine Learning Research, 9:815-857.
14. Peters, J., Mülling, K., and Altun, Y. (2010). Relative entropy policy search. In Proceedings of the Twenty- Fourth AAAI Conference on Artificial Intelligence, Atlanta, Georgia, USA.
15. Peters, J. and Schaal, S. (2008). Natural actor-critic. Neurocomputing, 71(7-9):1180-1190.
16. Sutton, R. S. (1996). Generalization in reinforcement learning: succesful examples using sparse coarse coding. In Advances in Neural Information Processing Systems 9, pages 1038-1044.
17. Sutton, R. S. and Barto, A. G. (1998). Reinforcement Learning: An Introduction. MIT Press, Cambridge, Massachusetts.
18. Sutton, R. S., McAllester, D., Singh, S., and Mansour, Y. (1999). Policy gradient methods for reinforcement learning with function approximation. In Advances in Neural Information Processing Systems 12, pages 1057-1063, Denver, Colorado, USA.
19. Szepesvari, C. (2009). Reinforcement learning algorithms for mdps - a survey. Technical Report TR09-13, Department of Computing Science, University of Alberta, Edmonton, Alberta, Canada.
20. Thiery, C. and Scherrer, B. (2010). Least-squares policy iteration: Bias- variance trade-off in control problems. In Proceedings of the 27th Annual International Conference on Machine Learning.
21. Todorov, E. (2006). Linearly-solvable markov decision problems. In Proceedings of the 20th Annual Conference on Neural Information Processing Systems, pages 1369-1376, Vancouver, British Columbia, Canada.
22. Wang, T., Lizotte, D., Bowling, M., and Schuurmans, D. (2007). Stable dual dynamic programming. In Proceedings of the 21st Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada.