Action-gap phenomenon in reinforcement learning

Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011

Volumn , Issue , 2011, Pages

  Farahmand, Amir Massoud ^a  

^a MCGILL UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ITERATIVE METHODS;

APPROXIMATE VALUE ITERATION ALGORITHM; GAP PHENOMENON; GREEDY POLICY; LEARNING PROBLEM; OPTIMAL PERFORMANCE; PERFORMANCE; PERFORMANCE LOSS; REINFORCEMENT LEARNINGS; VALUE FUNCTIONS;

REINFORCEMENT LEARNING;

EID: 85162479771     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. Michail G. Lagoudakis and Ronald Parr. Least-squares policy iteration. Journal of Machine Learning Research, 4:1107-1149, 2003.
2. Dimitri P. Bertsekas and John N. Tsitsiklis. Neuro-Dynamic Programming (Optimization and Neural Computation Series, 3). Athena Scientific, 1996.
3. Enno Mammen and Alexander B. Tsybakov. Smooth discrimination analysis. The Annals of Statistics, 27(6):1808-1829, 1999.
4. Alexander B. Tsybakov. Optimal aggregation of classifiers in statistical learning. The Annals of Statistics, 32 (1):135-166, 2004.
5. Jean-Yves Audibert and Alexander B. Tsybakov. Fast learning rates for plug-in classifiers. The Annals of Statistics, 35(2):608-633, 2007.
6. Alessandro Rinaldo and Larry Wasserman. Generalized density clustering. The Annals of Statistics, 38(5):2678-2722, 2010.
7. Michail G. Lagoudakis and Ronald Parr. Reinforcement learning as classification: Leveraging modern classifiers. In ICML '03: Proceedings of the 20th international conference on Machine learning, pages 424-431, 2003.
8. Alessandro Lazaric, Mohammad Ghavamzadeh, and Rémi Munos. Analysis of a classificationbased policy iteration algorithm. In Proceedings of the 27th International Conference on Machine Learning (ICML-10), pages 607-614. Omnipress, 2010.
9. Omar Syed and Robert E. Schapire. A reduction from apprenticeship learning to classification. In J. Lafferty, C. K. I.Williams, J. Shawe-Taylor, R.S. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems (NIPS - 23), pages 2253-2261, 2010.
10. András Antos, Csaba Szepesvári, and Rémi Munos. Learning near-optimal policies with Bellman-residual minimization based fitted policy iteration and a single sample path. Machine Learning, 71:89-129, 2008.
11. Rémi Munos and Csaba Szepesvári. Finite-time bounds for fitted value iteration. Journal of Machine Learning Research, 9:815-857, 2008.
12. Amir-massoud Farahmand, Mohammad Ghavamzadeh, Csaba Szepesvári, and Shie Mannor. Regularized fitted Q-iteration for planning in continuous-space Markovian Decision Problems. In Proceedings of American Control Conference (ACC), pages 725-730, June 2009.
13. Amir-massoud Farahmand, Mohammad Ghavamzadeh, Csaba Szepesvári, and Shie Mannor. Regularized policy iteration. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, editors, Advances in Neural Information Processing Systems (NIPS - 21), pages 441-448. MIT Press, 2009.
14. Odalric Maillard, Rémi Munos, Alessandro Lazaric, and Mohammad Ghavamzadeh. Finitesample analysis of Bellman residual minimization. In Proceedings of the Second Asian Conference on Machine Learning (ACML), 2010.
15. Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning). The MIT Press, 1998.
16. Csaba Szepesvári. Algorithms for Reinforcement Learning. Morgan Claypool Publishers, 2010.
17. Hamid Reza Maei, Csaba Szepesvári, Shalabh Bhatnagar, and Richard S. Sutton. Toward off-policy learning control with function approximation. In Johannes Fürnkranz and Thorsten Joachims, editors, Proceedings of the 27th International Conference on Machine Learning (ICML-10), pages 719-726, Haifa, Israel, June 2010. Omnipress.
18. J. Zico Kolter and Andrew Y. Ng. Regularization and feature selection in least-squares temporal difference learning. In ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning, pages 521-528. ACM, 2009.
19. Martin Riedmiller. Neural fitted Q iteration - first experiences with a data efficient neural reinforcement learning method. In 16th European Conference on Machine Learning, pages 317-328, 2005.
20. Damien Ernst, Pierre Geurts, and Louis Wehenkel. Tree-based batch mode reinforcement learning. Journal of Machine Learning Research, 6:503-556, 2005.
21. Daniela Pucci de Farias and Benjamin Van Roy. The linear programming approach to approximate dynamic programming. Operations Research, 51(6):850-865, 2003.
22. Marek Petrik and Shlomo Zilberstein. Constraint relaxation in approximate linear programs. In Proceedings of the 26th Annual International Conference on Machine Learning, ICML '09, pages 809-816, New York, NY, USA, 2009. ACM.
23. Rémi Munos. Error bounds for approximate policy iteration. In ICML 2003: Proceedings of the 20th Annual International Conference on Machine Learning, pages 560-567, 2003.
24. p norm for approximate value iteration. SIAM Journal on Control and Optimization, pages 541-561, 2007.
25. Amir-massoud Farahmand, Rémi Munos, and Csaba Szepesvári. Error propagation for approximate policy and value iteration. In J. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R.S. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems (NIPS - 23), pages 568-576. 2010.