The fixed points of off-policy TD

Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011

Volumn , Issue , 2011, Pages

  Kolter, J Zico ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION ALGORITHMS; REINFORCEMENT LEARNING;

APPROXIMATORS; FIXED POINTS; FUNCTIONS APPROXIMATIONS; KEY ELEMENTS; LEARN+; POLICY DISTRIBUTION; POLICY LEARNING; REINFORCEMENT LEARNINGS; SOLUTION QUALITY; VALUE FUNCTIONS;

SET THEORY;

EID: 85162349973     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (19)

1. L. C. Baird. Residual algorithms: Reinforcement learning with function approximation. In Proceedings of the International Conference on Machine Learning, 1995.
2. S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
3. P. Dayan. The convergence of TD(λ) for general λ. Machine Learning, 8(3-4), 1992.
4. T. Jaakkola, M. I. Jordan, and S. P. Singh. On the convergence of stochastic iterative dynamic programming algorithms. Neural Computation, 6(6), 1994.
5. M. Journee, F. Bach, P.A. Absil, and R. Sepulchre. Low-rank optimization on the cone of positive semidefinite matrices. SIAM Journal on Optimization, 20(5):2327-2351, 2010.
6. J.Z. Kolter and A.Y. Ng. Regularization and feature selection in least-squares temporal difference learning. In Proceedings of the International Conference on Machine Learning, 2009.
7. M. G. Lagoudakis and R. Parr. Least-squares policy iteration. Journal of Machine Learning Research, 4:1107-1149, 2003.
8. A. Lazaric, M. Ghavamzadeh, and R.Munos. Finite-sample analysis of LSTD. In Proceedings of the International Conference on Machine Learning, 2010.
9. H.R. Maei and R.S. Sutton. GQ(λ): A general gradient algorithm for temporal-difference prediction learning with eligibility traces. In Proceedings of the Third Conference on Artificial General Intelligence, 2010.
10. H.R. Maei, Cs. Szepesvari, S. Bhatnagar, and R.S. Sutton. Toward off-policy learning control with function approximation. In Proceedings of the International Conference on Machine Learning, 2010.
11. R. Munos. Error bounds for approximate policy iteration. In Proceedings of the International Conference on Machine Learning, 2003.
12. J. Nocedal and S.J. Wright. Numerical Optimization. Springer, 1999.
13. B. Scherrer. Personal communication, 2011.
14. M. Schmidt. minfunc, 2005. Available at http://www.cs.ubc.ca/~schmidtm/ Software/minFunc.html.
15. R.S. Sutton, H.R. Maei, D. Precup, S. Bhatnagar, D. Silver, Cs. Szepesvari, and E. Wiewiora. Fast gradient-descent methods for temporal-difference learning with linear function approximation. In Proceedings of the International Conference on Machine Learning, 2009.
16. R.S. Sutton, Cs. Szepesvari, and H.R. Maei. A convergent O(n) algorithm for off-policy temporal-different learning with linear function approximation. In Advances in Neural Information Processing, 2008.
17. J.N. Tsitsiklis and B. Van Roy. An analysis of temporal-difference learning with function approximation. IEEE Transactions and Auotomatic Control, 42:674-690, 1997.
18. J.N. Tsitsiklis and B. Van Roy. Average cost temporal difference learning. Automatica, 35(11):1799-1808, 1999.
19. H. Yu and D. P. Bertsekas. Error bounds for approximations from projected linear equations. Mathematics of Operations Research, 35:306-329, 2010.