LSTD with random projections

Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010

Volumn , Issue , 2010, Pages

  Ghavamzadeh, Mohammad ^a   Lazaric, Alessandro ^a   Maillard, Odalric Ambrym ^a   Munos, Rémi ^a  

^a INRIA   (France)

Author keywords

[No Author keywords available]

Indexed keywords

ITERATIVE METHODS; REINFORCEMENT LEARNING;

HIGH DIMENSIONAL SPACES; LEAST SQUARES POLICY ITERATIONS; LEAST-SQUARES TEMPORAL DIFFERENCES; NUMBER OF SAMPLES; PERFORMANCE BOUNDS; POLICY ITERATION ALGORITHMS; RANDOM PROJECTIONS; REINFORCEMENT LEARNINGS; TEMPORAL DIFFERENCE LEARNING;

LEARNING ALGORITHMS;

EID: 85162046948     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (21)

1. A. Antos, Cs. Szepesvari, and R. Munos. Learning near-optimal policies with Bellman-residual minimization based fitted policy iteration and a single sample path. Machine Learning Journal, 71:89-129, 2008.
2. J. Boyan. Least-squares temporal difference learning. Proceedings of the 16th International Conference on Machine Learning, pages 49-56, 1999.
3. S. Bradtke and A. Barto. Linear least-squares algorithms for temporal difference learning. Machine Learning, 22:33-57, 1996.
4. A. M. Farahmand, M. Ghavamzadeh, Cs. Szepesvári, and S. Mannor. Regularized policy iteration. In Proceedings of Advances in Neural Information Processing Systems 21, pages 441-448. MIT Press, 2008.
5. A. M. Farahmand, M. Ghavamzadeh, Cs. Szepesvári, and S. Mannor. Regularized fitted Qiteration for planning in continuous-space Markovian decision problems. In Proceedings of the American Control Conference, pages 725-730, 2009.
6. M. Ghavamzadeh, A. Lazaric, O. Maillard, and R. Munos. LSPI with random projections. Technical Report inria-00530762, INRIA, 2010.
7. P. Keller, S. Mannor, and D. Precup. Automatic basis function construction for approximate dynamic programming and reinforcement learning. In Proceedings of the Twenty-Third International Conference on Machine Learning, pages 449-456, 2006.
8. Z. Kolter and A. Ng. Regularization and feature selection in least-squares temporal difference learning. In Proceedings of the Twenty-Sixth International Conference on Machine Learning, pages 521-528, 2009.
9. M. Lagoudakis and R. Parr. Least-squares policy iteration. Journal of Machine Learning Research, 4:1107-1149, 2003.
10. A. Lazaric, M. Ghavamzadeh, and R. Munos. Finite-sample analysis of least-squares policy iteration. Technical Report inria-00528596, INRIA, 2010.
11. A. Lazaric, M. Ghavamzadeh, and R. Munos. Finite-sample analysis of LSTD. In Proceedings of the Twenty-Seventh International Conference on Machine Learning, pages 615-622, 2010.
12. M. Loth, M. Davy, and P. Preux. Sparse temporal difference learning using lasso. In IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, pages 352-359, 2007.
13. S. Mahadevan. Representation policy iteration. In Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence, pages 372-379, 2005.
14. O. Maillard and R. Munos. Compressed least-squares regression. In Proceedings of Advances in Neural Information Processing Systems 22, pages 1213-1221, 2009.
15. O. Maillard and R. Munos. Brownian motions and scrambled wavelets for least-squares regression. Technical Report inria-00483014, INRIA, 2010.
16. I. Menache, S. Mannor, and N. Shimkin. Basis function adaptation in temporal difference reinforcement learning. Annals of Operations Research, 134:215-238, 2005.
17. R. Parr, C. Painter-Wakefield, L. Li, and M. Littman. Analyzing feature generation for valuefunction approximation. In Proceedings of the Twenty-Fourth International Conference on Machine Learning, pages 737-744, 2007.
18. M. Petrik, G. Taylor, R. Parr, and S. Zilberstein. Feature selection using regularization in approximate linear programs for Markov decision processes. In Proceedings of the Twenty- Seventh International Conference on Machine Learning, pages 871-878, 2010.
19. M. Rudelson and R. Vershynin. Non-asymptotic theory of random matrices: extreme singular values. In Proceedings of the International Congress of Mathematicians, 2010.
20. R. Sutton and A. Barto. Reinforcement Learning: An Introduction. MIP Press, 1998.
21. S. Vempala. The Random Projection Method. American Mathematical Society, 2004.