Incremental basis construction from temporal difference error

Proceedings of the 28th International Conference on Machine Learning, ICML 2011

Volumn , Issue , 2011, Pages 481-488

  Sun, Yi ^a   Gomez, Faustino ^a   Ring, Mark ^a   Schmidhuber, Jürgen ^a  

^a DALLE MOLLE INSTITUTE FOR ARTIFICIAL INTELLIGENCE IDSIA   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

BASIS FUNCTIONS; BELLMAN ERROR; DISCOUNT FACTORS; LINEAR COMBINATIONS; ONLINE VERSIONS; REWARD FUNCTION; TEMPORAL DIFFERENCE ERRORS; VALUE FUNCTIONS;

REINFORCEMENT; REINFORCEMENT LEARNING;

ESTIMATION;

EID: 80053457849     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (18)

1. L. Baird. Residual algorithms: Reinforcement learning with function approximation. In ICML'95, 1995.
2. B. Boots and G. J. Gordon. Predictive state temporal difference learning. In NIPS'10, 2010.
3. J. A. Boyan. Technical update: Least-squares temporal difference learning. Machine Learning, 49:233-246, 2002. ISSN 0885-6125.
4. S. J. Bradtke, A. G. Barto, and L. P. Kaelbling. Linear least-squares algorithms for temporal difference learning. Machine Learning, 22:33-57, 1996.
5. A. Geramifard, M. Bowling, and R. S. Sutton. Incremental least-squares temporal difference learning. In AAAI'06, 2006.
6. M. Ghavamzadeh, A. Lazaric, O. Maillard, and R. Munos. Lstd with random projections. In NIPS'10, 2010.
7. P. W. Keller, S. Mannor, and D. Precup. Automatic basis function construction for approximate dynamic programming and reinforcement learning. In ICML'06, 2006.
8. Z. J. Kolter and A. Y. Ng. Regularization and feature selection in least-squares temporal difference learning. In ICML'09, 2009.
9. M. G. Lagoudakis and R. Parr. Least-squares policy iteration. Journal of Machine Learning Research, 4: 1107-1149, 12 2003. ISSN 1532-4435.
10. H. R. Maei and R. S. Sutton. Gq(λ): A general gradient algorithm for temporal-difference prediction learning with eligibility traces. In AGI'10, 2010.
11. S. Mahadevan and B. Liu. Basis construction from power series expansions of value functions. In NIPS'10, 2010.
12. S. Mahadevan, M. Maggioni, and C. Guestrin. Proto-value functions: A laplacian framework for learning representation and control in markov decision processes. Journal of Machine Learning Research, 8: 2007, 2006.
13. R. Parr, C. Painter-Wakefield, L.-H. Li, and M. Littman. Analyzing feature generation for value-function approximation. In ICML'07, 2007.
14. R. S. Sutton. Learning to predict by the methods of temporal differences. Machine Learning, 3:9-44, 1988.
15. R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. MIT Press, 1998.
16. R. S. Sutton, C. Szepesvári, and H. R. Maei. A convergent o(n) algorithm for off-policy temporal-difference learning with linear function approximation. In NIPS'08, 2008.
17. R. S. Sutton, H. R. Maei, D. Precup, S. Bhatnagar, D. Silver, C. Szepesvári, and E. Wiewiora. Fast gradient-descent methods for temporal-difference learning with linear function approximation. In ICML'09, 2009.
18. J.-H. Wu and R. Givan. Feature-discovering approximate value iteration methods. In SARA'05, pages 321-331, 2005.