An analysis of linear models, linear value-function approximation, and feature selection for reinforcement learning

Proceedings of the 25th International Conference on Machine Learning

Volumn , Issue , 2008, Pages 752-759

  Parr, Ronald ^a   Li, Hong ^b   Taylor, Gavin ^a   Painter Wakefield, Christopher ^a   Littman, Michael L ^b  

^a Duke University   (United States)

^b RUTGERS UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

MACHINE LEARNING; REINFORCEMENT LEARNING; FUNCTIONS; LEARNING SYSTEMS; REINFORCEMENT; ROBOT LEARNING;

BELLMAN ERROR; FEATURE SELECTION ALGORITHM; LINEAR MODEL APPROXIMATIONS; MODEL APPROXIMATIONS; VALUE FUNCTION APPROXIMATION; VALUE FUNCTIONS;

FEATURE EXTRACTION; PROBABILITY DENSITY FUNCTION;

APPROXIMATION ERRORS; BELLMAN ERRORS; FEATURE SELECTIONS; LINEAR MODELS; VALUE-FUNCTION APPROXIMATIONS;

EID: 56449092660     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1390156.1390251     Document Type: Conference Paper

References (15)

1. Boyan, J. A. (1999). Least-squares temporal difference learning. ICML-99.
2. Bradtke, S., & Barto, A. (1996). Linear least-squares algorithms for temporal difference learning. Machine Learning, 2.
3. Dean, T., & Givan, R. (1997). Model minimization in Markov decision processes. AAAI-97.
4. Keller, P., Mannor, S., & Precup, D. (2006). Automatic basis function construction for approximate dynamic programming and reinforcement learning. ICML 2006.
5. Koller, D., & Parr, R. (1999). Computing factored value functions for policies in structured MDPs. IJCAI-99.
6. Lagoudakis, M., & Parr, R. (2003). Least squares policy iteration. JMLR, 4.
7. Mahadevan, S., & Maggioni, M. (2007). Proto-value functions: A Laplacian framework for learning representation and control in Markov decision processes. JMLR, 8.
8. Mallat, S. G., & Zhang, Z. (1993). Matching pursuits with time-frequency dictionaries. IEEE Trans. on Signal Processing, 41.
9. Parr, R., Painter-Wakefield, C., Li, L., & Littman, M. (2007). Analyzing feature generation for value-function approximation. ICML-07.
10. Petrik, M. (2007). An analysis of Laplacian methods for value function approximation in MDPs. IJCAI-07.
11. Sanner, S., & Boutilier. C. (2005). Approximate linear programming for first-order MDPs. UAI-05.
12. Sutton, R. S. (1988). Learning to predict by the methods of temporal differences. Machine Learning, 3.
13. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. The MIT Press.
14. Wu, J.-H., & Givan, R. (2004). Feature-discovering approximate value iteration methods (Technical Report TR-ECE-04-06). Purdue University.
15. Yu, H., & Bertsekas, D. (2006). Convergence results for some temporal difference methods based on least squares (Technical Report LIDS-2697). MIT.