Hybrid least-squares algorithms for approximate policy evaluation

Machine Learning

Volumn 76, Issue 2-3, 2009, Pages 243-256

  Johns, Jeff ^a   Petrik, Marek ^a   Mahadevan, Sridhar ^a  

^a University of Massachusetts Amherst   (United States)

Author keywords

Markov decision processes; Reinforcement learning

Indexed keywords

FIXED POINT METHODS; FIXED-POINT ALGORITHMS; GEOMETRIC INTERPRETATION; HYBRID ALGORITHMS; LARGE DOMAIN; LEAST-SQUARES ALGORITHMS; MARKOV DECISION PROCESSES; OPTIMIZATION CRITERIA; POLICY EVALUATION; POLICY ITERATION; RESIDUAL METHOD; TARGET VALUES;

CIRCUIT THEORY; MARKOV PROCESSES; OPTIMIZATION; REINFORCEMENT; REINFORCEMENT LEARNING;

ALGORITHMS;

EID: 68949099445     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-009-5128-4     Document Type: Conference Paper

References (15)

1. A. Antos C. Szepesvári R. Munos 2008 Learning near-optimal policies with Bellman-residual minimization based fitted policy iteration and a single sample path Machine Learning 71 1 89 129
2. Baird, L. (1995). Residual algorithms: Reinforcement learning with function approximation. In Proceedings of the 12th international conference on machine learning (pp. 30-37).
3. Boyan, J. (1999). Least-squares temporal difference learning. In Proceedings of the 16th international conference on machine learning (pp. 49-56).
4. S. Bradtke A. Barto 1996 Linear least-squares algorithms for temporal difference learning Machine Learning 22 1-3 33 57
5. Koller, D., & Parr, R. (2000). Policy iteration for factored MDPs. In Proceedings of the 16th conference on uncertainty in artificial intelligence (pp. 326-334). San Mateo: Morgan Kaufmann.
6. M. Lagoudakis R. Parr 2003 Least-squares policy iteration Journal of Machine Learning Research 4 1107 1149
7. Lagoudakis, M., Parr, R., & Littman, M. (2002). Least-squares methods in reinforcement learning for control. In Proceedings of the 2nd Hellenic conference on artificial intelligence (pp. 249-260).
8. Li, L. (2008). A worst-case comparison between temporal difference and residual gradient with linear function approximation. In Proceedings of the 25th international conference on machine learning (pp. 560-567).
9. Mahadevan, S. (2005). Representation policy iteration. In Proceedings of the 21st conference on uncertainty in artificial intelligence (pp. 372-379).
10. Munos, R. (2003). Error bounds for approximate policy iteration. In Proceedings of the 20th international conference on machine learning (pp. 560-567).
11. Puterman, M. (1994). Markov decision processes: discrete stochastic dynamic programming. New York: Wiley.
12. Schoknecht, R. (2003). Optimality of reinforcement learning algorithms with linear function approximation. In Advances in neural information processing systems (Vol. 15, pp. 1555-1562).
13. P. Schweitzer A. Seidmann 1985 Generalized polynomial approximations in Markovian decision processes Journal of Mathematical Analysis and Applications 110 568 582
14. R. Sutton 1988 Learning to predict by the methods of temporal differences Machine Learning 3 9 44
15. Sutton, R., & Barto, A. (1998). Reinforcement learning. Cambridge: MIT Press.