Least-Squares λ Policy Iteration: Bias-variance trade-off in control problems

ICML 2010 - Proceedings, 27th International Conference on Machine Learning

Volumn , Issue , 2010, Pages 1071-1078

  Thiery, Christophe ^a   Scherrer, Bruno ^a  

^a LORIA   (France)

Author keywords

[No Author keywords available]

Indexed keywords

IN-CONTROL; LARGE SPACES; LEAST SQUARE; PERFORMANCE BOUNDS; POLICY ITERATION; TETRIS GAME; TRAINING SAMPLE; VALUE FUNCTION APPROXIMATION; VALUE ITERATION;

LEARNING SYSTEMS;

EID: 77956525931     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (16)

1. Bertsekas, D. and Ioffe, S. Temporal differences-based policy iteration and applications in neuro-dynamic programming. Technical report, MIT, 1996.
2. Bertsekas, D.P. and Tsitsiklis, J.N. Neurodynamic Programming. Athena Scientific, 1996.
3. Boyan, J. A. Technical update: Least-squares temporal difference learning. Machine Learning, 49:233-246, 2002.
4. Bradtke, S. J. and Barto, A.G. Linear least-squares algorithms for temporal difference learning. Machine Learning, 22:33-57, 1996.
5. Kearns, M. and Singh, S. Bias-variance error bounds for temporal difference updates. In In Proceedings of the 13th Annual Conference on Computational Learning Theory, pp. 142-147, 2000.
6. Lagoudakis, M. G. and Parr, R. Least-squares policy iteration. Journal of Machine Learning Research, 4: 1107-1149, 2003.
7. Lagoudakis, Michail G., Parr, Ronald, and Littman, Michael L. Least-squares methods in reinforcement learning for control. In In SETN'02: Proceedings of the Second Hellenic Conference on AI, pp. 249-260. Springer-Verlag, 2002.
8. Nedić, A. and Bertsekas, D. P. Least squares policy evaluation algorithms with linear function approximation. Discrete Event Dynamic Systems, 13(1-2): 79-110, 2003.
9. Puterman, M. Markov Decision Processes. Wiley, New York, 1994.
10. Schoknecht, Ralf. Optimality of reinforcement learning algorithms with linear function approximation. In NIPS, pp. 1555-1562, 2002.
11. Sutton, R. S., Maei, H. R., Precup, D., Bhatna-gar, S., Silver, D., Szepesvári, C, and Wiewiora, E. Fast gradient-descent methods for temporal-difference learning with linear function approximation. In ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 993-1000, 2009.
12. Sutton, R.S. and Barto, A.G. Reinforcement Learning, An introduction. BradFord Book. The MIT Press, 1998.
13. Szepesvári, Csaba and Munos, Rémi. Finite time bounds for sampling based fitted value iteration. In ICML '05: Proceedings of the 22nd international conference on Machine learning, pp. 880-887. ACM, 2005.
14. Thiery, Christophe and Scherrer, Bruno. Building Controllers for Tetris. International Computer Games Association Journal, 32:3-11, 2009.
15. Thiery, Christophe and Scherrer, Bruno. Performance bound for Approximate Optimistic Policy Iteration. Technical report, 2010. URL http://hal.inria.fr/ inria-00480952.
16. Yu, H. and Bertsekas, D. P. Convergence Results for Some Temporal Difference Methods Based on Least Squares. IEEE Trans. Automatic Control, 54:1515-1531, 2009.