Regularized fitted Q-Iteration: Application to planning

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 5323 LNAI, Issue , 2008, Pages 55-68

  Farahmand, Amir Massoud ^a   Ghavamzadeh, Mohammad ^a   Szepesvári, Csaba ^a   Mannor, Shie ^b  

^a UNIVERSITY OF ALBERTA   (Canada)

^b MCGILL UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

PROBABILITY DENSITY FUNCTION; REINFORCEMENT LEARNING;

DECISION PROBLEMS; FUNCTION SPACES; KERNEL FUNCTIONS; KERNEL HILBERT SPACE; LEAST SQUARES (LS); MARKOVIAN;

REINFORCEMENT;

EID: 58449110583     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-540-89722-4_5     Document Type: Conference Paper

References (23)

1. Antes, A., Szepesvári, C., Munos, R.: Value-iteration based fitted policy iteration: learning with a single trajectory. In: IEEE ADPRL, pp. 330-337 (2007)
2. Antos, A., Munos, R., Szepesvári, C.: Fitted Q-iteration in continuous action-space MDPs. In: Advances in Neural Information Processing Systems 20, NIPS 2007 (in print, 2008)
3. Antos, A., Szepesvári, C., Munos, R.: Learning near-optimal policies with Bellman-residual minimization based fitted policy iteration and a single sample path. Machine Learning 71, 89-129 (2008)
4. Bertsekas, D.P., Shreve, S.E.: Stochastic Optimal Control (The Discrete Time Case). Academic Press, New York (1978)
5. Bunea, F., Tsybakov, A., Wegkamp, M.: Sparsity oracle inequalities for the lasso. Electronic Journal of Statistics 1, 169-194 (2007)
6. Engel, Y., Mannor, S., Meir, R.: The kernel recursive least squares algorithm. IEEE Transaction on Signal Processing 52(8), 2275-2285 (2004)
7. Engel, Y., Mannor, S., Meir, R.: Reinforcement learning with Gaussian processes. In: ICML 2005: Proceedings of the 22nd international conference on Machine learning, pp. 201-208. ACM, New York (2005)
8. Ernst, D., Geurts, P., Wehenkel, L.: Tree-based batch mode reinforcement learning. Journal of Machine Learning Research 6, 503-556 (2005)
9. Farahmand, A.M., Ghavamzadeh, M., Szepesvari, C., Mannor, S.: Regularized policy iteration. In: Advances in Neural Information Processing Systems 21, NIPS 2008 (to appear, 2008)
10. Györfi, L., Kohler, M., Krzyzak, A., Walk, H.: A distribution-free theory of non-parametric regression. Springer, New York (2002)
11. Jung, T., Polani, D.: Least squares SVM for least squares TD learning. In: ECAI, pp. 499-503 (2006)
12. Kearns, M., Mansour, Y., Ng, A.Y.: A sparse sampling algorithm for near-optimal planning in large Markovian decision processes. In: Proceedings of IJCAI 1999, pp. 1324-1331 (1999)
13. Lagoudakis, M.G., Parr, R.: Reinforcement learning as classification: Leveraging modern classifiers. In: ICML 2003, pp. 424-431 (2003)
14. Loth, M., Davy, M., Preux, P.: Sparse temporal difference learning using LASSO. In: IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning (2007)
15. Mannor, S., Menache, I., Shimkin, N.: Basis function adaptation in temporal difference reinforcement learning. Annals of Operations Research 134, 215-238 (2005)
16. Munos, R., Szepesvári, C.: Finite-time bounds for fitted value iteration. Journal of Machine Learning Research 9, 815-857 (2008)
17. Ng, A.Y., Jordan, M.: PEGASUS: A policy search method for large MDPs and POMDPs. In: Proceedings of the 16th Conference in Uncertainty in Artificial Intelligence, pp. 406-415 (2000)
18. Ormoneit, D., Sen, S.: Kernel-based reinforcement learning. Machine Learning 49, 161-178 (2002)
19. Parr, R., Painter-Wakefield, C., Li, L., Littman, M.L.: Analyzing feature generation for value-function approximation. In: ICML, pp. 737-744 (2007)
20. Schölkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002)
21. Srebro, N., Ben-David, S.: Learning bounds for support vector machines with learned kernels. In: Lugosi, G., Simon, H.U. (eds.) COLT 2006. LNCS, vol. 4005, pp. 169-183. Springer, Heidelberg (2006)
22. Xu, X., Hu, D., Lu, X.: Kernel-based least squares policy iteration for reinforcement learning. IEEE Trans, on Neural Networks 18, 973-992 (2007)
23. Zhou, D.-X.: Capacity of reproducing kernel spaces in learning theory. IEEE Transactions on Information Theory 49, 1743-1752 (2003)