Regularized fitted q-iteration for planning in continuous-space markovian decision problems

Proceedings of the American Control Conference

Volumn , Issue , 2009, Pages 725-730

  Farahmand, Amir Massoud ^a   Ghavamzadeh, Mohammad ^a   Szepesvári, Csaba ^a   Mannor, Shie ^b  

^a UNIVERSITY OF ALBERTA   (Canada)

^b MCGILL UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

FINITE SAMPLES; GENERALIZATION BOUND; ITERATION ALGORITHMS; MACHINE-LEARNING; MARKOVIAN DECISION PROBLEMS; NONLINEAR FUNCTIONS; PLANNING PROBLEM; REGULARIZATION PROCEDURE; SMALL SAMPLE SIZE; VALUE FUNCTIONS;

ALGORITHMS; EDUCATION; FUNCTIONS; REINFORCEMENT LEARNING; SAMPLING;

REINFORCEMENT;

EID: 70449644892     PISSN: 07431619     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ACC.2009.5160611     Document Type: Conference Paper

References (20)

1. A. Antos, R. Munos, and Cs. Szepesvári. Fitted q-iteration in continuous action-space mdps. In Advances in Neural Information Processing Systems, 2007. (accepted).
2. A. Antos, Cs. Szepesvári, and R. Munos. Learning near-optimal policies with Bellman-residual minimization based fitted policy iteration and a single sample path. Machine Learning, 71(1):89-129, April 2008. Published Online First: 14 Nov, 2007, DOI: 10.1007/s10994-007-5038-2.
3. D. P. Bertsekas and S.E. Shreve. Stochastic Optimal Control (The Discrete Time Case). Academic Press, New York, 1978.
4. F. Bunea, A. Tsybakov, and M. Wegkamp. Sparsity oracle inequalities for the lasso. Electronic Journal of Statistics, 1:169-194, 2007.
5. F. Chaumette and S. Hutchinson. Visual servo control, part I: Basic approaches. IEEE Robotics and Automation Magazine, 13(4):82-90, December 2006.
6. P.I. Corke. A robotics toolbox for MATLAB. IEEE Robotics and Automation Magazine, 3(1):24-32, March 1996.
7. Y. Engel, S. Mannor, and R. Meir. Reinforcement learning with Gaussian processes. In ICML '05: Proceedings of the 22nd inter- national conference on Machine learning, pages 201-208, New York, NY, USA, 2005. ACM.
8. D. Ernst, P. Geurts, and L. Wehenkel. Tree-based batch mode reinforcement learning. Journal of Machine Learning Research, 6:503-556, 2005.
9. A. M. Farahmand, M. Ghavamzadeh, Cs. Szepesvári, and Sh. Mannor. Regularized policy iteration. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, editors, Advances in Neural Information Processing Systems 21, pages 441-448. 2009.
10. L. Györfi, M. Kohler, A. Krzyżak, and H. Walk. A distribution-free theory of nonparametric regression. Springer-Verlag, New York, 2002.
11. T. Jung and D. Polani. Least squares SVM for least squares TD learning. In ECAI, pages 499-503, 2006.
12. M.G. Lagoudakis and R. Parr. Reinforcement learning as classification: Leveraging modern classifiers. In ICML-03, pages 424-431, 2003.
13. M. Loth, M. Davy, and P. Preux. Sparse temporal difference learning using LASSO. In IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning, 2007.
14. S. Mannor, I. Menache, and N. Shimkin. Basis function adaptation in temporal difference reinforcement learning. Annals of Operations Research, 134:215-238, 2005.
15. G.L. Mariottini and D. Prattichizzo. Egt: a toolbox for multiple view geometry and visual servoing. IEEE Robotics and Automation Magazine, 3(12), December 2005.
16. S. P. Meyn. Control Techniques for Complex Networks. Cambridge, 2008.
17. V. Mnih, Cs. Szepesvári, and J.-Y. Audibert. Empirical Bernstein stopping. In Proceedings of the 25th Annual International Conference on Machine Learning (ICML 2008), pages 672-679, 2008.
18. R. Parr, C. Painter-Wakefield, L. Li, and M.L. Littman. Analyzing feature generation for value-function approximation. In ICML, pages 737-744, 2007.
19. B. Schölkopf and A.J. Smola. Learning with Kernels. MIT Press, Cambridge, MA, 2002.
20. D-X. Zhou. Capacity of reproducing kernel spaces in learning theory. IEEE Transactions on Information Theory, 49:1743-1752, 2003.