Learning near-optimal policies with bellman-residual minimization based fitted policy iteration and a single sample path

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 4005 LNAI, Issue , 2006, Pages 574-588

  Antos, András ^a   Szepesvári, Csaba ^a   Munos, Rémi ^b  

^a INSTITUTE OF EXPERIMENTAL MEDICINE   (Hungary)

^b CENTRE DE MATHÉMATIQUES APPLIQUÉES   (France)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL COMPLEXITY; DECISION MAKING; ITERATIVE METHODS; LEARNING ALGORITHMS; MARKOV PROCESSES;

BEHAVIOUR POLICY; BELLMAN-RESIDUAL MINIMIZATION; MARKOVIAN DECISION PROBLEMS;

PROBLEM SOLVING;

EID: 33746032553     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/11776420_42     Document Type: Conference Paper

References (23)

1. M. Lagoudakis and R. Parr. Least-squares policy iteration. Journal of Machine Learning Research, 4:1107-1149, 2003.
2. A. Antos, Cs. Szepesvári, and R. Munos. Learning near-optimal policies with fitted policy iteration and a single sample path: approximate iterative policy evaluation, (submitted to ICML'2006, 2006.
3. D. P. Bertsekas and S.E. Shreve. Stochastic Optimal Control (The Discrete Time Case). Academic Press, New York, 1978.
4. R.S. Button and A.G. Barto. Toward a modern theory of adaptive networks: Expectation and prediction. In Proc. of the Ninth Annual Conference of Cognitive Science Society. Erlbaum, Hillsdale, NJ, USA, 1987.
5. R. Munos. Error bounds for approximate policy iteration. 19th International Conference on Machine Learning, pages 560-567, 2003.
6. S.P. Meyn and R. Tweedie. Markov Chains and Stochastic Stability. Springer-Verlag, New York, 1993.
7. M. Anthony and P. L. Bartlett. Neural Network Learning: Theoretical Foundations. Cambridge University Press, 1999.
8. B. Yu. Rates of convergence for empirical processes of stationary mixing sequences. The Annals of Probability, 22(1):94-116, January 1994.
9. A. Nobel. Histogram regression estimation using data-dependent partitions. Annals of Statistics, 24(3):1084-1105, 1996.
10. D. Haussler. Sphere packing numbers for subsets of the boolean n-cube with bounded Vapnik-Chervonenkis dimension. Journal of Combinatorial Theory Series A, 69:217-232, 1995.
11. A.L. Samuel. Some studies in machine learning using the game of checkers. IBM Journal on Research and Development, pages 210-229, 1959.
12. Reprinted in Computers and Thought, E.A. Feigenbaum and J. Feldman, editors, McGraw-Hill, New York, 1963.
13. R.E. Bellman and S.E. Dreyfus. Functional approximation and dynamic programming. Math. Tables and other Aids Comp., 13:247-251, 1959.
14. Dimitri P. Bertsekas and J. Tsitsiklis. Neuro-Dynamic Programming. Athena Scientific, 1996.
15. Richard S. Sutton and Andrew G. Barto. Reinforcement learning: An introduction. Bradford Book, 1998.
16. Geoffrey J. Gordon. Stable function approximation in dynamic programming. In Armand Prieditis and Stuart Russell, editors, Proceedings of the Twelfth International Conference on Machine Learning, pages 261-268, San Francisco, CA, 1995. Morgan Kaufmann.
17. J. N. Tsitsiklis and B. Van Roy. Feature-based methods for large scale dynamic programming. Machine Learning, 22:59-94, 1996.
18. Carlos Guestrin, Daphne Koller, and Ronald Parr. Max-norm projections for factored mdps. Proceedings of the International Joint Conference on Artificial Intelligence, 2001.
19. D. Ernst, P. Geurts, and L. Wehenkel. Tree-based batch mode reinforcement learning. Journal of Machine Learning Research, 6:503-556, 2005.
20. X. Wang and T.G. Dietterich. Efficient value function approximation using regression trees. In Proceedings of the IJCAI Workshop on Statistical Machine Learning for Large-Scale Optimization, Stockholm, Sweden, 1999.
21. T. G. Dietterich and X. Wang. Batch value function approximation via support vectors. In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems 14, Cambridge, MA, 2002. MIT Press.
22. Cs. Szepesvári and R. Munos. Finite time bounds for sampling based fitted value iteration. In ICML'2005, 2005.
23. R. Meir. Nonparametric time series prediction through adaptive model selection. Machine Learning, 39(1):5-34, April 2000.