Value-iteration based fitted policy iteration: Learning with a single trajectory

Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007

Volumn , Issue , 2007, Pages 330-337

  Antos, András ^a   Szepesvári, Csaba ^b   Munos, Rémi ^c  

^a INSTITUTE OF EXPERIMENTAL MEDICINE   (Hungary)

^b UNIVERSITY OF ALBERTA   (Canada)

^c UNIV LILLE   (France)

Author keywords

[No Author keywords available]

Indexed keywords

DECISION THEORY; ITERATIVE METHODS; MARKOV PROCESSES; POLYNOMIALS; PROBLEM SOLVING; PUBLIC POLICY;

APPROXIMATE VALUE ITERATION; INTERMEDIATE POLICIES; POLICY ITERATION;

REINFORCEMENT LEARNING;

EID: 34548752490     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ADPRL.2007.368207     Document Type: Conference Paper

References (17)

1. M. Lagoudakis and R. Parr. Least-squares policy iteration. Journal of Machine Learning Research, 4:1107-1149, 2003.
2. D. Ernst, P. Geurts, and L. Wehenkel. Tree-based batch mode reinforcement learning. Journal of Machine Learning Research, 6:503-556, 2005.
3. Cs. Szepesvári and R. Munos. Finite time bounds for sampling based fitted value iteration. In ICML'2005, pages 881-886, 2005.
4. D. P. Bertsekas and S.E. Shreve. Stochastic Optimal Control (The Discrete Time Case). Academic Press, New York, 1978.
5. B. Yu. Rates of convergence for empirical processes of stationary mixing sequences. The Annals of Probability, 22(1):94-116, January 1994.
6. R. Meir. Nonparametric time series prediction through adaptive model selection. Machine Learning, 39(1):5-34, April 2000.
7. A. Antos, Cs. Szepesvári, and R. Munos. Learning near-optimal policies with Bellman-residual minimization based fitted policy iteration and a single sample path. In COLT-19, pages 574-588, 2006.
8. M. Anthony and P. L. Bartlett. Neural Network Learning: Theoretical Foundations. Cambridge University Press, 1999.
9. A. Antos, Cs. Szepesvári, and R. Munos. Approximate action-value iteration in continuous state spaces: learning with a single trajectory. (submitted), 2006.
10. E.W. Cheney. Introduction to approximation theory. McGraw-Hill, London, New York, 1966.
11. L. Györfi, M. Kohler, A. Krzyżak, and H. Walk. A distribution-free theory of nonparametric regression. Springer-Verlag, New York, 2002.
12. A. Nobel. Histogram regression estimation using data-dependent partitions. Annals of Statistics, 24(3): 1084-1105, 1996.
13. D. Haussler. Sphere packing numbers for subsets of the boolean n-cube with bounded Vapnik-Chervonenkis dimension. Journal of Combinatorial Theory Series A, 69:217-232, 1995.
14. R. Munos. Error bounds for approximate policy iteration. In ICML'2003, pages 560-567, 2003.
15. R. Munos and Cs. Szepesvári. Finite time bounds for sampling based fitted value iteration. Journal of Machine Learning Research, 2005. submitted.
16. C.S. Chow and J.N. Tsitsiklis. An optimal multlgrid algorithm for continuous state discrete time stochastic control. IEEE Transactions on Automatic Control, 36(8):898-914, 1991.
17. P. Bougerol and N. Picard. Strict stationarity of generalized autoregressive processes. Annals of Probability, 20:1714-1730, 1992.