Statistically linearized least-squares temporal differences

2010 International Congress on Ultra Modern Telecommunications and Control Systems and Workshops, ICUMT 2010

Volumn , Issue , 2010, Pages 450-457

  Geist, Matthieu ^a   Pietquin, Olivier ^a  

^a UMI Georgia Tech CNRS 2958   (France)

Author keywords

Neural networks; Reinforcement learning; Statistical linearization; Value function approximation

Indexed keywords

ITERATIVE METHODS; LINEARIZATION; NEURAL NETWORKS; REINFORCEMENT LEARNING;

LEAST-SQUARES TEMPORAL DIFFERENCES; LINEAR PARAMETRIZATION; NONLINEAR PARAMETERIZATIONS; POLICY ITERATION; REAL-WORLD PROBLEM; STATISTICAL LINEARIZATION; VALUE FUNCTION APPROXIMATION; VALUE ITERATION;

LEARNING ALGORITHMS;

EID: 79951499926     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICUMT.2010.5676598     Document Type: Conference Paper

References (20)

1. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning), 3rd ed. The MIT Press, March 1998.
2. J. N. Tsitsiklisc and B. Van Roy, "An analysis of temporal-difference learning with function approximation," IEEE Transactions on Automatic Control, vol. 42, pp. 674-690, 1997.
3. S. J. Bradtke and A. G. Barto, "Linear Least-Squares algorithms for temporal difference learning," Machine Learning, vol. 22, no. 1-3, pp. 33-57, 1996.
4. H. Maei, C. Szepesvari, S. Bhatnagar, D. Precup, D. Silver, and R. Sutton, "Convergent Temporal-Difference Learning with Arbitrary Smooth Function Approximation," in Advances in Neural Information Processing Systems 22, 2009, pp. 1204-1212.
5. D. Choi and B. Van Roy, "A Generalized Kalman Filter for Fixed Point Approximation and Efficient Temporal-Difference Learning," Discrete Event Dynamic Systems, vol. 16, pp. 207-239, 2006.
6. L. C. Baird, "Residual Algorithms: Reinforcement Learning with Function Approximation," in Proceedings of the International Conference on Machine Learning (ICML 95), 1995, pp. 30-37.
7. Y. Engel, S. Mannor, and R. Meir, "Bayes Meets Bellman: The Gaussian Process Approach to Temporal Difference Learning," in Proceedings of the International Conference on Machine Learning (ICML 03), 2003, pp. 154-161.
8. M. Geist, O. Pietquin, and G. Fricout, "Kalman Temporal Differences: the deterministic case," in IEEE International Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL 2009), Nashville, TN, USA, April 2009.
9. Y. Engel, S. Mannor, and R. Meir, "Reinforcement Learning with Gaussian Processes," in Proceedings of International Conference on Machine Learning (ICML-05), 2005.
10. M. Geist and O. Pietquin, "Eligibility Traces through Colored Noises," in International Conference on Ultra Modern Control systems (ICUMT 2010 (Control Systems)), Moscow, Russia, October 2010.
11. A. Nedić and D. P. Bertsekas, "Least Squares Policy Evaluation Algorithms with Linear Function Approximation," Discrete Event Dynamic Systems: Theory and Applications, vol. 13, pp. 79-110, 2003.
12. M. G. Lagoudakis and R. Parr, "Least-Squares Policy Iteration," Journal of Machine Learning Research, vol. 4, pp. 1107-1149, 2003.
13. J. Peters, S. Vijayakumar, and S. Schaal, "Reinforcement Learning for Humanoid Robotics," in Third IEEE-RAS International Conference on Humanoid Robots (Humanoids 2003), 2003.
14. T. W. Anderson, An Introduction to Multivariate Statistical Analysis. Wiley, 1984.
15. S. J. Julier and J. K. Uhlmann, "Unscented filtering and nonlinear estimation," Proceedings of the IEEE, vol. 92, no. 3, pp. 401-422, 2004.
16. P. Nørg̊ard, N. Poulsen, and O. Ravn, "New developments in state estimation for nonlinear systems," Automatica, vol. 36, no. 11, pp. 1627-1638, 2000.
17. M. Geist and O. Pietquin, "Statistically Linearized Recursive Least Squares," in Proceedings of the IEEE International Workshop on Machine Learning for Signal Processing (MLSP 2010), Kittilä (Finland), August-September 2010, 5 pages, to appear.
18. P. E. Gill, G. H. Golub, W. Murray, and M. A. Saunders, "Methods for Modifying Matrix Factorization," Mathematics of Computation, vol. 28, no. 126, pp. 505-535, April 1974.
19. A. Antos, C. Szepesvári, and R. Munos, "Learning near-optimal policies with Bellman-residual minimization based fitted policy iteration and a single sample path," Machine Learning, vol. 71, no. 1, pp. 89-129, 2008.
20. M. Riedmiller, "Neural Fitted Q Iteration - First Experiences with a Data Efficient Neural Reinforcement Learning Method," in European Conference on Machine Learning, 2005, pp. 317-328.