Reinforcement learning of motor skills in high dimensions: A path integral approach

Proceedings - IEEE International Conference on Robotics and Automation

Volumn , Issue , 2010, Pages 2397-2403

  Theodorou, Evangelos ^a   Buchli, Jonas ^a   Schaal, Stefan ^a  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONTINUOUS STATE-ACTION SPACES; EMPIRICAL EVALUATIONS; ESTIMATION THEORY; GENERAL APPROACH; GRADIENT BASED; GRADIENT LEARNING; HIGH DIMENSIONS; HIGH-DIMENSIONAL; MATRIX INVERSIONS; MOTOR SKILLS; MOTOR SYSTEMS; NUMERICAL INSTABILITY; NUMERICALLY ROBUST; OPTIMAL CONTROL THEORY; PARAMETERIZED CONTROL; PATH INTEGRAL; PATH INTEGRAL APPROACH; PERFORMANCE IMPROVEMENTS; REAL-WORLD; STOCHASTIC OPTIMAL CONTROL;

CONTROL; OPTIMIZATION; QUANTUM THEORY; REINFORCEMENT LEARNING; ROBOTICS; ROBOTS;

LEARNING ALGORITHMS;

EID: 77955836276     PISSN: 10504729     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ROBOT.2010.5509336     Document Type: Conference Paper

References (26)

1. J. Peters. Machine learning of motor skills for robotics. PhD thesis, 2007.
2. Christos Dimitrakakis and Michail G. Lagoudakis. Rollout sampling approximate policy iteration. Machine Learning, 72(3), September 2008.
3. P. Dayan and G. Hinton. Using em for reinforcement learning. Neural Computation, 9, 1997.
4. J. Koeber and J. Peters. Learning motor primitives in robotics. In D. Schuurmans, J. Benigio, and D. Koller, editors, Advances in Neural Information Processing Systems 21 (NIPS 2008), Vancouver, BC, Dec. 8-11, 2009. Cambridge, MA: MIT Press. clmc.
5. J. Peters and S. Schaal. Learning to control in operational space. International Journal of Robotics Research, 27:197-212, 2008. clmc.
6. M. Toussaint and A. Storkey. Probabilistic inference for solving discrete and continuous state markov decision processes, 2006.
7. Marc P. Deisenroth, Carl E. Rasmussen, and Jan Peters. Gaussian Process Dynamic Programming. Neurocomputing, 72(7-9):1508-1524, March 2009.
8. Mohammad Ghavamzadeh and Yaakov Engel. Bayesian actor-critic algorithms. In ICML '07: Proceedings of the 24th international conference on Machine learning, pages 297-304, New York, NY, USA, 2007. ACM.
9. Nikos Vlassis, Marc Toussaint, Georgios Kontes, and Savas Piperidis. Learning model-free robot control by a monte carlo em algorithm. Auton. Robots, 27(2):123-130, 2009.
10. H. J. Kappen, W. Wiegerinck, and B. van den Broek. A path integral approach to agent planning. In AAMAS, 2007.
11. B. van den Broek, W. Wiegerinck, and B. Kappen. Graphical model inference in optimal control of stochastic multi-agent systems. Journal of Artificial Intelligence Research, 32(1):95-122, 2008.
12. A. Ijspeert, J. Nakanishi, and S. Schaal. Learning attractor landscapes for learning motor primitives. In S. Becker, S. Thrun, and K. Obermayer, editors, Advances in Neural Information Processing Systems 15, pages 1547-1554. Cambridge, MA: MIT Press, 2003.
13. Richard S. Sutton and Andrew G. Barto. Reinforcement learning : An introduction. Adaptive computation and machine learning. MIT Press, Cambridge, 1998. 97026416 Richard S. Sutton and Andrew G. Barto. Includes bibliographical references (p. [291]-312) and index.
14. Robert F. Stengel. Optimal control and estimation. Dover books on advanced mathematics. Dover Publications, New York, 1994. 94020406 Robert F. Stengel. ill. ; 21 cm. Originally published: Stochastic optimal control. New York ; Wiley, c1986. With new pref. Includes bibliographical references and index.
15. Wendell Helms Fleming and H. Mete Soner. Controlled Markov processes and viscosity solutions. Applications of mathematics. Springer, New York, 2nd edition, 2006. 2005929857 Wendell H. Fleming, H. Mete Soner. 25 cm.
16. B. K. ksendal. Stochastic differential equations : an introduction with applications. Universitext. Springer, Berlin ; New York, 6th edition, 2003.
17. Jiongmin Yong. Relations among odes, pdes, fsdes, bsdes, and fbsdes. volume 3, pages 2779-2784 vol.3, Dec 1997.
18. H J Kappen. Path integrals and symmetry breaking for optimal control theory. Journal of Statistical Mechanics: Theory and Experiment, 2005(11):P11011, 2005.
19. H. J. Kappen. An introduction to stochastic control theory, path integrals and reinforcement learning. In J. Marro, P. L. Garrido, and J. J. Torres, editors, Cooperative Behavior in Neural Systems, volume 887 of American Institute of Physics Conference Series, pages 149-181, February 2007.
20. Hilbert J. Kappen. Linear theory for control of nonlinear stochastic systems. Phys. Rev. Lett., 95(20):200201, Nov 2005.
21. J. Peters and S. Schaal. Policy gradient methods for robotics. In Proceedings of the IEEE International Conference on Intelligent Robotics Systems (IROS 2006), Beijing, Oct. 9-15, 2006.
22. J. Peters and S. Schaal. Reinforcement learning for parameterized motor primitives. In Proceedings of the 2006 International Joint Conference on Neural Networks (IJCNN 2006), 2006. clmc.
23. E. Todorov. Classic maximum principles and estimation-control dualities for nonlinear stochastic systems. 2009. (Submitted).
24. Emanuel Todorov. Efficient computation of optimal actions. Proc Natl Acad Sci U S A, 106(28):11478-83.
25. H. J. Kappen. Linear theory for control of nonlinear stochastic systems. Phys Rev Lett, 95(20):200201, 2005. Journal Article United States.
26. W. Wiegerinck, B. van den Broek, and H. J. Kappen. Stochastic optimal control in continuous space-time multi-agent system. In UAI, 2006.