Learned graphical models for probabilistic planning provide a new class of movement primitives

Frontiers in Computational Neuroscience

Volumn , Issue DEC, 2012, Pages

  Rückert, Elmar A ^a   Neumann, Gerhard ^b   Toussaint, Marc ^c   Maass, Wolfgang ^a  

^a GRAZ UNIVERSITY OF TECHNOLOGY   (Austria)

^b DARMSTADT UNIVERSITY OF TECHNOLOGY   (Germany)

^c FREIE UNIVERSITÄT BERLIN   (Germany)

Author keywords

Graphical models; Motor planning; Movement primitives; Optimal control; Reinforcement learning

Indexed keywords

BIOLOGICAL MOVEMENTS; CONTROL POLICY; FUNCTION PARAMETERS; GRAPHICAL MODEL; ITS EFFICIENCIES; MOTOR PLANNING; MOTOR SKILL LEARNING; MOVEMENT PRIMITIVES; OPTIMAL CONTROLS; OPTIMALITY; PARAMETRIZATIONS; PROBABILISTIC INFERENCE; PROBABILISTIC PLANNING; REFERENCE TRAJECTORIES; SALIENT FEATURES; STOCHASTIC OPTIMAL CONTROL; SYSTEM DYNAMICS; VIA POINT;

COST FUNCTIONS; DYNAMICAL SYSTEMS; GRAPHIC METHODS; REINFORCEMENT LEARNING;

SPEECH RECOGNITION;

ARTICLE; BODY EQUILIBRIUM; HUMAN; INTRINSIC COST FUNCTION; INTRINSIC PROBABILISTIC PLANNING; LEARNED GRAPHICAL MODEL; LEARNING; MOTOR PERFORMANCE; MOVEMENT (PHYSIOLOGY); MOVEMENT PRIMITIVE; PARAMETRIC TEST; POWER OF A TEST; PROBABILITY; PROCESS OPTIMIZATION; STATISTICAL MODEL; STATISTICAL PARAMETERS; TASK PERFORMANCE;

EID: 84871259138     PISSN: 16625188     EISSN: None     Source Type: Journal    
DOI: 10.3389/fncom.2012.00097     Document Type: Article

References (35)

1. C. G. Atkeson, A. W. Moore, and S. Schaal. Locally Weighted Learning for Control. Artificial Intelligence Review, 11:75-113, 1997.
2. C.G. Atkeson and B. Stephens. Multiple Balance Strategies from one Optimization Criterion. In Proceedings of the 7th International Conference on Humanoid Robots, pages 57-64, Pittsburgh, USA, 2007.
3. N. A. Bernstein. The Co-ordination and regulation of movements. Pergamon Press Ltd., first english edition edition, 1967.
4. E. Bizzi, V.C.K Cheung, A. d'Avella, P. Saltiel, and Tresch M. Combining Modules for Movement. Brain Research Reviews, 57:125-133, 2008.
5. Manu Chhabra and Robert A. Jacobs. Properties of Synergies Arising from a Theory of Optimal Motor Behavior. Neural Computation, 18:2320-2342, 2006.
6. Andrea d'Avella and Dinesh K Pai. Modularity for Sensorimotor Control: Evidence and a New Prediction. Journal of Motor Behavior, 42(6):361-369, 2010.
7. Andrea d'Avella, Philippe Saltiel, and Emilio Bizzi. Combinations of Muscle Synergies in the Construction of a Natural Motor Behavior. Nature, 6(3): 300-308, March 2003.
8. N. Hansen, S.D. Muller, and P. Koumoutsakos. Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES). Evolutionary Computation, 11(1):1-18, 2003.
9. V. Heidrich-Meisner and C. Igel. Neuroevolution Strategies for Episodic Reinforcement Learning. Journal of Algorithms, 64(4):152-168, oct 2009a.
10. V. Heidrich-Meisner and C. Igel. Hoeffding and Bernstein Races for Selecting Policies in Evolutionary Direct Policy Search. In ICML'09: Proceedings of the 26th Annual International Conference on Machine Learning, pages 401-408. ACM, 2009b.
11. Auke Jan Ijspeert and Stefan Schaal. Learning Attractor Landscapes for Learning Motor Primitives. In Advances in Neural Information Processing Systems 15, (NIPS 2003), pages 1523-1530. MIT Press, Cambridge, MA, 2003.
12. Bert Kappen, Vicenç Gómez, and Manfred Opper. Optimal Control as a Graphical Model Inference Problem. Computing Research repository (CoRR), abs/0901.0633, 2009.
13. H. J. Kappen. An Introduction to Stochastic Control Theory, Path Integrals and Reinforcement Learning. In Cooperative Behavior in Neural Systems, volume 887 of American Institute of Physics Conference Series, pages 149-181, 2007.
14. S. M. Khansari-Zadeh and Aude Billard. Learning Stable Non-Linear Dynamical Systems with Gaussian Mixture Models. IEEE Transaction on Robotics, 27 (5):943-957, 2011.
15. J. Kober and J. Peters. Policy Search for Motor Primitives in Robotics. Machine Learning Journal, online first:1-33, 2010.
16. Jens Kober, Erhan Oztop, and Jan Peters. Reinforcement Learning to adjust Robot Movements to New Situations. In Proceedings of the 2010 Robotics: Science and Systems Conference (RSS 2010), 2010.
17. J. Kuffner and S. La Valle. RRT-Connect: An efficient Approach to Single-Query Path Planning. In Proceedings of the 2000 IEEEE International Conference on Robotics and Automation ICRA, San Francisco,CA, pages 995-1001, April 2000.
18. Djordje Mitrovic, Stefan Klanke, and Sethu Vijayakumar. Adaptive Optimal Feedback Control with Learned Internal Dynamics Models. In From Motor Learning to Interaction Learning in Robots, pages 65-84. 2010.
19. G. Neumann, W. Maass, and J. Peters. Learning Complex Motions by Sequencing Simpler Motion Templates. In Proceedings of the 26th International Conference on Machine Learning, (ICML 2009), pages 753-760, Montreal, Canada, 2009.
20. D.Nguyen-Tuong, J. Peters, M. Seeger, and B. Schölkopf. Learning Inverse Dynamics: A Comparison. In 16th European Symposium on Artificial Neural Networks, (ESANN 2008), pages 13-18, Bruges, Belgium, 2008a.
21. D Nguyen-Tuong, M Seeger, and J Peters. Local Gaussian Process Regression for Real Time Online Model Learning and Control. In Proceedings of 22nd Annual Conference on Neural Information Processing Systems, (NIPS 2008), pages 1193-1200, Vancouver, BC, Canada, 2008b.
22. P. Pastor, H. Hoffmann, T. Asfour, and S. Schaal. Learning and Generalization of Motor Skills by Learning from Demonstration. In International Conference on Robotics and Automation (ICRA 2009), 2009.
23. J. Peters and S. Schaal. Reinforcement Learning of Motor Skills with Policy Gradients. Neural Networks, (4):682-697, 2008.
24. J. Peters, M. Mistry, F. E. Udwadia, J. Nakanishi, and S. Schaal. A Unifying Methodology for Robot Control with Redundant DOFs. Autonomous Robots, (1):1-12, 2008.
25. Elmar A. Rückert and Gerhard Neumann. Stochastic Optimal Control Methods for Investigating the Power of Morphological Computation. Artificial Life, pages 1-33, 2012. ACCEPTED for publication.
26. Stefan Schaal, Jan Peters, Jun Nakanishi, and Auke Jan Ijspeert. Learning Movement Primitives. In International Symposium on Robotics Research, (ISRR 2003), pages 561-572, 2003.
27. F. Sehnke, C. Osendorfer, T. Rückstieß, A. Graves, J. Peters, and J. Schmidhu-ber. Parameter-Exploring Policy Gradients. Neural Networks, 23(4):551-559, May 2010.
28. E.Theodorou, J. Buchli, and S. Schaal. Reinforcement Learning of Motor Skills in High Dimensions: a Path Integral Approach. In Robotics and Automation (ICRA), 2010 IEEE International Conference on, pages 2397-2403, 2010.
29. E. Todorov and M. Jordan. Optimal feedback control as a theory of motor coordination. Nature Neuroscience, 5:1226-1235, 2002.
30. E. Todorov and Weiwei Li. A generalized Iterative LQG Method for Locally-Optimal Feedback Control of Constrained Nonlinear Stochastic Systems. In Proceedings of the 24th American Control Conference, volume 1 of (ACC 2005), pages 300- 306, Portland, Oregon, USA, 2005.
31. Marc Toussaint. Robot Trajectory Optimization using Approximate Inference. In Proceedings of the 26th International Conference on Machine Learning, (ICML 2009), pages 1049-1056, Montreal, Canada, 2009.
32. J. Trommershauser, S. Gepshtein, L. Maloney, and M. Banks. Optimal Compensation for Changes in Task-Relevant Movement Variability. Journal of Neuroscience, 25:7169-7178, 2005.
33. S. Vijayakumar, A. D'Souza, and S. Schaal. Incremental Online Learning in High Dimensions. Neural Computation, 17(12):2602-2634, dec 2005.
34. Daan Wierstra, Tom Schaul, Jan Peters, and Juergen Schmidhuber. Episodic Reinforcement Learning by Logistic Reward-Weighted Regression. In Proceedings of the 18th international conference on Artificial Neural Networks, Part I, (ICANN 2008), pages 407-416, Berlin, Heidelberg, 2008. Springer-Verlag.
35. Ronald J. Williams. Simple Statistical Gradient-Following Algorithms for Con-nectionist Reinforcement Learning. Machine Learning, 8:229-256, 1992.