Learning to pour with a robot arm combining goal and shape learning for dynamic movement primitives

Robotics and Autonomous Systems

Volumn 59, Issue 11, 2011, Pages 910-922

  Tamosiunaite, Minija ^a,b   Nemec, Bojan ^c   Ude, Aleş ^c   Wörgötter, Florentin ^a  

^a UNIVERSITY OF GÖTTINGEN   (Germany)

^b VYTAUTAS MAGNUS UNIVERSITY   (Lithuania)

^c JO EF STEFAN INSTITUTE   (Slovenia)

Author keywords

Dynamic movement primitives; Natural actor critic; PI2 method; Reinforcement learning; Value function approximation

Indexed keywords

ACTOR CRITIC; COMBINED METHOD; DIRECT POLICY SEARCH; DYNAMIC MOVEMENT PRIMITIVES; NATURAL ACTOR CRITIC; PARAMETER REGIMES; PATH INTEGRAL; POLICY SEARCH; ROBOT ARMS; ROBOT MOTION; ROBOTIC APPLICATIONS; SHAPE LEARNING; SHAPE PARAMETERS; TEMPORAL SCALING; VALUE FUNCTION APPROXIMATION;

REINFORCEMENT; ROBOTIC ARMS;

REINFORCEMENT LEARNING;

EID: 80052851862     PISSN: 09218890     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.robot.2011.07.004     Document Type: Article

References (26)

1. A.J. Ijspeert, J. Nakanishi, S. Schaal, Movement imitation with nonlinear dynamical systems in humanoid robots, in: Proc. IEEE Int. Conf. Robotics and Automation, ICRA, 2002, Washington, DC, 2002, pp. 13981403.
2. B. Perk, J. Slotine, Motion primitives for robotic flight control, 2008. arXiv:cs/0609140v2.
3. J. Peters, and S. Schaal Reinforcement learning of motor skills with policy gradients Neural Networks 21 2008 682 697
4. J. Kober, and J. Peters Policy search for motor primitives in robotics Adv Neural Inf. Process Syst. vol. 22 2009
5. E. Theodorou, J. Buchli, S. Schaal, Reinforcement learning of motor skills in high dimensions: a path integral approach, in: Int. Conf. of Robotics and Automation, ICRA, 2010, 2010, pp. 23972403.
6. J. Peters, J. Mulling, J. Kober, D. Nguyen-Tuong, O. Kroemer, Towards motor skill learning for robotics, in: Proc. 14th Int. Symp. on Robotics Research, ISRR 2009, 2009, pp. 114.
7. J. Kober, E. Oztop, and J. Peters Reinforcement learning to adjust robot movements to new situations Robotics: Science and Systems Conf., RSS 2010 2010 MIT Press Cambridge, MA, USA 1 8
8. C.G. Atkeson, and S. Schaal Robot learning from demonstration Proc. 14th Int. Conf. Machine Learning ICML, 1997 1997 Morgan Kaufmann San Francisco, CA 12 20
9. R. Sutton, and A. Barto Reinforcement Learning: An Introduction 1998 MIT Press Cambridge, MA
10. S.I. Reynolds, The stability of general discounted reinforcement learning with linear function approximation, in: UK Workshop on Compu. Intell., UKCI-02, 2002, pp. 139146.
11. H. Hoffmann, Peter, D.-H. Park, S. Schaal, Biologically-inspired dynamical systems for movement generation: automatic real-time goal adaptation and obstacle avoidance, in: Proc. IEEE Int. Conf. Robotics and Automation, ICRA, 2009, Kobe, Japan, 2009, pp. 25872592.
12. A. Ijspeert, J. Nakanishi, P. Pastor, H. Hoffmann, S. Schaal, Learning nonlinear dynamical systems models, Neural Computation (2011) (submitted for publication).
13. J. Baxter, and P. Bartlet Infinite-horizon policygradient estimation Journal of Artificial Intelligence Research 15 2001 319 350
14. M. Tamosiunaite, T. Asfour, and F. Wörgötter Learning to reach by reinforcement learning using a receptive field based function approximation approach with continuous actions Biological Cybernetics 100 3 2009 249 260
15. M. Wiering Convergence and divergence in standard and averaging reinforcement learning Proc. 15th Eur. Conf. Machine Learning, ECML, 2004 2004 Springer-Verlag 477 488 (Pubitemid 41050115)
16. B. Nemec, M. Tamosiunaite, F. Wörgötter, A. Ude, Task adaptation through exploration and action sequencing, in: Proc. 9th IEEE-RAS Int. Conf. on Humanoid Robots, 2009, pp. 610616.
17. S. Schaal, J. Peters, J. Nakanishi, and A. Ijspeert Learning movement primitives Int. Symp. on Robotics Res. ISRR2003 2004 springer 1805
18. J. Kober, and J. Peters Practical algorithms for motor primitives in robotics IEEE Robotics and Automation Magazine 17 2 2010 55 62
19. J. Kolter, A. Ng, Policy search via the signed derivative, in: Robotics: Science and Systems, RSS, 2009.
20. J. Morimoto, and C. Atkeson Nonparametric representation of an approximated Poincare map for learning biped locomotion Autonomous Robots 27 2009 131 144
21. G. Endo, J. Morimoto, T. Matsubara, J. Nakanishi, and G. Cheng Learning CPG-based biped locomotion with a policy gradient method: application to a humanoid robot The International Journal of Robotics Research 27 2 2008 213 228 (Pubitemid 351169715)
22. J. Peters, S. Schaal, Reinforcement learning for parameterized motor primitives, in: Int. Joint Conf. on Neural Networks, 2006, pp. 7380. http://www-clmc.usc.edu/publications/P/peters-IJCNN2006.pdf.
23. T.H. Meyer, M. Eriksson, and R.C. Maggio Gradient estimation from irregularly spaced data sets Journal of Mathematical Geology 33 6 2001 693 717 (Pubitemid 33744627)
24. S. Calinon, F. Guenter, and A. Billard On learning, representing, and generalizing a task in a humanoid robot IEEE Transactions on Systems, Man, and Cybernetics 32 2 2007 286 298 (Pubitemid 46523219)
25. M. Hersch, F. Guenter, S. Calinon, and A. Billard Dynamical system modulation for robot learning via kinesthetic demonstrations IEEE Transactions on Robotics 24 6 2008 1463 1467
26. P. Pastor, H. Hoffmann, T. Asfour, S. Schaal, Learning and generalization of motor skills by learning from demonstration, in: Proc. IEEE Int. Conf. Robotics and Automation, ICRA, 2009, Kobe, Japan, 2009, pp. 763768.