Reinforcement learning to adjust parametrized motor primitives to new situations

Autonomous Robots

Volumn 33, Issue 4, 2012, Pages 361-379

  Kober, Jens ^a,b   Wilhelm, Andreas ^c   Oztop, Erhan ^d,e,f   Peters, Jan ^a,b  

^a MAX PLANCK INSTITUTE FOR INTELLIGENT SYSTEMS   (Germany)

^b DARMSTADT UNIVERSITY OF TECHNOLOGY   (Germany)

^c UNIVERSITY OF APPLIED SCIENCES RAVENSBURG WEINGARTEN   (Germany)

^d NATIONAL INSTITUTE OF INFORMATION AND COMMUNICATIONS TECHNOLOGY   (Japan)

^e ADVANCED TELECOMMUNICATIONS RESEARCH INSTITUTE INTERNATIONAL   (Japan)

^f OZYEGIN UNIVERSITY   (Turkey)

Author keywords

Meta parameters; Motor primitives; Policy learning; Reinforcement learning; Skill learning

Indexed keywords

CURRENT SITUATION; GLOBAL PARAMETERS; HITTING MOVEMENTS; META-PARAMETERS; MOTOR PRIMITIVES; PHYSICAL ROBOTS; ROBOT TASKS; SKILL LEARNING; TABLE-TENNIS;

LEARNING ALGORITHMS; ROBOTS;

REINFORCEMENT LEARNING;

EID: 84868358933     PISSN: 09295593     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10514-012-9290-3     Document Type: Article

References (44)

1. Barto, A., & Mahadevan, S. (2003). Recent advances in hierarchical reinforcement learning. Discrete Event Dynamic Systems, 13(4), 341-379.
2. Bays, P., & Wolpert, D. (2007). Computational principles of sensorimotor control that minimise uncertainty and variability. Journal of Physiology, 578, 387-396. (Pubitemid 46092204)
3. Bentivegna, D. C., Ude, A., Atkeson, C. G., & Cheng, G. (2004). Learning to act from observation and practice. International Journal of Humanoid Robotics, 1(4), 585-611.
4. Bishop, C. M. (2006). Pattern recognition and machine learning. Berlin: Springer.
5. Caruana, R. (1997). Multitask learning. Machine Learning, 28, 41-75. (Pubitemid 127507169)
6. Cheng, G., Hyon, S., Morimoto, J., Ude, A., Hale, J. G., Colvin, G., Scroggin,W., & Jacobsen, S. C. (2007). CB: A humanoid research platform for exploring neuroscience. Advanced Robotics, 21(10), 1097-1114. (Pubitemid 47186901)
7. Dayan, P., & Hinton, G. E. (1997). Using expectation-maximization for reinforcement learning. Neural Computation, 9(2), 271-278. (Pubitemid 127635391)
8. Doya, K. (2002). Metalearning and neuromodulation. Neural Networks, 15(4-6), 495-506. (Pubitemid 34947460)
9. Engel, Y.,Mannor, S., &Meir, R. (2005). Reinforcement learning with Gaussian processes. In Proc. int. conf. machine learning (pp. 201- 208). (Pubitemid 43183334)
10. Grimes, D. B.,& Rao, R. P. N. (2008). Learning nonparametric policies by imitation. In Proc. int. conf. intelligent robots and system (pp. 2022-2028).
11. Huber, M., & Grupen, R. (1998). Learning robot control - using control policies as abstract actions. In NIPS'98 workshop: abstraction and hierarchy in reinforcement learning.
12. Ijspeert, A. J., Nakanishi, J., & Schaal, S. (2002). Learning attractor landscapes for learning motor primitives. In Advances in neural information processing systems (Vol. 15, pp. 1523-1530).
13. Jaakkola, T., Jordan,M. I., Singh, S. P. (1993). Convergence of stochastic iterative dynamic programming algorithms. In Advances in neural information processing systems (Vol. 6, pp. 703-710).
14. Jetchev, N., & Toussaint, M. (2009). Trajectory prediction: learning to map situations to robot trajectories. In Proc. int. conf. machine learning (p. 57).
15. Kober, J., & Peters, J. (2011a). Learning elementary movements jointly with a higher level task. In Proc. IEEE/RSJ int. conf. intelligent robots and systems (pp. 338-343).
16. Kober, J., & Peters, J. (2011b). Policy search for motor primitives in robotics. Machine Learning, 84(1-2), 171-203.
17. Kober, J., Mülling, K., Krömer, O., Lampert, C. H., Schölkopf, B., & Peters, J. (2010a). Movement templates for learning of hitting and batting. In Proc. IEEE int. conf. robotics and automation (pp. 853-858).
18. Kober, J., Oztop, E., & Peters, J. (2010b). Reinforcement learning to adjust robot movements to new situations. In Proc. robotics: science and systems conf. (pp. 33-40).
19. Kronander, K., Khansari-Zadeh, M. S., & Billard, A. (2011). Learning to control planar hitting motions in a minigolf-like task. In Proc. IEEE/RSJ int. conf. intelligent robots and systems (pp. 710-717).
20. Lampariello, R., Nguyen-Tuong, D., Castellini, C., Hirzinger, G., & Peters, J. (2011). Trajectory planning for optimal robot catching in real-time. In Proc. IEEE int. conf. robotics and automation (pp. 3719-3726).
21. Lawrence, G., Cowan, N., & Russell, S. (2003). Efficient gradient estimation for motor control learning. In Proc. int. conf. uncertainty in artificial intelligence (pp. 354-361).
22. Lens, T., Kunz, J., Trommer, C., Karguth, A., & von Stryk, O. (2010). Biorob-arm: A quickly deployable and intrinsically safe, lightweight robot arm for service robotics applications. In 41st international symposium on robotics/6th German conference on robotics (pp. 905-910).
23. Masters Games Ltd (2010). The rules of darts. http://www.mastersgames. com/rules/darts-rules.htm.
24. McGovern, A., & Barto, A. G. (2001). Automatic discovery of subgoals in reinforcement learning using diverse density. In Proc. int. conf. machine learning (pp. 361-368).
25. McGovern, A., Sutton, R. S., & Fagg, A. H. (1997). Roles of macroactions in accelerating reinforcement learning. In Grace Hopper celebration of women in computing.
26. Mülling, K., Kober, J., & Peters, J. (2010). Learning table tennis with a mixture of motor primitives. In Proc. IEEE-RAS int. conf. humanoid robots (pp. 411-416).
27. Mülling, K., Kober, J., & Peters, J. (2011). A biomimetic approach to robot table tennis. Adaptive Behavior, 9(5), 359-376.
28. Nakanishi, J., Morimoto, J., Endo, G., Cheng, G., Schaal, S., & Kawato, M. (2004). Learning from demonstration and adaptation of biped locomotion. Robotics and Autonomous Systems, 47(2-3), 79-91.
29. Park, D. H., Hoffmann, H., Pastor, P., & Schaal, S. (2008). Movement reproduction and obstacle avoidance with dynamic movement primitives and potential fields. In Proc. IEEE-RAS int. conf. humanoid robots (pp. 91-98).
30. Pastor, P., Hoffmann, H., Asfour, T., & Schaal, S. (2009). Learning and generalization of motor skills by learning from demonstration. In Proc. IEEE int. conf. robotics and automation (pp. 1293-1298).
31. Peters, J., & Schaal, S. (2008a). Learning to control in operational space. The International Journal of Robotics Research, 27(2), 197-212. (Pubitemid 351169714)
32. Peters, J., & Schaal, S. (2008b). Reinforcement learning of motor skills with policy gradients. Neural Networks, 21(4), 682-697.
33. Pongas, D., Billard, A., & Schaal, S. (2005). Rapid synchronization and accurate phase-locking of rhythmic motor primitives. In Proc. IEEE/RSJ int. conf. intelligent robots and systems (pp. 2911- 2916).
34. Rasmussen, C. E., & Williams, C. K. (2006). Gaussian processes for machine learning. Cambridge: MIT Press.
35. Russell, S. (1998). Learning agents for uncertain environments (extended abstract). In Proc. eleventh annual conference on computational learning theory (pp. 101-103). New York: ACM.
36. Schaal, S., Mohajerian, P., & Ijspeert, A. J. (2007). Dynamics systems vs. optimal control - a unifying view. Progress in Brain Research, 165(1), 425-445. (Pubitemid 47513886)
37. Schmidt, R., & Wrisberg, C. (2000). Motor learning and performance (2nd edn.). Champaign: Human Kinetics.
38. Sutton, R., & Barto, A. (1998). Reinforcement learning. Cambridge: MIT Press.
39. Sutton, R. S., McAllester, D., Singh, S., & Mansour, Y. (1999). Policy gradient methods for reinforcement learning with function approximation. In Advances in neural information processing systems (Vol. 12, pp. 1057-1063).
40. Ude, A., Gams, A., Asfour, T., & Morimoto, J. (2010). Task-specific generalization of discrete and periodic dynamic movement primitives. IEEE Transactions on Robotics, 26(5), 800-815.
41. Urbanek, H., Albu-Schäffer, A., & van der Smagt, P. (2004). Learning from demonstration repetitive movements for autonomous service robotics. In Proc. IEEE/RSJ int. conf. intelligent robots and systems (pp. 3495-3500). (Pubitemid 40276133)
42. Welling, M. (2010). The Kalman filter. Lecture notes.
43. Williams, R. J. (1992). Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8, 229-256.
44. Wulf, G. (2007). Attention and motor skill learning. Champaign: Human Kinetics.