Robot motor skill coordination with EM-based reinforcement learning

IEEE/RSJ 2010 International Conference on Intelligent Robots and Systems, IROS 2010 - Conference Proceedings

Volumn , Issue , 2010, Pages 3232-3237

  Kormushev, Petar ^a   Calinon, Sylvain ^a   Caldwell, Darwin G ^a  

^a ISTITUTO ITALIANO DI TECNOLOGIA   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

DYNAMIC MOVEMENT PRIMITIVES; EXPECTATION MAXIMIZATION; MOTOR CONTROL; MOTOR SKILLS; REACHING TASK;

DYNAMICAL SYSTEMS; REINFORCEMENT LEARNING;

INTELLIGENT ROBOTS;

EID: 78651490434     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IROS.2010.5649089     Document Type: Conference Paper

References (27)

1. A. Billard, S. Calinon, R. Dillmann, and S. Schaal, "Robot programming by demonstration," in Handbook of Robotics, B. Siciliano and O. Khatib, Eds. Secaucus, NJ, USA: Springer, 2008, pp. 1371-1394.
2. B. D. Argall, S. Chernova, M. Veloso, and B. Browning, "A survey of robot learning from demonstration," Robot. Auton. Syst., vol. 57, no. 5, pp. 469-483, 2009.
3. R. S. Sutton and A. G. Barto, Reinforcement learning : an introduction, ser. Adaptive computation and machine learning. Cambridge, MA, USA: MIT Press, 1998.
4. J.Peters and S.Schaal, "Policy gradient methods for robotics," in Proc. IEEE/RSJ Intl Conf. on Intelligent Robots and Systems (IROS), 2006.
5. E. Theodorou, J. Buchli, and S. Schaal, "Reinforcement learning of motor skills in high dimensions: a path integral approach," in Proc. IEEE Intl Conf. on Robotics and Automation (ICRA), 2010.
6. A. Coates, P. Abbeel, and A. Y. Ng, "Apprenticeship learning for helicopter control," Commun. ACM, vol. 52, no. 7, pp. 97-105, 2009.
7. F. Guenter, M. Hersch, S. Calinon, and A. Billard, "Reinforcement learning for imitating constrained reaching movements," Advanced Robotics, vol. 21, no. 13, pp. 1521-1544, 2007.
8. R. J. Williams, "Simple statistical gradient-following algorithms for connectionist reinforcement learning," Mach. Learn., vol. 8, no. 3-4, pp. 229-256, 1992.
9. J. Peters and S. Schaal, "Natural actor-critic," Neurocomput., vol. 71, no. 7-9, pp. 1180-1190, 2008.
10. J. Kober and J. Peters, "Learning motor primitives for robotics," in Proc. IEEE Intl Conf. on Robotics and Automation (ICRA), May 2009, pp. 2112-2118.
11. J. Kober, "Reinforcement learning for motor primitives," Master's thesis, University of Stuttgart, Germany, August 2008.
12. J. Kober and J. Peters, "Policy search for motor primitives in robotics," in Advances in Neural Information Processing Systems, 2009, vol. 21, pp. 849-856.
13. S. Schaal, P. Mohajerian, and A. J. Ijspeert, "Dynamics systems vs. optimal control a unifying view," Progress in Brain Research, vol. 165, pp. 425-445, 2007.
14. D. Pardo, "Learning rest-to-rest motor coordination in articulated mobile robots," PhD thesis, Technical University of Catalonia (UPC), 2009.
15. D. E. Pardo and C. Angulo, "Collaborative control in a humanoid dynamic task," in Proc. Intl Conf. on Informatics in Control, Automation and Robotics, Robotics and Automation (ICINCO), Angers, France, May 2007, pp. 174-180.
16. T. Flash and N. Hogan, "The coordination of the arm movements: an experimentally confirmed mathematical model," Neurology, vol. 5, no. 7, pp. 1688-1703, 1985.
17. E. Todorov and M. I. Jordan, "Optimal feedback control as a theory of motor coordination," Nature Neuroscience, vol. 5, pp. 1226-1235, 2002.
18. R. Huys, A. Daffertshofer, and P. J. Beek, "The evolution of coordination during skill acquisition: the dynamical systems approach," in Skill Acquisition in Sport: Research, Theory and Practice, A. M. Williams and N. J. Hodges, Eds. Routledge, 2004, pp. 351-373.
19. M. Bernikera, A. Jarcb, E. Bizzic, and M. C. Trescha, "Simplified and effective motor control based on muscle synergies to exploit musculoskeletal dynamics," in Proc. Natl Acad. Sci. USA, vol. 106, no. 18, 2009, pp. 7601-7606.
20. M. T. Rosenstein, A. G. Barto, and R. E. A. Van Emmerik, "Learning at the level of synergies for a robot weightlifter," Robotics and Autonomous Systems, vol. 54, no. 8, pp. 706-717, 2006.
21. S. Calinon, I. Sardellitti, and D. G. Caldwell, "Learning-based control strategy for safe human-robot interaction exploiting task and robot redundancies," in Proc. IEEE/RSJ Intl Conf. on Intelligent Robots and Systems (IROS), Taipei, Taiwan, October 2010.
22. A. J. Ijspeert, J. Nakanishi, and S. Schaal, "Trajectory formation for imitation with nonlinear dynamical systems," in Proc. IEEE Intl Conf. on Intelligent Robots and Systems (IROS), 2001, pp. 752-757.
23. H. Hoffmann, P. Pastor, D. H. Park, and S. Schaal, " Biologicallyinspired dynamical systems for movement generation: automatic real-time goal adaptation and obstacle avoidance," in Proc. IEEE Intl Conf. on Robotics and Automation (ICRA), 2009, pp. 2587-2592.
24. S. Calinon, F. D'halluin, D. G. Caldwell, and A. G. Billard, "Handling of multiple constraints and motion alternatives in a robot programming by demonstration framework," in Proc. IEEE-RAS Intl Conf. on Humanoid Robots (Humanoids), Paris, France, December 2009, pp. 582-588.
25. S. Calinon, F. D'halluin, E. L. Sauser, D. G. Caldwell, and A. G. Billard, "Learning and reproduction of gestures by imitation: An approach based on hidden Markov model and Gaussian mixture regression," IEEE Robotics and Automation Magazine, vol. 17, no. 2, pp. 44-54, 2010.
26. M. Khansari and A. G. Billard, "BM: An iterative method to learn stable non-linear dynamical systems with Gaussian mixture models," in Proc. IEEE Intl Conf. on Robotics and Automation (ICRA), Anchorage, Alaska, USA, May 2010, pp. 2381-2388.
27. R. Featherstone and D. E. Orin, "Dynamics," in Handbook of Robotics, B. Siciliano and O. O. Khatib, Eds. Secaucus, NJ, USA: Springer, 2008, pp. 35-65.