Reinforcement learning for imitating constrained reaching movements

Advanced Robotics

Volumn 21, Issue 13, 2007, Pages 1521-1544

  Guenter, Florent ^a   Hersch, Micha ^a   Calinon, Sylvain ^a   Billard, Aude ^a  

^a EPFL   (Switzerland)

Author keywords

DYNAMICAL SYSTEMS; GAUSSIAN MIXTURE MODEL; PROGRAMMING BY DEMONSTRATION; REINFORCEMENT LEARNING

Indexed keywords

ALGORITHMS; COMPUTER PROGRAMMING; DYNAMICAL SYSTEMS; ROBOTS;

CONSTRAINED REACHING MOVEMENTS; GAUSSIAN MIXTURE MODELS; PROGRAMMING BY DEMONSTRATION;

REINFORCEMENT LEARNING;

EID: 34948857495     PISSN: 01691864     EISSN: 15685535     Source Type: Journal    
DOI: 10.1163/156855307782148550     Document Type: Article

References (30)

1. Dautenhahn, K., and Nehaniv, C. L., 2001. Imitation in Animals and Artifacts, Cambridge, MA:MIT Press.
2. Billard, A., and Siegwart, R., 2004. Robotics and Autonomous Systems, Special Issue:Robot Learning From Demonstration, Amsterdam:Elsevier.
3. Schoner, G., Dose, M., and Engels, C., 1995. Dynamics of behaviour:theory and application for autonomous robot architecture. Robotics Autonomous Syst., 16:213–245.
4. Hersch, M., and Billard, A., 2006. “ A biologically-inspired model of reaching movements ”. In Proc. IEEE/RAS—EMBS Int. Conf. on Biomedical Robotics and Biomechatronics 1067–1071. Pisa
5. Iossifidis, I., and Schoner, G., 2004. “ Autonomous reaching and obstacle avoidance with anthropomorphic arm of a robotics assistant using the attractor dynamics approach ”. In Proc. IEEE Int. Conf. on Robotics and Automation 4295–4300. New Orleans, LA
6. Righetti, L., and Ijspeert, A., 2006. “ Programmable central pattern generators:a application to biped locomotion control ”. In Proc. IEEE Int. Conf. on Robotics and Automation 1585–1590. Orlando, FL
7. Atkeson, C. G., and Schaal, S., 1997. “ Learning tasks from a single demonstration ”. In Proc. IEEE Int. Conf. on Robotics and Automation 1706–1712. Albuquerque, NM
8. Schaal, S., Peters, J., Nakanishi, J., and Ijspeert, A., 2003. “ Learning movement primitives ”. In Proc. Int. Symp. on Robotics Research 1805–1815. Siena
9. Morimoto, J., and Doya, K., 2001. Acquisition of stand-up behavior by a real robot using hierarchical reinforcement learning. Robotics Autonomous Syst., 36:37–51.
10. Iida, S., Kanoh, M., Kato, S., and Itoh, H., 2004. “ Reinforcement learning for motion control of humanoid robots ”. In Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems 353–358. Sendai
11. Bratke, S. J., and Duff, M. O., 1994. “ Reinforcement learning methods for continuous-time Markov decision problems ”. In Proc. Neural Information Processing Systems Conf 393–400. Denver
12. Doya, K., 2000. Reinforcement learning in continuous time and space. Neural Comput., 12:219–245.
13. Sutton, R. S., and Barto, A. G., 1998. Reinforcement Learning:An Introduction, Cambridge, MA:MIT Press.
14. Konda, V. R., and Tsitsiklis, J. N., 2000. Actor—critic algorithms. Adv. Neural Information Process. Syst., 12:1008–1015.
15. Peters, J., Vijayakumar, S., and Schaal, S., 2003. “ Reinforcement learning for humanoid robotics ”. In Proc. IEEE-RAS Int. Conf. on Humanoid Robots (Humanoids2003) 225–230. München
16. Peters, J., Vijayakumar, S., and Schaal, S., 2005. “ Natural actor—critic ”. In Proc. 16th Eur. Conf. on Machine Learning 280–291. Porto
17. Nedic, A., and Bertsekas, D., 2001. “ Least-squares policy evaluation algorithms with linear function approximation ”. In LIDS Report LIDS-P-2537, Dec. 2001 Cambridge
18. Williams, R., 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learn., 8:229–256.
19. Sutton, R. S., McAllester, D., Singh, S., and Mansour, Y., 2000. Policy gradient methods for reinforcement learning with function approximation. Adv. Neural Information Process. Syst., 12:1057–1064.
20. Barto, A. G., and Mahadevan, S., 2003. Recent advances in hierarchical reinforcement learning. Discrete Event Dyn. Syst.:Theory Applic., 13:343–379.
21. Amari, S., 2000. Natural gradient works efficiently in learning. Neural Comput., 10:251–276.
22. Konidaris, G., and Barto, A., 2006. “ Autonomous shaping:knowledge transfer in reinforcement learning ”. In Proc. Int. Conf. on Machine Learning 497–504. Pittsburgh, PA
23. Abbeel, P., and Quigley, M., 2006. “ Using inaccurate models in reinforcement learning ”. In Proc. Int. Conf. on Machine Learning 9–16. Pittsburgh, PA
24. Simsek, O., and Barto, A., 2006. “ An intrinsic reward mechanism for efficient exploration ”. In Proc. Int. Conf. on Machine Learning 841–848. Pittsburgh, PA
25. Hersch, M., Guenter, F., Calinon, S., and Billard, A., 2006. “ Learning dynamical system modulation for constraint reaching tasks ”. In Proc. IEEE-RAS International Conference on Humanoid Robots 444–449. Genova
26. Calinon S., Guenter F., and Billard A., On learning, representing and generalizing a task in a humanoid robot, IEEE Trans. Syst. Man Cybernet. B (Special Issue on Robot Learning by Observation, Demonstration and Imitation) 37, in press (2007).
27. Ghahramani, Z., and Jordan, M. I., 1994. Supervised learning from incomplete data via an EM approach. Adv. Neural Information Process. Syst., 6:120–128.
28. Billard, A., Calinon, S., and Guenter, F., 2006. Discriminative and adaptative imitation in uni-manual and bi-manual tasks. Robotics and Autonomous Syst., 54:370–384.
29. Boyan, J. A., 2002. Technical update:least-squares temporal difference learning. Machine Learn., 49:233–246.
30. Bertsekas, D., and Tsitsiklis, J., 1996. Neuro-dynamic Programming, Belmont, MA:Athena Scientific.