Dynamical movement primitives: Learning attractor models formotor behaviors

Neural Computation

Volumn 25, Issue 2, 2013, Pages 328-373

  Ijspeert, Auke Jan ^a   Nakanishi, Jun ^b   Hoffmann, Heiko ^c   Pastor, Peter ^c   Schaal, Stefan ^c,d,e  

^a EPFL   (Switzerland)

^b UNIVERSITY OF EDINBURGH   (United Kingdom)

^c UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^d MAX PLANCK INSTITUTE FOR INTELLIGENT SYSTEMS   (Germany)

^e ADVANCED TELECOMMUNICATIONS RESEARCH INSTITUTE INTERNATIONAL   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ANIMAL; ARTIFICIAL INTELLIGENCE; HUMAN; LETTER; MOVEMENT (PHYSIOLOGY); NONLINEAR SYSTEM; ROBOTICS; THEORETICAL MODEL;

ANIMALS; ARTIFICIAL INTELLIGENCE; HUMANS; MODELS, THEORETICAL; MOVEMENT; NONLINEAR DYNAMICS; ROBOTICS;

EID: 84875592161     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/NECO_a_00393     Document Type: Letter

References (94)

1. Atkeson, C. G., Hale, J., Kawato, M., Kotosaka, S., Pollick, F., Riley, M., et al. (2000). Using humanoid robots to study human behaviour. IEEE Intelligent Systems, 15, 46-56.
2. Bernstein, N. A. (1967). The control and regulation of movements. London: Pergamon Press.
3. Billard, A., Calinon, S., Dillmann, R., & Schaal, S. (2008). Robot programming by demonstration. In B. Siciliano&O.Khatib (Eds.), Handbook of robotics. Cambridge, MA: MIT Press.
4. Billard, A., & Mataric, M. (2001). Learning human arm movements by imitation: Evaluation of a biologically-inspired architecture. Robotics and Autonomous Systems, 941, 1-16.
5. Bishop, C. M. (2006). Pattern recognition and machine learning. NewYork: Springer.
6. Buchli, J., Righetti, L., & Ijspeert, A. J. (2006). Engineering entrainment and adaptation in limit cycle systems-frombiological inspiration to applications in robotics. Biological Cybernetics, 95(6), 645-664.
7. Bühler, M., & Koditschek, D. E. (1990). From stable to chaotic juggling: Theory, simulation, and experiments. In Proceedings of the IEEE International Conference on Robotics and Automation (pp. 845-865). Piscataway, NJ: IEEE.
8. Bullock, D., & Grossberg, S. (1989). VITE and FLETE: Neural modules for trajectory formation and postural control. In W. A. Hersberger (Ed.), Volitional control (pp. 253-297). New York: Elsevier.
9. Burridge, R. R., Rizzi, A. A., & Koditschek, D. E. (1999). Sequential composition of dynamically dexterous robot behaviors. International Journal of Robotics Research, 18(6), 534-555.
10. Chevallereau, C., Westervelt, E. R., & Grizzle, J. W. (2005). Asymptotically stable running for a five-link, four-actuator, planar, bipedal robot. International Journal of Robotics Research, 24(6), 431-464.
11. Dayan, P., & Abbott, L. F. (2001). Theoretical neuroscience. Cambridge, MA:MIT Press.
12. Dijkstra, T. M., Schoner, G., Giese, M. A., & Gielen, C. C. (1994). Frequency dependence of the action-perception cycle for postural control in a moving visual environment: Relative phase dynamics. Biol Cybern, 71(6), 489-501.
13. Fajen, B. R., & Warren, W. H. (2003). Behavioral dynamics of steering, obstacle avoidance, and route selection. J. Exp. Psychol. Hum. Percept. Perform., 29(2), 343- 362.
14. Flash, T., & Hogan, N. (1985). The coordination of arm movements: An experimentally confirmed mathematical model. Journal of Neuroscience, 5(7), 1688- 1703.
15. Flash, T.,&Sejnowski, T. (2001).Computational approaches to motor control. Current Opinion in Neurobiology, 11, 655-662.
16. Friedland, B. (1986). Control system design: An introduction to state-space methods. New York: McGraw-Hill.
17. Gams, A., Ijspeert, A., Schaal, S., & Lenarcic, J. (2009). On-line learning and modulation of periodic movements with nonlinear dynamical systems. Autonomous Robots, 27(1), 3-23.
18. Getting, P.A. (1988).Comparative analysis of invertebrate central pattern generators. In A. H. Cohen, S. Rossignol, & S. Grillner (Eds.), Neural control of rhythmic movements in vertebrates (pp. 101-127). New York:Wiley.
19. Giszter, S. F., Mussa-Ivaldi, F. A.,&Bizzi, E. (1993).Convergent force fields organized in the frog's spinal cord. Journal of Neuroscience, 13, 467-491.
20. Gomi, H., & Kawato, M. (1996). Equilibrium-point control hypothesis examined by measured arm stiffness during multijoint movement. Science, 272, 117-220.
21. Gomi, H., & Kawato, M. (1997). Human arm stiffness and equilibrium-point trajectory during multi-joint movement. Biol. Cybern., 76(3), 163-171.
22. Gribovskaya, E., Khansari-Zadeh, M., & Billard, A. (2010). Learning nonlinear multivariate dynamics of motion in robotic manipulators. International Journal of Robotics Research, 30(1), 80-117.
23. Grillner, S. (1981). Control of locomotion in bipeds, tetrapods and fish. In V. B. Brooks (Ed.), Handbook of physiology: The nervous system, vol. 2: Motor control (pp. 1179-1236). Bethesda, MD: American Physiology Society.
24. Guckenheimer, J., & Holmes, P. (1983). Nonlinear oscillators, dynamical systems, and bifurcations of vector fields. NewYork: Springer.
25. Haken, H., Kelso, J. A. S., Fuchs, A., & Pandya, A. S. (1990). Dynamic pattern recognition of coordinated biological movement. Neural Networks, 3, 395-401.
26. Hatsopoulos, N. G., & Warren, W. H. J. (1996). Resonance tuning in rhythmic arm movements. Journal of Motor Behavior, 28(1), 3-14.
27. Hoffmann, H., Pastor, P., Park, D.-H., & Schaal, S. (2009). Biologically-inspired dynamical systems for movement generation: Automatic real-time goal adaptation and obstacle avoidance. In International Conference on Robotics and Automation (pp. 2587-2592). Piscataway, NJ: IEEE.
28. Hollerbach, J. M. (1984). Dynamic scaling of manipulator trajectories. Transactions of the ASME, 106, 139-156.
29. Ijspeert, A. J. (2001). A connectionist central pattern generator for the aquatic and terrestrial gaits of a simulated salamander. Biological Cybernetics, 84(5), 331-348.
30. Ijspeert, A. J. (2008). Central pattern generators for locomotion control in animals and robots: A review. Neural Netw., 21(4), 642-653.
31. Ijspeert, A. J., Hallam, J., & Willshaw, D. (1999). Evolving swimming controllers for a simulated lamprey with inspiration from neurobiology. Adaptive Behavior, 7(2), 151-172.
32. Ijspeert, A. J., Nakanishi, J., & Schaal, S. (2002a). Learning rhythmic movements by demonstration using nonlinear oscillators. In Proceedings of the IEEE/RSJ Int. Conference on Intelligent Robots and Systems (pp. 958-963). Piscataway, NJ: IEEE.
33. Ijspeert, A. J., Nakanishi, J., & Schaal, S. (2002b). Movement imitation with nonlinear dynamical systems in humanoid robots. In IEEE International Conference on Robotics and Automation (pp. 1398-1403). Piscataway, NJ: IEEE.
34. Ijspeert, A. J., Nakanishi, J., & Schaal, S. (2003). Learning control policies for movement imitation and movement recognition. In S. T. S. Becker & K. Obermayer(Eds.), Advances in neural information processing systems, 15 (pp. 1547-1554). Cambridge, MA: MIT Press.
35. Jackson, E. A. (1989). Perspectives of nonlinear dynamics. Cambridge: Cambridge University Press.
36. Jaeger, H., & Haas, H. (2004). Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science, 304(5667), 78-80.
37. Joshi, P., & Maass,W. (2005).Movement generation with circuits of spiking neurons. Neural Computation, 17(8), 1715-1738.
38. Kawato, M. (1996). Trajectory formation in armmovements:Minimization principles and procedures. In H. N. Zelaznik (Ed.), Advances in motor learning and control (pp. 225-259). Champaign, IL: Human Kinetics.
39. Kelso, J. A. S. (1995). Dynamic patterns: The self-organization of brain and behavior. Cambridge, MA: MIT Press.
40. Kelso, J. A. S., Scholtz, J. P.,&Schoner, G. (1988).Dynamics governs switching among patterns of coordination in biological movement. Physics Letters A, 134(1), 8-12.
41. Khansari-Zadeh, S.M.,&Billard, A. (2010). Imitation learning of globally stable nonlinear point-to-point robot motions using nonlinear programming. In Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 2676-2683). Piscataway, NJ: IEEE.
42. Khatib, O. (1986). Real-time obstacle avoidance for manipulators and mobile robots. International Journal of Robotics Research, 5(1), 90-98.
43. Klavins, E.,&Koditschek,D. (2001). Stability of coupled hybrid oscillators. In Proceedings of the IEEE International Conference on Robotics and Automation (pp. 4200-4207). Piscataway, NJ: IEEE.
44. Kober, J., & Peters, J. (2009). Learning motor primitives for robotics. In Proceedings of the IEEE International Conference on Robotics and Automation (pp. 2112-2118). Piscataway, NJ: IEEE.
45. Koditschek, D. E. (1987). Exact robot navigation by means of potential functions: Some topological considerations. In Proceedings of the IEEE International Conference on Robotics and Automation (pp. 211-223). Piscataway, NJ: IEEE.
46. Kugler, P. N., & Turvey, M. T. (1987). Information, natural law, and the self-assembly of rhythmic movement. Hillsdale, NJ: Erlbaum.
47. Kulvicius, T., Ning, K., Tamosiunaite, M., & Worgötter, F. (2012). Joining movement sequences: Modified dynamic movement primitives for robotics applications exemplified on handwriting. IEEE Transactions on Robotics, 28(1), 145-157.
48. Latash, M. L. (1993). Control of human movement. Champaign, IL: Human Kinetics. Li, P., & Horowitz, R. (1999). Passive velocity field control of mechanical manipulators. IEEE Transactions on Robotics and Automation, 15(4), 751-763.
49. Li, P., & Horowitz, R. (1999). Passive velocity field control of mechanical manipulators. IEEE Transactions on Robotics and Automation, 15(4), 751-763.
50. Lohmiller, W., & Slotine, J. J. (1998). On contraction analysis for nonlinear systems. Automatica, 34(6), 683-696.
51. Maass, W., Natschläger, T., & Markram, H. (2002). Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation, 14(11), 2531-2560.
52. Matthews, P. C., Mirollo, R. E., & Strogatz, S. H. (1991). Dynamics of a large system of coupled nonlinear oscillators. Physica D, 52, 293-331.
53. McCrea, D. A., & Rybak, I. A. (2008). Organization ofmammalian locomotor rhythm and pattern generation. Brain Research Reviews, 57, 134-146.
54. Miyamoto, H., Schaal, S., Gandolfo, F., Koike, Y., Osu, R., Nakano, E., et al. (1996). A kendama learning robot based on bi-directional theory. Neural Networks, 9, 1281-1302.
55. Mussa-Ivaldi, F. A. (1997). Nonlinear force fields: a distributed system of control primitives for representing and learning movements. In Proceedings of the IEEE International Symposium on Computational Intelligence in Robotics and Automation (pp. 84-90). San Mateo, CA: IEEE Computer Society.
56. Mussa-Ivaldi, F. A. (1999). Modular features of motor control and learning. Current Opinion in Neurobiology, 9(6), 713-717.
57. Nakanishi, J., Morimoto, J., Endo, G., Cheng, G., Schaal, S., & Kawato, M. (2004). Learning from demonstration and adaptation of biped locomotion. Robotics and Autonomous Systems, 47, 79-91.
58. Okada, M., Tatani, K., & Nakamura, Y. (2002). Polynomial design of the nonlinear dynamics for the brain-like information processing of whole body motion. In Proceedings of the IEEE International Conference on Robotics and Automation (pp. 1410-1415). Piscataway, NJ: IEEE.
59. Paine, R. W., & Tani, J. (2004). Motor primitive and sequence self-organization in a hierarchical recurrent neural network. Neural Networks, 17(8-9), 1291-1309.
60. Pastor, P., Hoffmann, H., Asfour, T., & Schaal, S. (2009). Learning and generalization of motor skills by learning from demonstration. In International Conference on Robotics and Automation (pp. 763-768). Piscataway, NJ: IEEE.
61. Perk, B. E., & Slotine, J. J. E. (2006). Motion primitives for robotic flight control. arXiv:cs/0609140v2 [cs.RO].
62. Peters, J., & Schaal, S. (2008). Reinforcement learning of motor skills with policy gradients. Neural Networks, 21(4), 682-697.
63. Pongas, D., Billard, A., & Schaal, S. (2005). Rapid synchronization and accurate phase-locking of rhythmic motor primitives. In Proceedings of the IEEEInternational Conference on Intelligent Robots and Systems (pp. 2911-2916). Piscataway, NJ: IEEE.
64. Righetti, L., Buchli, J., & Ijspeert, A. J. (2006). Dynamic Hebbian learning in adaptive frequency oscillators. Physica D, 216(2), 269-281.
65. Rimon, E., & Koditschek, D. (1992). Exact robot navigation using artificial potential functions. IEEE Transactions on Robotics and Automation, 8(5), 501-518.
66. Rizzi, A. A., & Koditschek, D. E. (1994). Further progress in robot juggling: Solvable mirror laws. In Proceedings of the IEEE International Conference on Robotics and Automation (pp. 2935-2940). Piscataway, NJ: IEEE.
67. Rizzolatti, G., & Arbib, M. A. (1998). Language within our grasp. Trends Neurosci., 21(5), 188-194.
68. Sakoe, H., & Chiba, S. (1987). Dynamic programming algorithm optimization for spoken word recognition. IEEE Transactions on Acoustics, Speech and Signal Processing, 26(1), 43-49.
69. Schaal, S. (1999). Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences, 3(6), 233-242.
70. Schaal, S., & Atkeson, C. G. (1994). Assessing the quality of learned local models. In J. Cowan, G. Tesauro, & J. Alspector (Eds.), Advances in neural information processing systems, 6 (pp. 160-167). San Mateo, CA: Morgan Kaufmann.
71. Schaal, S.,&Atkeson, C. G. (1998). Constructive incremental learning from only local information. Neural Computation, 10(8), 2047-2084.
72. Schaal, S., Ijspeert, A., & Billard, A. (2003). Computational approaches to motor learning by imitation. Philosophical Transactions of the Royal Society of London: Series B, Biological Sciences, 358(1431), 537-547.
73. Schaal, S., Mohajerian, P., & Ijspeert, A. (2007). Dynamics systems vs. optimal control-a unifying view. Prog. Brain Res., 165, 425-445.
74. Schaal, S., & Sternad, D. (1998). Programmable pattern generators. In Proceedings of the International Conference onComputational Intelligence in Neuroscience (pp. 48-51). Piscataway, NJ: IEEE.
75. Schaal, S., Sternad, D., Osu, R., & Kawato, M. (2004). Rhythmic movement is not discrete. Nature Neuroscience, 7(10), 1137-1144.
76. Schöner,G. (1990).Adynamic theory of coordination of discrete movement. Biological Cybernetics, 63, 257-270.
77. Schöner, G., & Kelso, J. A. S. (1988). Dynamic pattern generation in behavioral and neural systems. Science, 239, 1513-1520.
78. Schöner, G., & Santos, C. (2001). Control of movement time and sequential action through attractor dynamics: A simulation study demonstrating object interception and coordination. In Proceedings of the 9th International Symposium on Intelligent Robotic Systems. http://spiderman-2.laas.fr/sirs2001/proceedings/
79. Sciavicco, L., & Siciliano, B. (2000). Modelling and control of robot manipulators. New York: Springer.
80. Scott, A. (2005). Encyclopedia of nonlinear science. New York: Routledge.
81. Slotine, J. J. E., & Li, W. (1991). Applied nonlinear control. Upper Saddle River, NJ: Prentice Hall.
82. Sternad, D., Amazeen, E., & Turvey, M. (1996). Diffusive, synaptic, and synergetic coupling: An evaluation through inphase and antiphase rhythmic movements. Journal of Motor Behavior, 28, 255-269.
83. Strogatz, S.H. (1994). Nonlinear dynamics and chaos:With applications to physics, biology, chemistry, and engineering. Reading, MA: Addison-Wesley.
84. Swinnen, S. P., Li, Y., Dounskaia, N., Byblow,W., Stinear, C., &Wagemans, J. (2004). Perception-action coupling during bimanual coordination: The role of visual perception in the coalition of constraints that govern bimanual action. J. Mot. Behav., 36(4), 394-398, 402-407 (discussion 408-417).
85. Taga, G., Yamaguchi, Y.,&Shimizu, H. (1991). Self-organized control of bipedal locomotion by neural oscillators in unpredictable environment. Biological Cybernetics, 65, 147-159.
86. Thelen, E., & Smith, L. B. (1994). A dynamical systems approach to the development of cognition and action. Cambridge, MA: MIT Press.
87. Theodorou, E., Buchli, J., & Schaal, S. (2010). Reinforcement learning in high dimensional state spaces: A path integral approach. Journal ofMachine Learning Research, 2010(11), 3137-3181.
88. Todorov, E. (2004). Optimality principles in sensorimotor control. Nature Neuroscience, 7, 907-915.
89. Tsuji, T., Tanaka, Y.,Morasso, P. G., Sanguineti, V., & Kaneko, M. (2002). Bio-mimetic trajectory generation of robots via artificial potential field with time base generator. IEEE Transactions on Systems, Man, and Cybernetics-Part C, 32(4), 426- 439.
90. Turvey, M. T. (1990). Coordination. Am. Psychol., 45(8), 938-953.
91. Ude, A., Gams, A., Asfour, T., & Morimoto, J. (2010). Task-specific generalization of discrete and periodic dynamic movement primitives. IEEE Transactions on Robotics, 26(5), 800-815.
92. Wada, Y., & Kawato, M. (2004). A via-point time optimization algorithm for complex sequential trajectory formation. Neural Networks, 17(3), 353-364.
93. Wolpert, D. M. (1997). Computational approaches to motor control. Trends Cogn. Sci., 1(6), 209-216.
94. Wyffels, F., & Schrauwen, B. (2009). Design of a central pattern generator using reservoir computing for learning human motion. In ATEQUAL 2009: 2009 ECSIS Symposium on Advanced Technologies for Enhanced Quality of Life (LABRS and ARTIPED 2009): Proceedings (pp. 118-122). Los Alamitos, CA: IEEE Computer Society.