Computational approaches to motor learning by imitation

Philosophical Transactions of the Royal Society B: Biological Sciences

Volumn 358, Issue 1431, 2003, Pages 537-547

  Schaal, Stefan ^a,b   Ijspeert, Auke ^a,c   Billard, Aude ^a,c  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^b ADVANCED TELECOMMUNICATIONS RESEARCH INSTITUTE INTERNATIONAL   (Japan)

^c EPFL   (Switzerland)

Author keywords

Duality of movement generation and movement recognition; Imitation; Motor control; Motor primitives

Indexed keywords

IMITATION; LEARNING; MOVEMENT;

EID: 0037471828     PISSN: 09628436     EISSN: None     Source Type: Journal    
DOI: 10.1098/rstb.2002.1258     Document Type: Article

References (81)

1. Aboaf, E. W., Drucker, S. M. & Atkeson, C. G. 1989 Task-level robot learing: juggling a tennis ball more accurately. In Proc. IEEE Int. Conf. Robotics Automation, Scottsdale, AZ, 14-19 May, pp. 331-348. Piscataway, NJ: IEEE.
2. Alissandrakis, A., Nehaniv, C. L. & Dautenhahn, K. 2002 Imitating with ALICE: learning to imitate corresponding actions across dissimilar embodiments. IEEE Transact. Systems, Man & Cybernetics, Part A: Systems Hum. 32, 482-496.
3. Arbib, M. A. 1981 Perceptual structures and distributed motor control. In Handbook of physiology, section 2: The nervous system, vol. II, Motor control, part 1 (ed. V. B. Brooks), pp. 1449-1480. Bethesda, MD: American Physiological Society.
4. Arbib, M. A., Billard, A., Iacoboni, M. & Oztop, E. 2000 Synthetic brain imaging: grasping, mirror neurons and imitation. Neural Networks 13, 975-997.
5. Arulampalam, S., Maskell, S., Gordon, N. & Clapp, T. 2002 A tutorial on particle filters for on-line non-linear/non-Gaussian Bayesian tracking. IEEE Trans. Signal Processing 50, 174-188.
6. Atkeson, C. G. & Schaal, S. 1997a Learning tasks from a single demonstration. In IEEE Int. Conf. Robotics Automation (ICRA97), Albuquerque, NM, 20-25 April 1997, vol. 2, pp. 1706-1712. Piscataway, NJ: IEEE.
7. Atkeson, C. G. & Schaal, S. 1997b Robot learning from demonstration. In Machine learning: Proc. 14th Int. Conf. (ICML '97), Nashville, TN, 8-12 July 1997 (ed. D. H. Fisher Jr), pp. 12-20. San Mateo, CA: Morgan Kaufmann.
8. Baillieul, J. & Martin, D. P. 1990 Resolution of kinematic redundancy. In Proc. Symp. Appl. Math., San Diego, CA, May 1990, vol. 41, pp. 49-89. Providence, RI: American Mathematical Society.
9. Billard, A. 2000 Learning motor skills by imitation: a biologically inspired robotic model. Cybern. Systems 32, 155-193.
10. Billard, A. & Mataric, M. 2001 Learning human arm movements by imitation: evaluation of a biologically-inspired architecture. Robotics Autonomous Systems 941, 1-16.
11. Billard, A. & Schaal, S. 2001 A connectionist model for online robot learning by imitation. In IEEE Int. Conf. Intell. Robots Systems (IROS 2001), Maui, Hawaii, 29 October-3 November 2001.
12. Billard, A. & Schaal, S. 2002 Computational elements of robot learning by imitation. In Am. Math. Soc. Central Section Meeting, Madison, WI, 12-13 October 2002. Providence, RI: American Mathematical Society.
13. Bullock, D., Grossberg, S. & Guenther, F. H. 1993 A self-organizing neural model of motor equivalent reaching and tool use by a multijoint arm. J. Cogn. Neurosci. 5, 408-435.
14. Byrne, R. W. 2003 Imitation as behavior parsing. Phil. Trans. R. Soc. Lond. B 358, 529-536. (DOI 10.1098/rstb.2002. 1219.)
15. Byrne, R. W. & Russon, A. E. 1998 Learning by imitation: a hierarchical approach. Behav. Brain Sci. 21, 667-721.
16. Cacciatore, T. W. & Nowlan, S. J. 1994 Mixtures of controllers for jump linear and non-linear plants. In Advances in neural information processing systems 6 (ed. J. D. Cowen, G. Tesauro & J. Alspector), pp. 719-726. San Mateo, CA: Morgan Kaufmann.
17. Craig, J. J. 1986 Introduction to robotics. Reading, MA: Addison-Wesley.
18. D'Souza, A., Vijayakumar, S. & Schaal, S. 2001 Learning inverse kinematics. In IEEE Int. Conf. Intell. Robots Systems (IROS 2001), Hilton Head Island, SC, 13-15 June 2000.
19. Dautenhahn, K. 1995 Getting to know each other - artificial social intelligence for autonomous robots. Robotics Autonomous Systems 16, 333-356.
20. Dautenhahn, K. & Nehaniv, C. L. (eds) 2002 Imitation in animals and artifacts. Cambridge, MA: MIT Press.
21. Dayan, P., Hinton, G. E., Neal, R. M. & Zemel, R. S. 1995 The Helmholtz machine. Neural Comput. 7, 889-904.
22. Demiris, J. & Hayes, G. 2002 Imitation as a dual-route process featuring predictive and learning components: a biologically plausible computational model. In Imitation in animals and artifacts (ed. K. Dautenhahn & C. L. Nehaniv), pp. 327-361. Cambridge, MA: MIT Press.
23. Deutscher, J., Blake, A. & Reid, I. 2000 Articulated body motion capture by annealed particle filtering. In IEEE Comput. Vision Pattern Recognition (CVPR 2000). Piscataway, NJ: IEEE.
24. Dillmann, R., Kaiser, M. & Ude, A. 1995 Acquisition of elementary robot skills from human demonstration. In Int. Symp. Intell. Robotic Systems (SIRS'95), 10-14 July 1999, Pisa, Italy, pp. 1-38.
25. Dufay, B. & Latombe, J. C. 1984 An approach to automatic robot programming based on inductive learning. Int. J. Robot. Res. 3, 3-20.
26. Dyer, P. & McReynolds, S. R. 1970 The computation and theory of optimal control. New York: Academic Press.
27. Fagg, A. H. & Arbib, M. A. 1998 Modeling parietal-premotor interactions in primate control of grasping. Neural Networks 11, 1277-1303.
28. Gallese, V. & Goldman, A. 1998 Mirror neurons and the simulation theory of mind-reading. Trends Cogn. Sci. 2, 493-501.
29. Grudic, G. Z. & Lawrence, P. D. 1996, Human-to-robot skill transfer using the SPORE approximation. In Int. Conf. Robotics Automation, Minneapolis, MN, April 1996, pp. 2962-2967. Piscataway, NJ: IEEE.
30. Guenther, F. H. & Barreca, D. M. 1997 Neural models for flexible control of redundant systems. In Self-organization, computational maps, and motor control (ed. P. Morasso & V. Sanguineti), pp. 102-108. Amsterdam: Elsevier.
31. Hatsopoulos, N. G. 1996 Coupling the neural and physical dynamics in rhythmic movements. Neural Comput. 8, 567-581.
32. Hayes, G. & Demiris, J. 1994 A robot controller using learning by imitation. In Proc. 2nd Int. Symp. Intell. Robotic Systems, Grenoble, France, July 1994 (ed. A. Borkowski & J. L. Crowley), pp. 198-204. Grenoble, France: LIFTA-IMAG.
33. Hovland, G. E., Sikka, P. & McCarragher, B. J. 1996, Skill acquisition from human demonstration using a hidden Markov Model. In IEEE Int. Conf. Robotics Automation, Minneapolis, MN, April 1996, pp. 2706-2711. Piscataway, NJ: IEEE.
34. Ijspeert, J. A., Nakanishi, J. & Schaal, S. 2002a Learning rhythmic movements by demonstration using nonlinear oscillators. In IEEE Int. Conf. Intell. Robots Systems (IROS 2002), Lausanne, 30 September-4 October 2002. Piscataway, NJ: IEEE.
35. Ijspeert, J. A., Nakanishi, J. & Schaal, S. 2002b Movement imitation with nonlinear dynamical systems in humanoid robots. In Int. Conf. Robotics Automation (ICRA2002), Washington, DC, 11-15 May 2002. Piscataway, NJ: IEEE.
36. Ikeuchi, K., Kawade, M. & Suehiro, T. 1993 Assembly task recognition with planar, curved and mechanical contacts. In Proc. IEEE Int. Conf. Robotics Automation, Atlanta, GA, May 1993, vol. 2, pp. 688-693. Piscataway, NJ: IEEE.
37. Kawamura, S. & Fukao, N. 1994 Interpolation for input torque patterns obtained through learning control. In Int. Conf. Automation, Robotics Computer Vis. (ICARCV '94), Singapore, November 1994, pp. 183-191.
38. Kawato, M. 1996 Bi-directional theory approach to integration. In Attention and performance XVI (ed. J. Konczak & E. Thelen), pp. 335-367. Cambridge, MA: MIT Press.
39. Kawato, M. 1999 Internal models for motor control and trajectory planning. Curr. Opin. Neurobiol. 9, 718-727.
40. Kuniyoshi, Y., Inaba, M. & Inoue, H. 1989 Teaching by showing: generating robot programs by visual observation of human performance. In Proc. Int. Symp. Industrial Robots, Tokyo, Japan, 4-6 October 1989, pp. 119-126.
41. Kuniyoshi, Y., Inaba, M. & Inoue, H. 1994 Learning by watching: extracting reusable task knowledge from visual observation of human performance. IEEE Trans. Robotics Automation 10, 799-822.
42. Lacquaniti, F. & Soechting, J. F. 1986 Simulation studies on the control of posture and movement in a multi-jointed limb. Biol. Cybern. 54, 367-378.
43. Levas, A. & Selfridge, M. 1984 A user-friendly high-level robot teaching system. In Int. Conf. Robotics, Atlanta, GA, March 1984, pp. 413-416. Piscataway, NJ: IEEE.
44. Lin, L.-J. 1991 Programming robots using reinforcement learning and teaching. In Proc. 9th Natl Conf. Artificial Intell., Anaheim, CA, 14-19 July 1991, vol. 2, pp. 781-786. Menlo Park, CA: AAAI.
45. Lozano-Pérez, T. 1982 Task-Planning. In Robot motion: planning and control (ed. M. Brady, J. M. Hollerbach, T. L. Johnson, T. Lozano-Pérez & M. T. Mason), pp. 473-498. Cambridge, MA: MIT Press.
46. Mataric, M. J. & Pomplun, M. 1998 Fixation behavior in observation and imitation of human movement. Cogn. Brain Res. 7, 191-202.
47. Meltzoff, A. N. & Moore, M. K. 1994 Imitation, memory, and the representation of persons. Infant Behav. Dev. 17, 83-99.
48. Meltzoff, A. N. & Moore, M. K. 1997 Explaining facial imitation: a theoretical model. Early Dev. Parenting 6, 179-192.
49. Miall, R. C. & Wolpert, D. M. 1996 Forward models for physiological motor control. Neural Networks 9, 1265-1285.
50. Miyamoto, H. & Kawato, M. 1998 A tennis serve and upswing learning robot based on bi-directional theory. Neural Networks 11, 1331-1344.
51. Miyamoto, H., Schaal, S., Gandolfo, F., Koike, Y., Osu, R., Nakano, E., Wada, Y. & Kawato, M. 1996 A Kendama learning robot based on bi-directional theory. Neural Networks 9, 1281-1302.
52. Nechyba, M. C. & Xu, Y. 1995 Human skill transfer: neural networks as learners and teachers. In IEEE/RSJ Int. Conf. Intell. Robots Systems, Pittsburgh, PA, 5-9 August 1995, vol. 3, pp. 314-319. Piscataway, NJ: IEEE.
53. Nehaniv, C. L. & Dautenhahn, K. 1999 Of hummingbirds and helicopters: an algebraic framework for interdisciplinary studies of imitation and its applications. In Learning robots: an interdisciplinary approach (ed. J. Demiris & A. Birk). Singapore: World Scientific.
54. Oztop, E. & Arbib, M. A. 2002 Schema design and implementation of the grasp-related mirror neuron system. Biol. Cybern. 87, 116-140.
55. Pawelzik, K., Kohlmorgen, J. & Müller, K. R. 1996 Annealed competition of experts for a segmentation and classification of switching dynamics. Neural Comput. 8, 340-356.
56. Piaget, J. 1951 Play, dreams, and imitation in childhood. New York: Norton.
57. Pook, P. K. & Ballard, D. H. 1993 Recognizing teleoperated manipulations. In Proc. IEEE Int. Conf. Robotics Automation, Atlanta, GA, May 1993, vol. 3, pp. 913-918. Piscataway, NJ: IEEE.
58. Rittscher, J. & Blake, A. 2003 Mathematical modelling of animate and intentional motion. Phil. Trans. R. Soc. Lond. B 358, 475-490. (DOI 10.1098/rstb.2002.1259.)
59. Rizzolatti, G. & Arbib, M. A. 1998 Language within our grasp. Trends Neurosci. 21, 188-194.
60. Rizzolatti, G., Fadiga, L., Gallese, V. & Fogassi, L. 1996 Premotor cortex and the recognition of motor actions. Cogn. Brain Res. 3, 131-141.
61. Saltzman, E. & Kelso, S. J. A. 1987 Skilled actions: a task-dynamic approach. Psychol. Rev. 94, 84-106.
62. Sammut, C., Hurst, S., Kedzier, D. & Michie, D. 1992 Learning to fly. In Proc. 9th Int. Machine Learning Conf. (ML '92), Aberdeen, Scotland, 1-3 July 1992 (ed. D. Sleeman & P. Edwards), pp. 385-393. San Mateo, CA: Morgan Kaufmann.
63. Schaal, S. 1997 Learning from demonstration. In Advances in neural information processing systems 9 (ed. M. C. Mozer, M. Jordan & T. Petsche), pp. 1040-1046. Cambridge, MA: MIT Press.
64. Schaal, S. 1999 Is imitation learning the route to humanoid robots? Trends Cogn. Sci. 3, 233-242.
65. Schaal, S., Sternad, D., Dean, W., Kotoska, S., Osu, R. & Kawato, M. 2000 Reciprocal excitation between biological and robotic research. In Sensor fusion and decentralized control in robotic systems III, Proc. of SPIE, Boston, MA, 5-8 November 1992, vol. 4196, pp. 30-40.
66. Schwenker, F., Sommer, F. T. & Palm, G. 1996 Iterative retrieval of sparsely coded associateve memory patterns. Neural Networks 9, 445-455.
67. Segre, A. B. & DeJong, G. 1985 Explanation-based manipulator learning: acquisition of planning ability through observation. In IEEE Conf. Robotics Automation, St Louis, MO, March 1985, pp. 555-560. Piscataway, NJ: IEEE.
68. Segre, A. M. 1988 Machine learning of robot assembly plans. Kluwer International Series in Engineering and Computer Science. Knowledge representation, learning, and expert systems. Boston, MA: Kluwer.
69. Sternad, D. & Schaal, D. 1999 Segmentation of endpoint trajectories does not imply segmented control. Exp. Brain Res. 124, 118-136.
70. Sutton, R. S. & Barto, A. G. 1998 Reinforcement learning: an introduction. Adaptive computation and machine learning. Cambridge, MA: MIT Press.
71. Sutton, R. S., Singh, S., Precup, D. & Ravindran, B. 1999 Improved switching among temporally abstract actions. In Advances in neural information processing systems, vol. 11. Cambridge, MA: MIT Press.
72. Tomasello, M., Savage-Rumbaugh, S. & Kruger, A. C. 1993 Imitative learning of actions on objects by children, chimpanzees, and enculturated chimpanzees. Child Dev. 64, 1688-1705.
73. Tung, C. P. & Kak, A. C. 1995 Automatic learning of assembly task using a DataGlove system. In IEEE/RSJ Int. Conf. Intell. Robots Systems, Pittsburgh, PA, vol. 1, pp. 1-8.
74. Van Essen, D. C. & Maunsell, J. M. R. 1983 Hierachical organization and functional streams in the visual cortex. Trends Neurosci. 6, 370-375.
75. Wada, Y. & Kawato, M. 1995 A theory for cursive handwriting based on the minimization principle. Biol. Cybern. 73, 3-13.
76. Weigend, A. S., Mangeas, M. & Srivastava, A. N. 1995 Non-linear gated experts for time series: discovering regimes and avoiding overfitting. Int. J. Neural Systems 6, 373-399.
77. Widrow, B. & Smith, F. W. 1964 Pattern recognizing control systems. In 1963 Comput. Inform. Sci. (COINS) Symp. Proc., pp. 288-317. Spartan.
78. Wolpert, D. M. 1997 Computational approaches to motor control. Trends Cogn. Sci. 1, 209-216.
79. Wolpert, D. M. & Kawato, M. 1998 Multiple paired forward and inverse models for motor control. Neural Networks 11, 1317-1329.
80. Wolpert, D. M., Miall, R. C. & Kawato, M. 1998 Internal models in the cerebellum. Trends Cogn. Sci. 2, 338-347.
81. Wolpert, D. M., Doya, K. & Kawato, M. 2003 A unifying computational framework for motor control and social interaction. Phil. Trans. R. Soc. Lond. B 358, 593-602. (DOI 10.1098/rstb.2002.1238.)