Modular features of motor control and learning

Current Opinion in Neurobiology

Volumn 9, Issue 6, 1999, Pages 713-717

  Mussa Ivaldi, Ferdinando A ^a  

^a NORTHWESTERN UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

HUMAN; LEARNING; MOTOR CONTROL; MOTOR CORTEX; MOTOR SYSTEM; PRIORITY JOURNAL; RECEPTIVE FIELD; REVIEW;

EID: 0032795108     PISSN: 09594388     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0959-4388(99)00029-X     Document Type: Review

References (60)

1. Burridge R., Rizzi A., Koditschek D. Sequential composition of dynamically dexterous robot behaviours. Int J Robotics Res. 18:1999;534-555. The paper describes both in formal and in intuitive terms an elegant approach to implementing complex manipulation skills in robot manipulators. In particular, this study focuses on problems such as controlling the bouncing of a ball on a paddle, where it is difficult for an engineer to define a single control procedure that may work across the entire state space. To deal with these problems, the authors divide the state space of the controlled system (for example, the space of all possible positions and velocities of the paddle and of the ball) into smaller non-overlapping regions. If a region is sufficiently small, then a simple control procedure - called a 'local controller' - is sufficient to ensure that the controlled system, once it has entered the region, can be bought to a pre-defined state. After this fragmentation is completed, the local controllers are combined into a single composite procedure that is capable of bringing the system from any state to the desired final state. The work makes use of the formalism of differential geometry to develop this idea.
2. Mason M. Mechanics and planning of manipulator pushing operations. Int J Robotics Res. 5:1986;53-71.
3. Bicchi A., Sorrentino R. Dexterous manipulation through rolling. Proceedings of the IEEE International Conference on Robotics and Automation. 1995;425-457 IEEE, Piscataway, New Jersey.
4. Lynch K., Mason M. Dynamic nonprehensile manipulation: controllability, planning and experiments. Int J Robotics Res. 18:1999;64-92. Conventional robot manipulators establish a rigid contact with the manipulated object. This simplifies the geometrical relation between the object and manipulator coordinates; however, it also constrains the range of possible operations. Humans and other animals handle objects by establishing 'nonprehensile' contacts: that is, contacts that allow slipping or rolling of the object. This type of contact is more difficult to describe mathematically, but it offers a broader repertoire of control policies. Lynch and Mason provide a mathematically insightful approach for solving this problem and some impressive demonstrations.
5. Ghahramani Z., Wolpert D. Modular decomposition in visuomotor learning. Nature. 386:1997;392-395.
6. Marr D. A theory of cerebellar cortex. J Physiol. 202:1969;437-470.
7. Albus J. The theory of cerebellar function. Math Biosci. 10:1971;25-61.
8. Bizzi E., Mussa-Ivaldi F.A. The acquisition of motor behaviour. Daedalus. 127:1998;217-232.
9. Schaal S. Learning from demonstration. Mozer M., Jordan M., Petsche T. Advances in Neural Information Processing Systems. 1997;1040-1046 MIT Press, Cambridge, Massachusetts.
10. Atkeson C., Schaal S. Robot learning from demonstration. Fisher D.H. Jr. Machine Learning: Proceedings of the Fourteenth International Conference (ICML '97). 1997;12-20 Morgan Kaufmann, San Fransisco.
11. Schaal S. Is imitation learning the route to humanoid robots? Trends Cog Sci. 3:1999;233-242. In previous work, Schaal and Atkeson demonstrated that learning by imitation is most effective when the demonstration data are used to prime an internal model. In this paper, Schaal compares a variety of different approaches to learning by imitation, including model-based learning, symbolic approaches and direct learning of control policies.
12. Whiten A. Imitation of the sequential structure of actions by chimpanzees (Pan troglodytes). J Comp Psychol. 112:1998;270-281.
13. Byrne R., Russon A. Learning by imitation: a hierarchical approach. Behav Brain Sci. 21:1998;667-684.
14. Mataric M., Pomplun M. Fixation behaviour in observation and imitation of human movement. Brain Res. 7:1998;191-202.
15. Jordan M., Rumelhart D. Forward models: supervised learning with a distal teacher. Cog Sci. 16:1992;307-354.
16. Shadmehr R., Mussa-Ivaldi F.A. Adaptive representation of dynamics during learning of a motor task. J Neurosci. 14:1994;3208-3224.
17. Johansson R. Sensory input and control of grip. Novartis Foundation Symposium. 218:1998;45-59.
18. Kawato M., Wolpert D. Internal models for motor control. Novartis Foundation Symposium. 218:1998;291-304.
19. McIntyre J., Berthoz A., Lacquaniti F. Reference frames and internal models. Brain Res Rev. 28:1998;143-154.
20. Blakemore S., Goodbody S., Wolpert D. Predicting the consequences of our own actions: the role of sensorimotor. J Neurosci. 18:1998;7511-7518.
21. Miall R., Wolpert D. Forward models for physiological motor control. Neural Networks. 9:1996;1265-1279.
22. Wolpert D., Kawato M. Multiple paired forward and inverse models for motor control. Neural Networks. 11:1998;1317-1329. This paper describes a computational architecture in which motor commands are generated by a collection of paired forward and inverse models. The approach is similar to the 'mixture of experts' proposed earlier by Jacobs et al. [60]. Here, different forward models generate independent state estimations. Comparison of these predictions with the actual state is used to establish the degree to which each corresponding inverse model should contribute to the motor output.
23. Wolpert D., Miall R., Kawato M. Internal models in the cerebellum. Trends Cog Sci. 2:1998;338-347.
24. Flash T., Gurevich I. Arm stiffness and movement adaptation to external loads. Proc Annu Con Eng Med. 13:1992;885-886.
25. Brashers-Krug T., Shadmehr R., Bizzi E. Consolidation in human motor memory. Nature. 382:1996;252-255.
26. Gottlieb G. On the voluntary movement of compliant (inertial-viscoelastic) loads by parcellated control mechanisms. J Neurophysiol. 76:1996;3207-3229.
27. Flanagan J., Wing A. The role of internal models in motion planning and control: evidence from grip force adjustments during movements of hand-held loads. J Neurosci. 17:1997;1519-1528.
28. Sabes P., Jordan M., Wolpert D. The role of inertial sensitivity in motor planning. J Neurosci. 18:1998;5948-5957. The authors demonstrate that when an obstacle must be circumvented during a reaching movement, the hand trajectory is planned so that the point of minimum distance to the object is reached with the largest inertia in the direction between the hand and obstacle. This course of action ensures that when the hand is closest to the object, there is a minimal sensitivity to perturbations that could cause a collision. This paper provides evidence that internal models of limb dynamics are used in motor planning.
29. Hore J., Ritchi R., Watts S. Finger opening in an overarm throw is not triggered by proprioceptive feedback from elbow extension or wrist flexion. Exp Brain Res. 125:1999;301-312.
30. Conditt M.A., Gandolfo F., Mussa-Ivaldi F.A. The motor system does not learn the dynamics of the arm by rote memorization of past experience. J Neurophysiol. 78:1997;554-560.
31. Conditt M.A., Mussa-Ivaldi F.A. Central representation of time during motor learning. Proc Natl Acad Sci USA. 96:1999;11625-11630. Earlier studies showed that the motor system compensates for a perturbing force field by representing the dependence of the experience force upon the position and velocity of the limb. This paper demonstrates that when the experimental force depends explicitly on time instead of position or velocity, the internal representation is not accurate because the motor system still compensates for these forces as though they depended upon limb state. As a consequence, adaptation does not generalize correctly to different movements.
32. Bhushan N., Shadmehr R. Computational nature of human adaptive control during learning of reaching movements in force fields. Biol Cyber. 81:1999;39-60. This paper provides the first compelling evidence suggesting that a forward and an inverse model of arm dynamics operate jointly during adaptation to a novel force field. The authors suggest that a forward model is needed to provide the motor system with the opportunity to mentally rehearse new dynamics following a limited period of practice. This rehearsal would allow the consolidation of an inverse model, a process that has been observed in subjects after an initial exposure to a novel force field.
33. Marr D. Vision. 1982;WH Freeman and Co, San Francisco, California.
34. Sutton R., Barto A. Reinforcement Learning: an Introduction. 1998;MIT Press, Cambridge, Massachusetts.
35. Schaal S., Atkeson C. Constructive incremental learning from only local information. Neural Computation. 10:1998;2047-2084. The authors describe a learning model, called receptive field weighted regression, in which simple locally linear elements are combined to approximate unknown functions in parallel with the collection of input-output data. This type of learning is called on-line learning to distinguish it from batch-learning methods, in which all the data are first acquired and then used subsequently. The authors show that the receptive fields of the linear elements can be adapted rapidly to minimize the amount of negative interference between different regions of the input space. It is worthwhile to observe that the local linear elements of this model correspond mathematically to the Gaussian fields described and used in [38].
36. Bishop C. Neural Networks for Pattern Recognition. 1996;Oxford University Press, Oxford.
37. Platt J.A. resource-allocating network for function interpolation. Neural Computation. 3:1991;213-225.
38. Mussa-Ivaldi F.A. Nonlinear force fields: a distributed system of control primitives for representing and learning movements. Proceedings of the 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation: 1997, July 10-11. 1997;84-90 The Computer Society Press, Monterey, Los Alanutos, CA.
39. Giszter S., Mussa-Ivaldi F.A., Bizzi E. Convergent force fields organized in the frog's spinal cord. J Neurosci. 13:1993;467-491.
40. Mussa-Ivaldi F.A., Giszter S., Bizzi E. Linear combinations of primitives in vertebrate motor control. Proc Natl Acad Sci USA. 91:1994;7534-7538.
41. Tresch M., Saltiel P., Bizzi E. The construction of movement by the spinal cord. Nat Neurosci. 2:1999;162-167. These authors used non-linear regression techniques to fit electromyographic (EMG) data obtained after cutaneous stimulation of several sites of the frog hindlimb. They found that the linear combination of only four muscle synergies was sufficient to reproduce 126 EMG activities induced in nine muscles by electrically stimulating the leg in 14 different locations. These results are consistent with the hypothesis that the neural circuits of the spinal cord implement a small number of independent muscle synergies.
42. D'Avella A., Bizzi E. Low dimensionality of supraspinally induced force fields. Proc Natl Acad Sci USA. 95:1998;7711-7714.
43. Hogan N. The mechanics of multi-joint posture and movement control. Biol Cybernetics. 52:1985;315-331.
44. Won J., Hogan N. Stability properties of human reaching movements. Exp Brain Res. 107:1995;125-136.
45. Bizzi E., Hogan N., Mussa-Ivaldi F.A., Giszter S. Does the nervous system use equilibrium-point control to guide single and multiple joint movements? Behav Brain Sci. 15:1992;603-613.
46. Lohmiller W., Slotine J.-J. On contraction analysis for nonlinear systems. Automatica. 34:1998;683-696. This paper describes an approach to investigating the stability of a control system based on a paradigm from nonlinear fluid dynamics. The approach is centered on the definition of a contraction region as a region in which the Jacobian of the dynamics equations is uniformly negative definite. The analysis of the contraction regions of a dynamic system allows one to make strong inferences about the exponential convergence of the system toward a nominal trajectory.
47. Georgopoulos A.P., Kalaska J.F., Camintini R., Massey J.T. On the relations between the direction of two dimensional arm movements and cell discharge in primate motor cortex. J Neurosci. 2:1982;1527-1537.
48. Georgopoulos A., Lurito J., Petrides M., Schwartz A., Massey J. Mental rotation of the neuronal population vector. Science. 243:1989;234-236.
49. Georgopoulos A., Taira M., Lukashin A. Cognitive neurophysiology of the motor cortex. Science. 260:1993;47-52.
50. Mussa-Ivaldi F. Do neurons in the motor cortex encode movement direction? An alternative hypothesis. Neurosci Lett. 91:1988;106-111.
51. Sanger T. Theoretical considerations for the analysis of population coding in motor cortex. Neural Computation. 6:1994;12-21.
52. Caminiti R., Johnson P., Urbano A. Making arm movements within different parts of space: dynamic aspects in the primate motor cortex. J Neurosci. 10:1990;2039-2058.
53. Sergio L., Kalaska J. Systematic changes in directional tuning of motor cortex cell activity. J Neurophysiol. 78:1997;1170-1174.
54. Scott S., Kalaska J. Reaching movements with similar hand paths but different arm. J Neurophysiol. 77:1997;826-852.
55. Fetz E., Finocchio D. Operant conditioning of isolated activity in specific muscles and precentral cells. Brain Res. 40:1972;19-23.
56. Carpenter A., Georgopoulos A., Pellizzer G. Motor cortical encoding of serial order in a context-recall task. Science. 283:1999;1752-1757. This is a study of motor cortical activity during a context-recall task. In this task, a monkey was instructed to look at a sequence of targets; it was then cued to exert a force in a particular direction. A significant number of cells were found to modulate their activity exclusively in relation to the serial order of a target, regardless of its spatial location. This indicates a role for the motor cortex in higher-order planning of motor sequences.
57. Chapin J., Moxon K., Markowitz R., Nicolelis M. Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex. Nat Neurosci. 2:1999;664-670. These investigators used the cortical and thalamic signals associated with a movement of the forelimb for controlling a motorized lever that performed the same task as the forelimb by releasing a water reward. As the action was transferred from the limb-operated lever to the motorized lever, the limb ceased to move while the neural activities remained correlated with the task performed by the artificial mechanisms. This study provides a convincing demonstration that cortical activity patterns may be dissociated from the action of muscles.
58. Rizzolatti G., Fadiga L. Grasping objects and grasping action meanings: the dual role of monkey rostroventral premotor cortex (area F5). Novartis Foundation Symposium. 218:1998;81-95.
59. Rizzolatti G., Arbib M. Language within our grasp. Trends Neurosci. 21:1998;188-194. This is a thought-provoking interpretation of the role played by mirror neurons, which were described by Rizzolatti's group earlier (see [58]). Considering the correspondence of area F5 is monkey with Broca's area in human, the authors suggest that mirror neurons provide the foundation for the development of symbolic and linguistic processing in humans. This idea is based on the observation that language evolved not from animal calls but, more likely, from the mechanisms of communication associated with imitative behaviours.
60. Jacobs R.A., Jordan M.I., Nomlan S.J., Hinton G.E. Adoptive mixtures of local experts. Neural Computation. 3:1991;79-87.