Internal models for motor control and trajectory planning

Current Opinion in Neurobiology

Volumn 9, Issue 6, 1999, Pages 718-727

  Kawato, Mitsuo ^a  

^a ADVANCED TELECOMMUNICATIONS RESEARCH INSTITUTE INTERNATIONAL   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

HUMAN; MOTOR CONTROL; NEUROSCIENCE; NONHUMAN; PRIORITY JOURNAL; PSYCHOLOGY; PURKINJE CELL; REVIEW; THEORY;

EID: 0032787485     PISSN: 09594388     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0959-4388(99)00028-8     Document Type: Review

References (80)

1. Ito M. Neurophysiological aspects of the cerebellar motor control system. Int J Neurol. 7:1970;162-176.
2. Kawato M., Furawaka K., Suzuki R. A hierarchical neural network model for the control and learning of voluntary movements. Biol Cybern. 56:1987;1-17.
3. Jordan M.I., Rumelhart D.E. Forward models: supervised learning with a distal teacher. Cogn Sci. 16:1992;307-354.
4. Miall R.C., Weir D.J., Wolpert D.M., Stein J.F. Is the cerebellum a Smith Predictor? J Motor Behav. 25:1993;203-216.
5. Shadmehr R., Mussa-Ivaldi F.A. Adaptive representation of dynamics during learning of a motor task. J Neurosci. 14:1994;3208-3224.
6. Lackner J.R., Dizio P. Rapid adaptation to Coriolis force perturbations of arm trajectory. J Neurophysiol. 72:1994;299-313.
7. Lackner J.R., Dizio P. Gravitoinertial force background level affects adaptation to Coriolis force perturbations of reaching movements. J Neurophysiol. 80:1998;546-553. In a rotating room environment [6], the transient, movement-dependent Coriolis forces and the static centrifugal forces were generated, and their combined effects on the adaptation of reaching movements toward a just-extinguished visual target were examined. An increase in background force level above normal gravity did not affect the adaptation to restore the straight paths, but did hinder restoration of reaching accuracy. This suggests that the inverse dynamics model for trajectory control and the inverse statics model for posture maintenance are anatomically separate.
8. Johansson R.S., Westling G. Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects. Exp Brain Res. 56:1984;550-564.
9. Flanagan J.R., Wing A.M. The role of internal models in motion planning and control: evidence from grip force adjustments during movements of hand-held loads. J Neurosci. 17:1997;1519-1528.
10. Tamada T., Miyauchi S., Imamizu H., Yoshioka T., Kawato M. Activation of the cerebellum in grip force and load force coordination: an fMRI study. Neuroimage. 6:1999;S492. Functional magnetic resonance imaging activation during hand movements while grasping an object in precision grip was found to be larger than during power grip in the lateral cerebellum. In consideration of the computational model of grip-force-load-force coupling shown in Figure 2, which is necessary only for the precision grip, it is suggested that a forward model of the object is located in the cerebellum.
11. Scarchilli K., Vercher J.L. The oculomanual coordination control center takes into account the mechanical properties of the arm. Exp Brain Res. 124:1999;42-52. When the eyes tracked a target that was controlled by an arm movement, the smooth pursuit latency was 0 ms whereas it was 150 ms for an external target. When an electromagnetic brake impeded the arm motion unexpectedly, the eye movement phase led the target motion. This strongly supports the coordination model shown in Figure 2, originally developed for grip-force-load-force coupling. In the figure, the grip force controller should be replaced by the smooth pursuit controller.
12. Snyder L. This way up: illusions and internal models in the vestibular system. Nat Neurosci. 2:1999;396-398.
13. Hess B.J., Angelaki D.E. Oculomotor control of primary eye position discriminates between translation and tilt. J Neurophysiol. 81:1999;394-398.
14. Merfeld D.M., Zupan L., Peterka R.J. Humans use internal models to estimate gravity and linear acceleration. Nature. 398:1999;615-618.
15. Feldman A.G. Functional tuning of nervous system with control of movement or maintenance of a steady posture. 2. Controllable parameters of the muscles. Biophysics. 11:1966;565-578.
16. Bizzi E., Accornero N., Chapple W., Hogan N. Posture control and trajectory formation during arm movement. J Neurosci. 4:1984;2738-2744.
17. Flash T. The control of hand equilibrium trajectories in multi-joint arm movements. Biol Cybern. 57:1987;257-274.
18. ••]).
19. Latash M.L., Gottlieb G.L. Reconstruction of shifting elbow joint compliant characteristics during fast and slow movements. Neuroscience. 43:1991;697-712.
20. Katayama M., Kawato M. Virtual trajectory and stiffness ellipse during multijoint arm movement predicted by neural inverse models. Biol Cybern. 69:1993;353-362.
21. Gomi H., Kawato M. Equilibrium-point control hypothesis examined by measured arm-stiffness during multi-joint movement. Science. 272:1996;117-120.
22. •].
23. Morasso P.G., Schieppati M. Can muscle stiffness alone stabilize upright standing? J Neurophysiol. 82:1999;1622-1626. The authors showed that intrinsic muscle stiffness is not strong enough to stabilize upright posture. Usage of internal models was suggested as the alternative.
24. Milner T.E., Cloutier C. Compensation for mechanically unstable loading in voluntary wrist movement. Exp Brain Res. 94:1993;522-532.
25. Sanger T.D. Neural network learning control of robot manipulators using gradually increasing task difficulty. IEEE Trans Robotics Automat. 10:1994;323-333.
26. th Annual International Conference of the IEEE Engineering in Medicine and Biology Society; 29 Oct-1 Nov 1998, Hong Kong. IEEE; 1998:2370-2373.
27. Doya K. What are the computations of cerebellum, basal ganglia, and cerebral cortex? Neural Networks. 12:1999;961-974. A new, large-scale theory is proposed to explain how the three brain regions - the cerebral cortex, the basal ganglia and the cerebellum - collaborate to solve difficult behavioral learning problems. Computationally, these three regions were characterized by unsupervised learning, reinforcement learning and supervised learning, respectively.
28. Marr D. A theory of cerebellar cortex. J Physiol. 202:1969;437-470.
29. Albus J.S. A theory of cerebellar function. Math Biosci. 10:1971;25-61.
30. Kawato M., Gomi H. The cerebellum and VOR/OKR learning models. Trends Neurosci. 15:1992;445-453.
31. Thach W.T. A role for the cerebellum in learning movement coordination. Neurobiol Learn Mem. 70:1998;177-188. This is a review of the author's brilliant contributions to the elucidation of cerebellar functions in motor learning and coordination. A proposed cerebellar theory of coordination between different degrees of freedom by the use of long parallel fibers was a large leap forward from the classical Marr-Albus-Ito theory, and is very closely related to internal model theories in computational essence.
32. Kawano K., Takemura A., Inoue Y., Kitama T., Kobayashi Y., Mustari M.J. Visual inputs to cerebellar ventral paraflocculus during ocular following responses. Prog Brain Res. 112:1996;415-422.
33. Kobayashi Y., Kawano K., Takemura A., Inoue Y., Kitama T., Gomi H., Kawato M. Temporal firing patterns of Purkinje cells in the cerebellar ventral paraflocculus during ocular following responses in monkeys. II. Complex spikes. J Neurophysiol. 80:1998;832-848. Although the firing rates of Purkinje cell complex spikes are very low, when many trials were averaged during ocular following responses, their temporal firing patterns conveyed high-frequency and smooth waveforms which were well reconstructed from a linear combination of acceleration, velocity and position of eye movement. This supports the most critical assumption of the cerebellar feedback-error-learning theory [2] that climbing fiber inputs carry error signals in the motor command coordinates.
34. Shidara M., Kawano K., Gomi H., Kawato M. Inverse-dynamics model eye movement control by Purkinje cells in the cerebellum. Nature. 365:1993;50-52.
35. Gomi H., Shidara M., Takemura A., Inoue Y., Kawano K., Kawato M. Temporal firing patterns of Purkinje cells in the cerebellar ventral paraflocculus during ocular following responses in monkeys. I. Simple spikes. J Neurophysiol. 80:1998;818-831. The simple spike firing patterns of Purkinje cells were well reconstructed from a linear combination of acceleration, velocity and position of eye movement under various stiumulus conditions. The findings of an earlier report [34] was strengthened by the examination of a larger number of cells and a larger number of stimulus conditions. In combination with [37], this suggests that the cerebellar cortex is a major site of the inverse dynamics model of the eye.
36. ••] was applied to cat flocculus during optokinetic eye movements. The results not only confirmed previous studies but also strengthened the negative coefficient of the position even when the slide term, that is the time derivative of the firing rates, is included in the linear regression.
37. Takemura A., Inoue Y., Gomi H., Kawato M., Kawano K. Analysis of neuronal activities during ocular following responses in alert monkeys. Tech Rep IEICE. NC99-22:1999;77-84.
38. Yamamoto K., Kobayashi Y., Takemura A., Kawano K., Kawato M. A mathematical model that reproduces vertical ocular following responses from visual stimuli. Neurosci Res. 29:1997;161-169.
39. Yamamoto K., Kobayashi Y., Takemura A., Kawano K., Kawato M. A computational simulation on the adaptation of vertical ocular following responses. Tech Rep IEICE. NC97-131:1998;229-236.
40. Ebner T.J. A role for the cerebellum in the control of limb movement velocity. Curr Opin Neurobiol. 8:1998;762-769.
41. ••].
42. Imamizu H., Miyauchi S., Tamada T., Sasaki Y., Takino R., Putz B., Yoshioka T., Kawato M. Human cirebellar activity reflecting an aquired internal model of a novel tool. Nature. 1999;. in press. Previous neuroimaging studies reported a decrease in cerebellar activities along with motor or cognitive learning. These results have often been interpreted as suggesting that the cerebellum is important only in the early phase of learning and that it is not the memory site. In these interpretations, the internal models are not located in the cerebellum. The authors showed that cerebellar activity remains even after sufficient learning by dexterously equalizing error levels between the test and control conditions.
43. ••], were known to be activated by unexpected events but not by physically equivalent but predictable events. It is still puzzling why the cerebellar activity during self-produced tickling is lower than the rest condition. Simple cancellation can explain activity negation, but cannot explain activity reduction.
44. Miall R.C., Keating J.G., Malkmus M., Thach W.T. Purkinje cell complex spikes are predicted by simple spike activity. Nat Neurosci. 1:1998;13-15. Mean simple spike firing rates triggered by a complex spike during intertask intervals of monkey cerebellum were examined. It was found that the simple spike firing rates about 150 ms before the complex spike occurrence were statistically larger than background. This could be interpreted in the framework of the forward model predicting sensory consequences. The simple spikes make predictions about future sensory events, and these are compared with the real sensory feedback after a 150 ms sensory time delay; their consequences are represented as complex spikes.
45. Baizer J.S., Kralj-Hans I., Glickstein M. Cerebellar lesions and prism adaptation in macaque monkeys. J Neurophysiol. 81:1999;1960-1965. It was suggested that the cerebellum is involved in prism adaptation because lesion of almost all of the cerebellar cortex, which receives mossy fiber visual inputs, abolished prism adaptation capability in a monkey. Internal models for prism adaptation could also be located in the cerebellum.
46. Schweighofer N., Spoelstra J., Arbib M.A., Kawato M. Role of the cerebellum in reaching quickly and accurately. II. A detailed model of the intermediate cerebellum. Eur J Neurosci. 10:1998;95-105. This is a simulation study of the learning inverse dynamics model of the arm in the intermediate cerebellum based on the feedback-error-learning model. Realistic time delays were assumed and the inverse model was distributed over the cerebellum and the cerebral cortex.
47. De Zeeuw C.I., Simpson J.I., Hoogenraad C.C., Galjart N., Koekkoek S.K., Ruigrok T.J. Microcircuitry and function of the inferior olive. Trends Neurosci. 21:1998;391-400.
48. Schweighofer N., Doya K., Kawato M. Electrophysiological properties of inferior olive neurons: a compartmental model. J Neurophysiol. 82:1999;804-817. A detailed biophysical model of an inferior olive neuron was developed. The model comprises the soma and the dentritic compartments, and is based on known spatial distributions of gap junctions and diverse ionic conductances. The model reproduced many peculiar characteristics found in electrophysiological experiments in vitro. Most surprisingly, it showed that electrical junctions have desynchronization effects on coupled inferior olive neurons for a wide range of parameters.
49. Imamizu H., Uno Y., Kawato M. Internal representation of motor apparatus: implications from generalization in visuo-motor learning. J Exp Psychol [Hum Percept Perform]. 21:1995;1174-1198.
50. Ghahramani Z., Wolpert D.M., Jordan M.I. Generalization to local remappings of the visuomotor coordinate transformation. J Neurosci. 16:1996;7085-7096.
51. Gandolfo F., Mussa-Ivaldi F.A., Bizzi E. Motor learning by field approximation. Proc Natl Acad Sci USA. 93:1996;3843-3846.
52. Conditt M.A., Gandolfo F., Mussa-Ivaldi F.A. The motor system does not learn the dynamics of the arm by rote memorization of past experience. J Neurophysiol. 78:1997;554-560.
53. Sainburg R.L., Ghez C., Kalakainis D. Intersegmental dynamics are controlled by sequential anticipatory, error correction, and postural mechanisms. J Neurophysiol. 81:1999;1045-1055. Previous adaptation studies introduced force fields that are dependent only on velocity and position (e.g. viscous force fields, Coriolis forces, spring force fields), but inertial force fields were difficult to implement even with a high-performance robotic manipulandum. The authors utilized an air sled system and an off-axis load to introduce an interesting novel inertial force field. Generalization of learning was limited to a small range of direction angles, which is in accordance with previous studies using viscous force fields [51,52] or kinematic rotation [49].
54. Kitazawa S., Kimura T., Uka T. Prism adaptation of reaching movements: specificity for the velocity of reaching. J Neurosci. 17:1997;1481-1492.
55. Goodbody S.J., Wolpert D.M. Temporal and amplitude generalization in motor learning. J Neurophysiol. 79:1998;1825-1838. A viscous, rotating-type force field was applied by a robotic interface to the arm. Subjects were trained with a reaching movement having a specific amplitude and duration. Then, generalization was examined for different durations and amplitudes. Most surprisingly, and in contrast to almost all previous studies (particularly [54]), generalization was best modeled by a linear map - that is, generalization was global rather than local. It is possible that parallel fiber inputs to the cerebellum encode velocity information by their firing rates while other information, such as position and direction, is encoded by population coding. Thus, we expect good generalization only for velocities because only velocity-dependent terms are represented linearly in Purkinje cell outputs on this assumption.
56. Jacobs R.A., Jordan M.I., Nowlan S.J., Hinton G.E. Adaptive mixtures of local experts. Neural Comp. 3:1991;79-87.
57. Shadmehr R., Brashers-Krug T. Functional stages in the formation of human long-term motor memory. J Neurosci. 17:1997;409-419.
58. Shadmehr R., Holcomb H.H. Inhibitory control of competing motor memories. Exp Brain Res. 126:1999;235-251.
59. Ghahramani Z., Wolpert D.M. Modular decomposition in visuomotor learning. Nature. 386:1997;392-395.
60. ••].
61. Flanagan R., Nakano E., Imamizu H., Osu R., Yoshioka T., Kawato M. Composition and decomposition of internal models in motor learning under altered kinematic and dynamic environments. J Neurosci. 19(Rapid Communication 34):1999;1-5. Multiplicity of internal models for dynamic and kinematic transformations was investigated in order to determine whether multiple internal model can be composed or decomposed when occasion calls. When subjects learned a visuomotor rotation and viscous curl field first, reaching errors were smaller for a combination of these transformation than without prior experiences.
62. Wolpert D.M., Kawato M. Multiple paired forward and inverse models for motor control. Neural Networks. 11:1998;1317-1329. It was postulated that the best way to control learning and the contributions of different motor primitives (controllers, or inverse models) is to use the prediction goodness of corresponding predictors (forward models). This basic idea, strengthened by a responsibility predictor, provides an attractive Bayesian framework to explain rich behaviors in complicated environments.
63. ••]. Multiple pairs of forward and inverse models were automatically acquired for different behavioral situations. A finite number of multiple models were shown to be able to deal with a much larger number of situations. The 'size-weight illusion' was also simulated.
64. Abend W., Bizzi E., Morasso P. Human arm trajectory formation. Brain. 105:1982;331-348.
65. Flash T., Hogan N. The coordination of arm movements: an experimentally confirmed mathematical model. J Neurosci. 5:1985;1688-1703.
66. Uno Y., Kawato M., Suzuki R. Formation and control of optimal trajectory in human multijoint arm movement - minimum torque-change model. Biol Cybern. 61:1989;89-101.
67. Flanagan J.R., Rao A.K. Trajectory adaptation to a nonlinear visuo-motor transformation: evidence of motion planning in visually perceived space. J Neurophysiol. 74:1995;2174-2178.
68. Wolpert D.M., Ghahramani Z., Jordan M.I. Perceptual distortion contributes to the curvature of human reaching movements. Exp Brain Res. 98:1994;153-156.
69. Wolpert D.M., Ghahramani Z., Jordan M.I. Are arm trajectories planned in kinematic or dynamic coordinates? An adaptation study. Exp Brain Res. 103:1995;460-470.
70. Okadome T., Honda M. Kinematic construction of the trajectory of sequential arm movements. Biol Cybern. 80:1999;157-169.
71. Osu R., Uno Y., Koike Y., Kawato M. Possible explanations for trajectory curvature in multijoint arm movements. J Exp Psychol [Hum Percept Perform]. 23:1997;890-913.
72. Todorov E.V., Jordan M.I. Smoothness maximization along a predefined path accurately predicts the speed profiles of complex arm movements. J Neurophysiol. 80:1998;696-714. The authors proposed a constrained optimization model to explain known regularities between velocity profiles and hand path curvature. For an observed path of the hand in space, the velocity profile along that path will be the one that minimizes jerk. This model explains the data better than the 2/3 power-law.
73. Sabes P.N., Jordan M.I., Wolpert D.M. The role of inertial sensitivity in motor planning. J Neurosci. 18:1998;5948-5957. The kinematics of reaching movements was examined when a visual obstacle was set in three-dimensional space. The distribution of the near points, where the paths came closest to the obstacle, showed a strong anisotropy, clustering at the poles of a preferred axis through the center of the obstacle. The mobility minor axis, which represents the most inertially stable direction to a force perturbation, was not significantly different from the preferred near point axis. These results suggest that the dynamic characteristics of the arm are taken into account in the trajectory planning process.
74. Nakano E., Imamizu H., Osu R., Uno Y., Gomi H., Yoshioka T., Kawato M. Quantitative examinations of internal representations for arm trajectory planning: minimum commanded torque change model. J Neurophysiol. 81:1999;2140-2155. As a computable version of the minimum motor-command change model, a minimum commanded torque change model was proposed and compared with other optimization models based on a large database of horizontal and vertical two-joint arm movements. The best performance of the model supports dynamic trajectory planning while taking into account not only environmental and link dynamics but also muscle dynamics.
75. Kawato M. Trajectory formation in arm movements: minimization principles and procedures. Zelaznik H.N. Advances in Motor Learning and Control. 1996;225-259 Human Kinetics Publishers, Illinois.
76. Soechting J.F., Flanders M. Movement planning: kinematics, dynamics, both or neither? Harris L.R., Jenkin M. Vision and Action. 1998;332-349 Cambridge University Press, Cambridge. The final posture of the arm in a reaching movement can be predicted according to a criterion that is related to energy expenditure, and the hand paths during planar arm movements can be at least partly predicted by another criterion (minimum muscle force change) that is also related to energy expenditure. The authors conclude that movement planning must involve dynamics.
77. Harris C.M., Wolpert D.M. Signal-dependent noise determines motor planning. Nature. 394:1998;780-784. A unifying theory on trajectory formation of saccades and arm movements is proposed. It is assumed that the neural control signals are corrupted by noise, which increases with the size of the control signal. A simple principle to minimize the variance of the final eye or hand position could thus reproduce known invariant trajectory characteristics, Fitts' law, and the 2/3 power law. The final position error is the time integral of the noise in the motor command, which is filtered by time-variant linearized dynamics of the motor plant. In this interpretation, the minimum variance model could be viewed as a temporally weighted version of the minimum motor-command change model, because the noise variance is related to the square of the motor command.
78. Kawato M. Bidirectional theory approach to consciousness. Ito M., Miyashita Y., Rolls E.T. Cognition, Computation and Consciousness. 1997;223-248 Oxford University Press, Oxford.
79. Middleton F.A., Strick P.L. The cerebellum: an overview. Trends Cog Sci. 2:1998;305-306. [Also published in Trends Neurosci 1998, 21:367-368.].
80. ••]. Pars opecularis and pars triangularis were shown to be functionally connected with this part of the cerebellum. This is interesting because some theories advocate that language evolution is an extension of human tool usage capability.