Optimality principles in sensorimotor control

Nature Neuroscience

Volumn 7, Issue 9, 2004, Pages 907-915

  Todorov, Emanuel ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACCURACY; ADAPTATION; BEHAVIOR; BIOMECHANICS; CONTROL; ELECTROMYOGRAPHY; EVOLUTION; FEEDBACK SYSTEM; HUMAN; KINEMATICS; LEARNING; MODEL; MOTOR PERFORMANCE; MUSCLE FUNCTION; MUSCULOSKELETAL SYSTEM; PREDICTION; PRIORITY JOURNAL; REVIEW; SENSORIMOTOR FUNCTION; SENSORY STIMULATION; TASK PERFORMANCE; TIME; VOLUNTARY MOVEMENT;

ADAPTATION, PHYSIOLOGICAL; ANIMALS; COMPUTER SIMULATION; FEEDBACK; HUMANS; MODELS, NEUROLOGICAL; MOVEMENT; PSYCHOMOTOR PERFORMANCE; STOCHASTIC PROCESSES;

EID: 4344685154     PISSN: 10976256     EISSN: None     Source Type: Journal    
DOI: 10.1038/nn1309     Document Type: Review

References (119)

1. Chow, C.K. & Jacobson, D.H. Studies of human locomotion via optimal programming. Math. Biosci. 10, 239-306 (1971).
2. Hatze, H. & Buys, J.D. Energy-optimal controls in the mammalian neuromuscular system. Biol. Cybern. 27, 9-20 (1977).
3. Davy, D.T. & Audu, M.L. A dynamic optimization technique for predicting muscle forces in the swing phase of gait. J. Biomech. 20, 187-201 (1987).
4. Collins, J.J. The redundant nature of locomotor optimization laws. J. Biomech. 28, 251-267 (1995).
5. Popovic, D., Stein, R.B., Oguztoreli, N., Lebiedowska, M. & Jonic, S. Optimal control of walking with functional electrical stimulation: a computer simulation study. IEEE Trans. Rehabil. Eng. 7, 69-79 (1999).
6. Anderson, F.C. & Pandy, M.G. Dynamic optimization of human walking. J. Biomech. Eng. 123, 381-390 (2001).
7. Rasmussen, J., Damsgaard, M. & Voigt, M. Muscle recruitment by the min/max criterion - a comparative numerical study. J. Biomech. 34, 409-415 (2001).
8. Nelson, W.L. Physical principles for economies of skilled movements. Biol. Cybern. 46, 135-147 (1983).
9. Hogan, N. An organizing principle for a class of voluntary movements. J. Neurosci. 4, 2745-2754 (1984)
10. Flash, T. & Hogan, N. The coordination of arm movements: an experimentally confirmed mathematical model. J. Neurosci. 5, 1688-1703 (1985).
11. Pandy, M.G., Garner, B.A. & Anderson, F.C. Optimal control of non-ballistic muscular movements: a constraint-based performance criterion for rising from a chair. J. Biomech. Eng. 117, 15-26 (1995).
12. Todorov, E. & Jordan, M.I. Smoothness maximization along a predefined path accurately predicts the speed profiles of complex arm movements. J. Neurophysiol. 80, 696-714 (1998).
13. Smeets, J.B. & Brenner, E. A new view on grasping. Motor Control 3, 237-271 (1999).
14. Uno, Y., Kawato, M. & Suzuki, R. Formation and control of optimal trajectory in human multijoint arm movement: Minimum torque-change model. Biol. Cybern. 61, 89-101 (1989).
15. Nakano, E. et al. Quantitative examinations of internal representations for arm trajectory planning: minimum commanded torque change model. J. Neurophysiol. 81, 2140-2155 (1999).
16. Harris, C.M. & Wolpert, D.M. Signal-dependent noise determines motor planning. Nature 394, 780-784 (1998).
17. Harwood, M.R., Mezey, L.E. & Harris, C.M. The spectral main sequence of human saccades. J. Neurosci. 19, 9098-9106 (1999).
18. Hamilton, A.F. & Wolpert, D.M. Controlling the statistics of action: obstacle avoidance. J. Neurophysiol. 87, 2434-2440 (2002).
19. Pandy, M.G., Zajac, F.E., Sim, E. & Levine, W.S. An optimal control model for maximum-height human jumping. J. Biomech. 23, 1185-1198 (1990).
20. Valero-Cuevas, F.J., Zajac, F.E. & Burgar, C.G. Large index-fingertip forces are produced by subject-independent patterns of muscle excitation. J. Biomech. 31, 693-703 (1998).
21. Todorov, E. Cosine tuning minimizes motor errors. Neural Comput. 14, 1233-1260 (2002).
22. Fagg, A.H., Shah, A. & Barto, A.G. A computational model of muscle recruitment for wrist movements. J. Neurophysiol. 88, 3348-3358 (2002).
23. Ivanchenko, V. & Jacobs, R.A. A developmental approach aids motor learning. Neural Comput. 15, 2051-2065 (2003).
24. Loeb, G.E., Levine, W.S. & He, J. Understanding sensorimotor feedback through optimal control. Cold Spring Harb. Symp. Quant. Biol. 55, 791-803 (1990).
25. Hoff, B. A Computational Description of the Organization of Human Reaching and Prehension. Ph.D. Thesis, Univ. Southern California (1992).
26. Hoff, B. & Arbib, M.A. Models of trajectory formation and temporal interaction of reach and grasp. J. Mot. Behav. 25, 175-192 (1993).
27. Kuo, A.D. An optimal control model for analyzing human postural balance. IEEE Trans. Biomed. Eng. 42, 87-101 (1995).
28. Shimansky, Y.P. Spinal motor control system incorporates an internal model of limb dynamics. Biol. Cybern. 83, 379-389 (2000).
29. Ijspeert, A.J. A connectionist central pattern generator for the aquatic and terrestrial gaits of a simulated salamander. Biol. Cybern. 84, 331-348 (2001).
30. Todorov, E. & Jordan, M. Optimal feedback control as a theory of motor coordination. Nat. Neurosci. 5, 1226-1235 (2002).
31. Shimansky, Y.P., Kang, T. & He, J.P. A novel model of motor learning capable of developing an optimal movement control law online from scratch. Biol. Cybern. 90, 133-145 (2004).
32. Li, W., Todorov, E. & Pan, X. Hierarchical optimal control of redundant biomechanical systems. in Proceedings of the 26th Annual International Conferences of the IEEE Engineering in Medicine and Biology Society (IEEE Press, ISSN 1094-687X, in the press).
33. Marr, D. Vision (Freeman, San Francisco, 1982).
34. Feldman, A.G. & Levin, M.F. The origin and use of positional frames of reference in motor control. Behav. Brain Sci. 18, 723-744 (1995).
35. Bizzi, E., Hogan, N., Mussa-Ivaldi, F.A. & Giszter, S. Does the nervous system use equilibrium-point control to guide single and multiple joint movements? Behav. Brain Sci. 15, 603-613 (1992).
36. Kelso, J.A.S. Dynamic Patterns: The Self-Organization of Brain and Behavior (MIT Press, Cambridge, Massachusetts, 1995).
37. Sporns, O. & Edelman, G.M. Solving Bernstein's problem: a proposal for the development of coordinated movement by selection. Child Dev. 64, 960-981 (1993).
38. Kalaska, J., Sergio, L.E. & Cisek, P. in Sensory Guidance of Movement: Novartis Foundation Symposium (ed. Glickstein, M.) 176-201 (Wiley, Chichester, UK, 1998).
39. Bernstein, N.I. The Coordination and Regulation of Movements (Pergamon, Oxford, 1967).
40. Loeb, G.E., Brown, I.E. & Cheng, E.J. A hierarchical foundation for models of sensorimotor control. Exp. Brain Res. 126, 1-18 (1999).
41. Todorov, E. & Ghahramani, Z. Unsupervised learning of sensory-motor primitives. in Proceedings of the 25th Annual International Conference of the IEEE Engineering in Biology and Medicine Society 1750-1753 (IEEE Press, ISSN 1094-687X, 2003).
42. Kalman, R. When is a linear control system optimal? Trans. AMSE J. Basic Eng. Ser. D 86, 51-60 (1964).
43. Moylan, P. & Anderson, B. Nonlinear regulator theory and an inverse optimal control problem. IEEE Trans. Automatic Control AC-18, 460-465 (1973).
44. Ng, A. & Russell, S. Algorithms for inverse reinforcement learning. in Proceedings of the 17th International Conference on Machine Learning (Morgan Kaufmann, San Francisco, 2000).
45. Scott, S. Optimal feedback control and the neural basis of volitional motor control. Nat. Neurosci. Rev. 5, 534-546 (2004).
46. Kording, K. & Wolpert, O. The loss function of sensorimotor learning. Proc. Natl. Acad. Sci. USA 101, 9839-9842 (2004).
47. Lacquaniti, F., Terzuolo, C. & Viviani, P. The law relating the kinematic and figural aspects of drawing movements. Acta Psychol. 54, 115-130 (1983).
48. Sutton, G.G. & Sykes, K. The variation of hand tremor with force in healthy subjects. J. Physiol. (Lond.) 191, 699-711 (1967).
49. Schmidt. R.A., Zelaznik, H., Hawkins, B., Frank, J.S. & Quinn, J.T. Jr. Motor-output variability: a theory for the accuracy of rapid notor acts. Psychol. Rev. 86, 415-451 (1979).
50. Sabes, P.N., Jordan, M.I. & Wolpert, D.M. The role of inertial sensitivity in motor planning. J. Neurosci. 18, 5948-5957 (1998).
51. Rosenbaum, D.A. Human Motor Control (Academic, San Diego, 1991).
52. Burdet, E., Osu, R., Franklin, D.W., Milner, T.E. & Kawato, M. The central nervous system stabilizes unstable dynamics by learning optimal impedance. Nature 414, 446-449 (2001).
53. Hoffman, D.S. & Strick, P.L. Step-tracking movements of the wrist. IV. Muscle activity associated with movements in different directions. J. Neurophysiol. 81, 319-333 (1999).
54. Zhu, G., Rotea, M.A. & Skelton, R. A convergent algorithm for the output covariance constraint control problem. Siam J. Control Optimization 35, 341-361 (1997).
55. Bryson, A. & Ho, Y. Applied Optimal Control (Blaisdell, Waltham, Massachusetts, 1969).
56. Kirk, D. Optimal Control Theory: An Introduction (Prentice Hall, Englewood Cliffs, New Jersey, 1970).
57. Davis, M.H. & Vinter, R.B. Stochastic Modelling and Control (Chapman and Hall, London, 1985).
58. Bertsekas, D. & Tsitsiklis, J. Neuro-Dynamic Programming (Athena Scientific, Belmont, Massachusetts, 1997).
59. Sutton, R.S. & Barto, A.G. Reinforcement Learning: An Introduction (MIT Press, Cambridge, Massachusetts, 1998).
60. Todorov, E. & Li, W. Optimal control methods suitable for biomechanical systems. in Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Press, ISSN 1094-687X, 2003).
61. st International Conference on Informatics in Control, Automation, and Robotics (Kluwer, in the press).
62. Montague, P.R., Dayan, P., Person, C. & Sejnowski, T.J. Bee foraging in uncertain environments using predictive hebbian learning. Nature 377, 725-728 (1995).
63. Schultz, W., Dayan, P. & Montague, P.R. A neural substrate of prediction and reward. Science 275, 1593-1599 (1997).
64. O'Doherty, J. et al. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452-454 (2004).
65. Seymour, B. et al. Temporal difference models describe higher-order learning in humans. Nature 429, 664-667 (2004).
66. Knill, D. & Richards, W. Perception as Bayesian Inference (Cambridge Univ. Press, 1996).
67. Wolpert, D., Gharahmani, Z. & Jordan, M. An internal model for sensorimotor integration. Science 269, 1880-1882 (1995).
68. Kording, K.P. & Wolpert, D.M. Bayesian integration in sensorimotor learning. Nature 427, 244-247 (2004).
69. Saunders, J.A. & Knill, D.C. Visual feedback control of hand movements. J. Neurosci. 24, 3223-3234 (2004).
70. Shadmehr, R. & Mussa-Ivaldi, F.A. Adaptive representation of dynamics during learning of a motor task. J. Neurosci. 14, 3208-3224 (1994).
71. Flanagan, J.R. & Wing, A.M. The role of internal models in motion planning and control: Evidence from grip force adjustments during movements of hand-held loads. J. Neurosci. 17, 1519-1528 (1997).
72. Kawato, M. Internal models for motor control and trajectory planning, Curr. Opin. Neurobiol. 9, 718-727 (1999).
73. Flanagan, J.R. & Lolley, S. The inertial anisotropy of the arm is accurately predicted during movement planning. J. Neurosci. 21, 1361-1369 (2001).
74. Li, C.S., Padoa-Schioppa, C. & Bizzi, E. Neuronal correlates of motor performance and motor learning in the primary motor cortex of monkeys adapting to an external force field. Neuron 30, 593-607 (2001).
75. Gribble, P.L. & Scott, S.H. Overlap of internal models in motor cortex for mechanical loads during reaching. Nature 417, 938-941 (2002).
76. Ostry, D.J. & Feldman, A.G. A critical evaluation of the force control hypothesis in motor control. Exp. Brain Res. 153, 275-288 (2003).
77. Thoroughman, K.A. & Shadmehr, R. Learning of action through adaptive combination of motor primitives. Nature 407, 742-747 (2000).
78. Wang, T., Dordevic, G.S. & Shadmehr, R. Learning the dynamics of reaching movements results in the modification of arm impedance and long-latency perturbation responses. Biol. Cybern. 85, 437-448 (2001).
79. Flanagan, J.R., Vetter, P., Johansson, R.S. & Wolpert, D.M. Prediction precedes control in motor learning. Curr. Biol. 13, 146-150 (2003).
80. Korenberg, A. Computational and Psychophysical Studies of Motor Learning. PhD Thesis, University College London (2003).
81. Desmurget, M. & Grafton, S. Forward modeling allows feedback control for fast reaching movements. Trends Cogn. Sci. 4, 423-431 (2000).
82. Gribble, P.L. & Ostry, D.J. Compensation for interaction torques during single- and multijoint limb movement. J. Neurophysiol. 82, 2310-2326 (1999).
83. Humphrey, D.R. & Reed, D.J. in Advances in Neurology: Motor Control Mechanisms in Health and Disease (ed. Desmedt, J.E) 347-372 (Raven, New York, 1983).
84. Conditt, M.A. & Mussa-Ivaldi, F.A. Central representation of time during motor learning. Proc. Natl. Acad. Sci. USA 96, 11625-11630 (1999).
85. Woodworth, R.S. The accuracy of voluntary movement. Psychol. Rev. Monogr. 3 (1899).
86. Fitts, P. The information capacity of the human motor system in controlling the amplitude of movements. J. Exp. Psychol. 47, 381-391 (1954).
87. Meyer, D.E., Abrams, R.A., Kornblum, S., Wright, C.E. & Smith, J.E.K. Optimality in human motor performance: Ideal control of rapid aimed movements. Psychol. Rev. 95, 340-370 (1988).
88. Milner, T.E. & Ijaz, M.M. The effect of accuracy constraints on three-dimensional movement kinematics. Neuroscience 35, 365-374 (1990).
89. Flash, T. & Henis, E. Arm trajectory modifications during reaching towards visual targets. J. Cogn. Neurosci. 3, 220-230 (1991).
90. Komilis, E., Pelisson, D. & Prablanc, C. Error processing in pointing at randomly feedback-induced double-step stimuli. J. Motor Behav. 25, 299-308 (1993).
91. Todorov, E. & Jordan, M. in Advances in Neural Information Processing Systems vol. 15 (eds. Becker, S., Thrun, S. & Obermayer, K.) 27-34 (MIT Press, Cambridge, Massachusetts, 2002).
92. Cole, K.J. & Abbs, J.H. Kinematic and electromyographic responses to perturbation of a rapid grasp. J. Neurophysiol. 57, 1498-1510 (1987).
93. Gracco, V.L. & Abbs, J.H. Dynamic control of the perioral system during speech: kinematic analyses of autogenic and nonautogenic sensorimotor processes. J. Neurophysiol. 54, 418-432 (1985).
94. Robertson, E.M. & Miall, R.C. Multi-joint limbs permit a flexible response to unpredictable events. Exp. Brain Res. 117, 148-152 (1997).
95. Scholz, J.P. & Schoner, G. The uncontrolled manifold concept: identifying control variables for a functional task. Exp. Brain Res. 126, 289-306 (1999).
96. Scholz, J.P., Schoner, G. & Latash, M.L. Identifying the control structure of multijoint coordination during pistol shooting. Exp. Brain Res. 135, 382-404 (2000).
97. Li, Z.M., Latash, M.L. & Zatsiorsky, V.M. Force sharing among fingers as a model of the redundancy problem. Exp. Brain Res. 119, 276-286 (1998).
98. Gordon, J., Ghilardi, M.F., Cooper, S. & Ghez, C. Accuracy of planar reaching movements. II. Systematic extent errors resulting from inertial anisotropy. Exp. Brain Res. 99,112-130 (1994).
99. Messier, J. & Kalaska, J.F. Comparison of variability of initial kinematics and end-points of reaching movements. Exp. Brain Res. 125, 139-152 (1999).
100. D'Avella, A., Saltiel, P. & Bizzi, E. Combinations of muscle synergies in the construction of a natural motor behavior. Nat. Neurosci. 6, 300-308 (2003).
101. Ivanenko, Y.P. et al. Temporal components of the motor patterns expressed by the human spinal cord reflect foot kinematics. J. Neurophysiol. 90, 3555-3565 (2003).
102. Latash, M.L. in Motor Control, Today and Tomorrow (eds. Gantchev, G., Mori, S. & Massion, J.) 181-196 (Academic Publishing House "Prof. M. Drinov", Sofia, 1999).
103. Nicols, T.R. & Houk, J.C. Improvement in linearity and regulations of stiffness that result from actions of stretch reflex. J. Neurophysiol. 39, 119-142 (1976).
104. Bernstein, N.I. On the Construction of Movements (Medgiz, Moscow, 1947).
105. Bernstein, N.I. in Dexterity and its Development (eds. Latash, M.L. & Turvey, M.) 97-170 (Lawrence Erlbaum, Mahwah, New Jersey, 1996).
106. Cisek, P., Grossberg, S. & Bullock, D. A cortico-spinal model of reaching and proprioception under multiple task constraints. J. Cogn. Neurosci. 10, 425-444 (1998).
107. Barlow, H. in Sensory Communication (ed. Rosenblith, W.) 217-234 (MIT Press, Cambridge, Massachusetts, 1961).
108. Olshausen, B.A. & Field, D.J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607-609 (1996).
109. Lewicki, M.S. Efficient coding of natural sounds. Nat. Neurosci. 5, 356-363 (2002).
110. Khatib, O. A unified approach to motion and force control of robotic manipulators: The operational space formulation. IEEE J. Robotics Automation RA-3, 43-53 (1987).
111. Pratt, J., Chew, C.M., Torres, A., Dilworth, P. & Pratt, G. Virtual model control: An intuitive approach for bipedal locomotion. Intl. J. Robotics Res. 20, 129-143 (2001).
112. Loeb, G.E., Peck, R.A., Moore, W.H. & Hook, K. BION system for distributed neural prosthetic interfaces. Med. Eng. Physics 23, 9-18 (2001).
113. Hinton, G.E. Parallel computations for controlling an arm. J. Motor Behav. 16, 171-194 (1984).
114. Pellionisz, A. Coordination: a vector-matrix description of transformations of over-complete CNS coordinates and a tensorial solution using the Moore-Penrose generalized inverse. J. Theoret. Biol. 110, 353-375 (1984).
115. Bullock, D. & Grossberg, S. Neural dynamics of planned arm movements: emergent invariants and speed-accuracy properties during trajectory formation. Psychol. Rev. 95, 49-90 (1988).
116. Torres, E.B. & Zipser, D. Reaching to grasp with a multi-jointed arm. I. Computational model. J. Neurophysiol. 88, 2355-2367 (2002).
117. Camacho, E.F. & Bordons, C. Model Predictive Control (Springer, London, 1999).
118. Bizzi, E., Accornero, N., Chapple, W. & Hogan, N. Posture control and trajectory formation during arm movement. J. Neurosci. 4, 2738-2744 (1984).
119. Todorov, E. Direct cortical control of muscle activation in voluntary arm movements: a model. Nat. Neurosci. 3, 391-398 (2000).