Computational motor control in humans and robots

Current Opinion in Neurobiology

Volumn 15, Issue 6, 2005, Pages 675-682

  Schaal, Stefan ^a   Schweighofer, Nicolas ^a  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; BAYES THEOREM; BEHAVIOR; BRAIN FUNCTION; CEREBELLUM; DECISION MAKING; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; LEARNING; MOTOR CONTROL; MOTOR CORTEX; MOTOR PERFORMANCE; NOISE; PRIORITY JOURNAL; REVIEW; TASK PERFORMANCE;

ALGORITHMS; ANIMALS; BAYES THEOREM; BRAIN; COMPUTERS; HUMANS; LEARNING; MODELS, NEUROLOGICAL; MOVEMENT; ROBOTICS; STOCHASTIC PROCESSES;

EID: 27944433769     PISSN: 09594388     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.conb.2005.10.009     Document Type: Review

References (79)

1. N. Wiener Cybernetics 1948 J. Wiley
2. R. Bellman Dynamic programming 1957 Princeton University Press
3. K.S. Narendra, and A.M. Annaswamy Stable adaptive systems 2005 Dover Publications
4. J.J.E. Slotine, and W. Li Applied Nonlinear Control 1991 Prentice Hall
5. R. Shadmehr, and S.P. Wise The Computational Neurobiology of Reaching and Pointing: A Foundation for Motor Learning 2005 MIT Press This book is currently the most comprehensive and modern summary of computational motor control for arm movements, with many tutorials that enable less computationally trained readers to quickly understand the topics of state-of-the-art research.
6. R.B. Stein, M.N. Ogusztöreli, and C. Capaday What is optimized in muscular movements? N.L. Jones N. McCartney A.J. McComas Human Muscle Power 1986 Human Kinetics Publisher 131 150
7. R.C. Miall, D.J. Weir, D.M. Wolpert, and J.F. Stein Is the cerebellum a Smith predictor? J Mot Behav 25 1993 203 216
8. B. Mehta, and S. Schaal Forward models in visuomotor control J Neurophysiol 88 2002 942 953
9. D.A. Robinson The use of control systems analysis in the neurophysiology of eye movements Annu Rev Neurosci 4 1981 463 503
10. S.G. Lisberger The neural basis for motor learning in the vestibulo-ocular reflex in monkeys Trends Neurosci 11 1988 147 152
11. P.N. Sabes The planning and control of reaching movements Curr Opin Neurobiol 10 2000 740 746
12. M. Kawato Internal models for motor control and trajectory planning Curr Opin Neurobiol 9 1999 718 727
13. R.A. Jacobs, M.I. Jordan, S.J. Nowlan, and G.E. Hinton Adaptive mixtures of local experts Neural Comput 3 1991 79 87
14. D.M. Wolpert, and M. Kawato Multiple paired forward and inverse models for motor control Neural Netw 11 1998 1317 1329
15. K. Doya, K. Samejima, K. Katagiri, and M. Kawato Multiple model-based reinforcement learning Neural Comput 14 2002 1347 1369
16. H. Imamizu, T. Kuroda, S. Miyauchi, T. Yoshioka, and M. Kawato Modular organization of internal models of tools in the human cerebellum Proc Natl Acad Sci USA 100 2003 5461 5466
17. R. Paz, T. Boraud, C. Natan, H. Bergman, and E. Vaadia Preparatory activity in motor cortex reflects learning of local visuomotor skills Nat Neurosci 6 2003 882 890
18. H. Imamizu, T. Kuroda, T. Yoshioka, and M. Kawato Functional magnetic resonance imaging examination of two modular architectures for switching multiple internal models J Neurosci 24 2004 1173
19. I. Kurtzer, T.M. Herter, and S.H. Scott Random change in cortical load representation suggests distinct control of posture and movement Nat Neurosci 8 2005 498 504
20. E. Salinas Fast remapping of sensory stimuli onto motor actions on the basis of contextual modulation J Neurosci 24 2004 1113 1118
21. S.K. Wainscott, O. Donchin, and R. Shadmehr Internal models and contextual cues: Encoding serial order and direction of movement J Neurophysiol 93 2005 786 800 The authors developed a reaching task in which forces on the hand depended both on the direction of the movement and on the order of the movement in a pre-defined sequence (a contextual cue). It is demonstrated that the contextual cue multiplicatively modulated the basis functions that form the inverse dynamics internal model.
22. C.M. Harris, and D.M. Wolpert Signal-dependent noise determines motor planning Nature 394 1998 780 784
23. E. Todorov, and M.I. Jordan Optimal feedback control as a theory of motor coordination Nat Neurosci 5 2002 1226 1235
24. M. Haruno, and D.M. Wolpert Optimal control of redundant muscles in step-tracking wrist movements Journal of Neurophysiology 2005 Epub ahead of print
25. A.F.D. Hamilton, K.E. Jones, and D.M. Wolpert The scaling of motor noise with muscle strength and motor unit number in humans Exp Brain Res 157 2004 417 430 The authors showed experimentally that using proximal joints during movements might help to decrease end-point variability. Furthermore, simulations demonstrated that the reduced noise in larger muscles is due to the fact that larger muscles have more motor units: thus, neuronal redundancy reduces variability.
26. D.J. Reinkensmeyer, M.G. Iobbi, L.E. Kahn, D.G. Kamper, and C.D. Takahashi Modeling reaching impairment after stroke using a population vector model of movement control that incorporates neural firing-rate variability Neural Comput 15 2003 2619 2642 The authors studied, both in simulation and experimentally with stroke patients, the effect of cortical neuron noise on the variability of reaching movement direction. Simulation results showed that the number of neurons controlling hand direction (via directional tuning) correlated with end-point errors. Further, experimental results demonstrated that stroke severity also correlated with directional variability. Thus, hypothesizing that stroke severity correlates with lesion size (which is far from being always true - see, Winstein et al. CSM abstract, J. Neurol. Phys. Ther. 2004 vol 28 4), it was concluded that the reduced number of surviving neurons in more affected patients causes greater end-point variability.
27. R.J. van Beers, P. Haggard, and D.M. Wolpert The role of execution noise in movement variability J Neurophysiol 91 2004 1050 To find out how different types of noise could best explain the variability of behavioral data, the authors fit the noise parameters in a model of arm reaching to data of human end-point reaching data. Their results show that most of the end-point variability could be explained by movement execution noise, with a combination of noise in the magnitude and noise in the duration of the motor output.
28. K.E. Jones, A.F. Hamilton, and D.M. Wolpert Sources of signal-dependent noise during isometric force production J Neurophysiol 88 2002 1533 1544
29. R. Osu, N. Kamimura, H. Iwasaki, E. Nakano, C.M. Harris, Y. Wada, and M. Kawato Optimal impedance control for task achievement in the presence of signal-dependent noise J Neurophysiol 92 2004 1199
30. D.W. Franklin, E. Burdet, K.P. Tee, R. Osu, M. Kawato, and T.E. Milner Feedback drives the learning of feedforward motor commands for subsequent movements (abstract) Society for Neuroscience Abstracts 2004 871.9
31. C.M. Bishop Neural Networks for Pattern Recognition 1995 Oxford University Press
32. K.A. Thoroughman, and R. Shadmehr Learning of action through adaptive combination of motor primitives Nature 407 2000 742 747
33. O. Donchin, J.T. Francis, and R. Shadmehr Quantifying generalization from trial-by-trial behavior of adaptive systems that learn with basis functions: theory and experiments in human motor control J Neurosci 23 2003 9032 9045
34. K.P. Körding, and D.M. Wolpert The loss function of sensorimotor learning Proc Natl Acad Sci USA 101 2004 9839 9842
35. N. Schweighofer, J. Spoelstra, M.A. Arbib, and M. Kawato Role of the cerebellum in reaching movements in humans. II. A neural model of the intermediate cerebellum Eur J Neurosci 10 1998 95 105
36. N. Schweighofer, K. Doya, H. Fukai, J.V. Chiron, T. Furukawa, and M. Kawato Chaos may enhance information transmission in the inferior olive Proc Natl Acad Sci USA 101 2004 4655 4660 The authors developed a realistic inferior olive network model and showed that moderate electrical coupling between cells produced chaotic firing, which maximized the input-output mutual information. This 'chaotic resonance' enables rich error signals to reach the Purkinje cells, even at low inferior olive neuron firing rates, and might enable efficient cerebellar learning.
37. T.D. Sanger Failure of motor learning for large initial errors Neural Comput 16 2004 1873 1886
38. R. Osu, S. Hirai, T. Yoshioka, and M. Kawato Random presentation enables subjects to adapt to two opposing forces on the hand Nat Neurosci 7 2004 111 112
39. T.D. Sanger Neural network learning control of robot manipulators using gradually increasing task difficulty IEEE Trans Rob Autom 10 1994 323 333
40. V. Ivanchenko, and R.A. Jacobs A developmental approach AIDS motor learning Neural Comput 15 2003 2051 2065
41. N.A. Bernstein The Control and Regulation of Movements 1967 Pergamon Press
42. J.P. Scholz, and G. Schoner The uncontrolled manifold concept: identifying control variables for a functional task Exp Brain Res 126 1999 289 306
43. O. Khatib A united approach to motion and force control of robot manipulators: The operational space formulation International Journal of Robotics Research 31 1987 43 53
44. Y. Nakamura, and H. Hanafusa Inverse kinematics solutions with singularity robustness for robot manipulator control J Dyn Syst Meas Control 108 1986 163 171
45. Nakanishi J, Cory R, Mistry M, Peters J, Schaal S: Comparative experiments on task space control with redundancy resolution. In IEEE International Conference on Intelligent Robots and Systems (IROS 2005): 2005.
46. Mistry M, Mohajerian P, Schaal S: Arm movement experiments with joint space force fields using an exoskeleton robot. In IEEE Ninth International Conference on Rehabilitation Robotics; Chicago, Illinois, June 28-July 1: 2005. By using an exoskeleton robot manipulandum, the authors perturbed the elbow joint of subjects with a force field during point-to-point reaching movements with the unconstrained arm. Subjects learned to compensate for this force field and return after training to the same endpoint trajectories as before training. Interestingly, however, joint space trajectories did not return to the original performance. These results indicate that for reaching tasks, no position reference trajectory was generated in joint space. Rather, the biological controller behaved like a velocity-based or acceleration-based operational space controller with flexible resolution of redundancy in the null space of the task.
47. H. Cruse, E. Wischmeyer, M. Bruwer, P. Brockfeld, and A. Dress On the cost functions for the control of the human arm movement Biol Cybern 62 1990 519 528
48. K. Ohta, M.M. Svinin, Z.W. Luo, S. Hosoe, and R. Laboissiere Optimal trajectory formation of constrained human arm reaching movements Biol Cybern 91 2004 23 36
49. Bruyninckx H, Khatib O: Gauss' principle and the dynamics of redundant and constrained manipulators. In Proceedings of the 2000 IEEE International Conference on Robotics & Automation; San Francisco, CA, April 2000: 2000:2563-2568.
50. O. Khatib, L. Sentis, J. Park, and J. Warrent Whole-body dynamic behavior and control of human-like robots International Journal of Humanoid Robotics 1 2004 1 15 Based on more than a decade of research on task-level control in redundant control systems, the authors develop a comprehensive control approach of how multiple tasks can be executed in a hierarchy of priorities, for example, as in bringing a cup to the mouth while maintaining postural balance and avoiding obstacles. This task-space control approach is among the most elegant and computationally clean approaches in humanoid robotics.
51. S. Schaal Is imitation learning the route to humanoid robots? Trends Cogn Sci 3 1999 233 242
52. A. Fishbach, S.A. Roy, C. Bastianen, L.E. Miller, and J.C. Houk Kinematic properties of on-line error corrections in the monkey Exp Brain Res 164 2005 442 457
53. K.E. Novak, L.E. Miller, and J.C. Houk The use of overlapping submovements in the control of rapid hand movements Exp Brain Res 144 2002 351 364
54. B. Rohrer, S. Fasoli, H.I. Krebs, B. Volpe, W.R. Frontera, J. Stein, and N. Hogan Submovements grow larger, fewer, and more blended during stroke recovery Motor Control 8 2004 472 483
55. R. Sosnik, B. Hauptmann, A. Karni, and T. Flash When practice leads to co-articulation: the evolution of geometrically defined movement primitives Experimental Brain Research 156 2004 422 438
56. S. Schaal, D. Sternad, R. Osu, and M. Kawato Rhythmic movement is not discrete Nat Neurosci 7 2004 1136 1143 Subjects performed rhythmic and discrete wrist movements in a 4T fMRI scanner. The results demonstrated that rhythmic movement only activated rather few brain regions that can be classified as ipsilateral primary motor areas. Discrete movement, by contrast, activated a large number of additional brain areas, in particular premotor and parietal regions. Given that discrete movement activated much more brain circuitry than rhythmic movement, the study provided strong evidence that rhythmic movement does not seem to be composed of discrete strokes, because otherwise the same or more brain areas should be active during rhythmic movement than during discrete movement.
57. A. Ijspeert, J. Nakanishi, and S. Schaal Learning attractor landscapes for learning motor primitives S. Becker S. Thrun K. Obermayer Advances in Neural Information Processing Systems 15 2003 MIT Press 1547 1554 The authors model discrete and rhythmic movement with nonlinear differential equations that form stable point and limit cycle attractors, respectively. The novel finding of this modeling approach was that such equations can be trivially learned with optimization methods, such that even rather complex movements such as a tennis forehand can be represented by such differential equations. Thus, this modeling approach created one of the first computational bridges between dynamic systems models to motor control and optimization theories.
58. R.B. Ivry, and R.M. Spencer The neural representation of time Curr Opin Neurobiol 14 2004 225 232
59. P.A. Lewis, and R.C. Miall Distinct systems for automatic and cognitively controlled time measurement: evidence from neuroimaging Curr Opin Neurobiol 13 2003 250 255
60. P.N. Kugler, and M.T. Turvey Information, Natural Law, and The Self-Assembly of Rhythmic Movement 1987 Erlbaum
61. J.A.S. Kelso Dynamic Patterns: The Self-Organization of Brain and Behavior 1995 MIT Press
62. M.S. Graziano, C.S. Taylor, and T. Moore Probing cortical function with electrical stimulation Nat Neurosci 5 2002 921
63. M.C. Tresch, P. Saltiel, A. d'Avella, and E. Bizzi Coordination and localization in spinal motor systems Brain Res Brain Res Rev 40 2002 66 79
64. D.J.C. MacKay Information Theory, Inference, and Learning Algorithms 2003 Cambridge University Press
65. D. Kersten, and A. Yuille Bayesian models of object perception Curr Opin Neurobiol 13 2003 150 158
66. Y. Weiss, E.P. Simoncelli, and E.H. Adelson Motion illusions as optimal percepts Nat Neurosci 5 2002 598 604
67. K.P. Körding, and D.M. Wolpert Bayesian integration in sensorimotor learning Nature 427 2004 244 247 In a very elegant experimental design, the authors investigated the bias and variance of end-point errors in a point-to-point reaching task in a virtual environment with noisy feedback. Initially, subjects learned to compensate for a random virtual displacement of the feedback of their hand relative to the target, which introduced a 1cm displacement prior in their behavior. In a second experimental condition, subjects midway through the movement received noisy feedback of their finger position relative to the target. The question was how subjects would integrate this noisy feedback with the learned bias from the initial experimental condition. Interestingly, only information integration according to Bayes rule was able to account for the results.
68. T.A. Martin, J.G. Keating, H.P. Goodkin, A.J. Bastian, and W.T. Thach Throwing while looking through prisms. I. Focal olivocerebellar lesions impair adaptation Brain 119 1996 1183 1198
69. T.A. Martin, J.G. Keating, H.P. Goodkin, A.J. Bastian, and W.T. Thach Throwing while looking through prisms. II. Specificity and storage of multiple gaze-throw calibrations Brain 119 1996 1199 1211
70. K.P. Körding, S.P. Ku, and D.M. Wolpert Bayesian integration in force estimation J Neurophysiol 92 2004 3161 3165
71. S.J. Sober, and P.N. Sabes Flexible strategies for sensory integration during motor planning Nat Neurosci 8 2005 490 The authors demonstrate in a reaching task in a virtual environment that the relative weighting of visual and proprioceptive feedback depends both on the sensory modality of the target and on the information content of the visual feedback, and that these factors affect different stages of motor planning in different ways. This weighting seems to be chosen such that the errors arising from the transformation of sensory signals between coordinate frames are minimized.
72. D.C. Knill, and A. Pouget The Bayesian brain: the role of uncertainty in neural coding and computation Trends Neurosci 27 2004 712 The authors demonstrate that the high variability in the responses of cortical neurons can be used to generate probabilistic population codes that represent Gaussian and non Gaussian probability distributions. Moreover, these codes reduce a broad class of Bayesian inference, such as decision making and cue integration, to simple linear operations. This result holds both for realistic neuronal variability and for tuning curves of arbitrary shape and provides a biologically plausible mechanism for Bayesian inference in the brain.
73. J.A. Saunders, and D.C. Knill Humans use continuous visual feedback from the hand to control both the direction and distance of pointing movements Exp Brain Res 162 2005 458
74. Ng AY, Russell S: Algorithms for inverse reinforcement learning. In Proceedings of the Seventeenth International Conference on Machine Learning. Morgan Kauffmann; 2000:663-670.
75. R.A. Andersen, S. Musallam, and B. Pesaran Selecting the signals for a brain-machine interface Curr Opin Neurobiol 14 2004 720 726
76. J.R. Flanagan, and R.S. Johansson Action plans used in action observation Nature 424 2003 769 771 The authors demonstrate that in a block stacking task, the pattern of eye movements is approximately the same when the subjects perform the task themselves, or when the subjects are watching the same task as performed by a teacher, that is, eye movements anticipate the movement of the hand. Strikingly, such eye movement patterns change drastically when the subjects cannot observe the teacher's movement, that is, eye movements become purely reactive and much more random. The results indicate that eye movements provide information about the higher level motor program that steers purposeful action.
77. R.A. Andersen, J.W. Burdick, S. Musallam, B. Pesaran, and J.G. Cham Cognitive neural prosthetics Trends Cogn Sci 8 2004 486 493
78. M. Haruno, T. Kuroda, K. Doya, K. Toyama, M. Kimura, K. Samejima, H. Imamizu, and M. Kawato A neural correlate of reward-based behavioral learning in caudate nucleus: a functional magnetic resonance imaging study of a stochastic decision task J Neurosci 24 2004 1660 1665
79. S.C. Tanaka, K. Doya, G. Okada, K. Ueda, Y. Okamoto, and S. Yamawaki Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops Nat Neurosci 7 2004 887 893