The neural optimal control hierarchy for motor control

Journal of Neural Engineering

Volumn 8, Issue 6, 2011, Pages

  DeWolf, T ^a   Eliasmith, C ^a  

^a UNIVERSITY OF WATERLOO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ANATOMICAL ELEMENTS; BASAL GANGLIA; BIOLOGICAL MOTOR CONTROL; BIOLOGICAL MOTORS; CLINICAL DATA; CORTICAL AREAS; HUNTINGTON'S DISEASE; MODEL-BASED OPC; MOTOR BEHAVIOR; MOTOR CONTROL; NEURAL STRUCTURES; OPTIMAL CONTROLS; PLAUSIBLE MODEL; QUANTITATIVE CHARACTERIZATION; SPIKING NEURON; THEORETIC MODEL;

CONTROL; NEUROPHYSIOLOGY; OPTIMIZATION;

BEHAVIORAL RESEARCH;

ARM MOVEMENT; BASAL GANGLION; BELLMAN EQUATION; CEREBELLUM INJURY; CONCEPTUAL FRAMEWORK; DEGREE OF FREEDOM; HUMAN; HUNTINGTON CHOREA; KERNEL METHOD; MATHEMATICAL ANALYSIS; MOTOR CONTROL; MOTOR COORDINATION; MOTOR CORTEX; MOTOR DYSFUNCTION; NERVE CELL; NEURAL OPTIMAL CONTROL HIERARCHY FRAMEWORK; NONHUMAN; PRIORITY JOURNAL; REVIEW; SENSORY FEEDBACK; SIMULATION; THEORETICAL MODEL; VELOCITY; WALKING; WHITE NOISE; ACTION POTENTIAL; ARTICLE; BIOLOGICAL MODEL; EVOKED MUSCLE RESPONSE; PHYSIOLOGY;

PRIMATES;

ACTION POTENTIALS; EVOKED POTENTIALS, MOTOR; MODELS, NEUROLOGICAL; MOTOR CORTEX; NEURONS;

EID: 81855221557     PISSN: 17412560     EISSN: 17412552     Source Type: Journal    
DOI: 10.1088/1741-2560/8/6/065009     Document Type: Review

References (93)

1. Lorinez A 1997 Common control principles of basal ganglia - thalamocortical loops and the hippocampus Neural Netw. World 1-30
2. Abdelghani M N 2011 The role of sensitivity derivatives in sensorimotor learning PhD Thesis University of Toronto
3. Andani M E, Bahrami F, Jabehdar Maralani P and Ijspeert A J 2009 MODEM: a multi-agent hierarchical structure to model the human motor control system Biol. Cybern. 101 361-77
4. Athans M 1971 The role and use of the stochastic linear-quadratic- Gaussian problem in control system design Trans. Autom. Control 16 529-52
5. Bar-Gad I, Morris G and Bergman H 2003 Information processing, dimensionality reduction and reinforcement learning in the basal ganglia Prog. Neurobiol. 71 439-73 (Pubitemid 38167597)
6. Bastian A 2006 Learning to predict the future: the cerebellum adapts feedforward movement control Curr. Opin. Neurobiol. 16 645-9 (Pubitemid 44791921)
7. Berniker M, Jarc A, Bizzi E and Tresch M C 2009 Simplified and effective motor control based on muscle synergies to exploit musculoskeletal dynamics Proc. Natl Acad. Sci. USA 106 7601-6
8. Billard A, Epars Y, Calinon S, Schaal S and Cheng G 2004 Discovering optimal imitation strategies Robot. Auton. Syst. 47 69-77
9. Bloxham C A, Mindel T A and Frith C D 1984 Initiation and execution of predictable and unpredictable movements in Parkinson's disease Brain 107 371-84 (Pubitemid 14108826)
10. Blumenfield H 2002 Neuroanatomy Through Clinical Cases (Sunderland, MA: Sinauer Associates)
11. Bortoff G and Strick P L 1993 Corticospinal terminations in two new-world primates: further evidence that corticomotoneuronal connections provide part of the neural substrate for manual dexterity J. Neurosci. 13 5105-18 (Pubitemid 24000997)
12. Bostan A C, Dum R P and Strick P L 2010 The basal ganglia communicate with the cerebellum Proc. Natl Acad. Sci. USA 107 8452-6
13. Caligiore D, Ferrauto T, Parisi D, Accornero N, Capozza M and Baldassarre G 2008 Using motor babbling and Hebb rules for modeling the development of reaching with obstacles and grasping Int. Conf. on Cognitive Systems pp E1-8
14. Chan S S and Moran D W 2006 Computational model of a primate arm: from hand position to joint angles, joint torques and muscle forces J. Neural Eng. 3 327-37
15. Churchland M M, Cunningham J P, Kaufman M T, Ryu S I and Shenoy K V 2010 Cortical preparatory activity: representation of movement or first COG in a dynamical machine? Neuron 68 387-400
16. Crespi A and Ijspeert A J 2009 Salamandra robotica: a biologically inspired amphibious robot that swims and walks Robotics pp 3-36
17. da Silva M, Durand F and Popovic J 2009 Linear bellman combination for control of character animation Proc. SIGGRAPH ACM Trans. Graph. 28 82:1-82:10
18. Demiris Y and Meltzoff A 2008 The robot in the crib: a developmental analysis of imitation skills in infants and robots Infant Child Dev. 53 43-53
19. Deneve S and Pouget A 2003 Basis functions for object-centered representations Neuron 37 347-59 (Pubitemid 36143828)
20. DeWolf T 2010 NOCH: a framework for biologically plausible models of neural motor control PhD Thesis University of Waterloo
21. Doya K 1999 What are the computations of the cerebellum, the basal ganglia and the cerebral cortex Neural Netw. 12 961-74 (Pubitemid 29389710)
22. Doya K 2009 How can we learn efficiently to act optimally and flexibly? Proc. Natl Acad. Sci. USA 106 11429-30
23. Eliasmith C and Anderson C H 2003 Neural Engineering: Computation, Representation and Dynamics in Neurobiological Systems (Cambridge, MA: MIT Press)
24. Felleman D J and Van Essen D C 1991 Distributed hierarchical processing in the primate cerebral cortex Cereb. Cortex 1 1-47
25. Fishbach A, Roy S A, Bastianen C, Miller L E and Houk J C 2007 Deciding when and how to correct a movement: discrete submovements as a decision making process Exp. Brain Res. 177 45-63 (Pubitemid 46197828)
26. Fix J 2007 Atlas of the Human Brain and Spinal Cord (Jones & Bartlett Learning)
27. Fried I, Katz A, McCarthy G, Sass K J, Williamson P, Spencer S S and Spencer D D 1991 Functional organization of human supplementary motor cortex studied by electrical stimulation J. Neurosci. 11 3656-66
28. Gail A and Andersen R A 2006 Neural dynamics in monkey parietal reach region reflect context-specific sensorimotor transformations J. Neurosci. 26 9376-84 (Pubitemid 44396850)
29. Gentilucci M and Negrotti A 1999 The control of an action in Parkinson's disease Exp. Brain Res. 269-77 (Pubitemid 29525250)
30. Georgopoulos A P, Schwartz A B and Kettner R E 1986 Neuronal population coding of movement direction Science 233 1416-9 (Pubitemid 17169374)
31. Gershwin S B 1987 A hierarchical framework for discrete event scheduling in manufacturing systems Discrete Event Systems: Models and Application (Lecture Notes in Computer Science vol 103) ed P Varaiya and H Kurzhanski (Berlin: Springer) pp 197-216
32. Ghez C and Thach T 2000 The Cerebellum 4th edn (New York: McGraw-Hill) chapter 42
33. Ghez C and Thach T 2000 The Organization of Movement 4th edn (New York: McGraw-Hill) chapter 33
34. Gluck M A, Allen M T, Myers C E and Thompson R F 2001 Cerebellar substrates for error correction in motor conditioning Neurobiol. Learn. Mem. 76 314-41 (Pubitemid 34001404)
35. Gopnik A, Meltzoff A N and Kuhl P K 2000 The Scientist in the Crib: What Early Learning Tells us About the Mind (Perennial)
36. Guibas L, Sharir M and Sifrony S 1988 On the general motion planning problem with two degrees of freedom Proc. 4th Annu. Symp. on Computational Geometry pp 289-98
37. Gurney K, Prescott T J and Redgrave P 2001 A computational model of action selection in the basal ganglia: I. A new functional anatomy Biol. Cybern. 84 401-10 (Pubitemid 33605190)
38. Harper P S 1992 Review Article: the epidemiology of Huntington's disease Hum. Genet. 365-76
39. Haruno M, Wolpert D M and Kawato M 2001 Mosaic model for sensorimotor learning and control Neural Comput. 13 2201-20 (Pubitemid 33676881)
40. Hinton G E and Salakhutdinov R R 2006 Reducing the dimensionality of data with neural networks Science 313 504-7 (Pubitemid 44148451)
41. Hoffman D S and Strick P L 1995 Effects of a primary motor cortex lesion on step-tracking movements of the wrist J. Neurophysiol. 73 891-5
42. Hooper S L and DiCaprio R A 2004 Crustacean motor pattern generator networks Neuro-Signals 13 50-69 (Pubitemid 38324530)
43. Hoshi E, Tremblay L, Féger J, Carras P L and Strick P L 2005 The cerebellum communicates with the basal ganglia Nat. Neurosci. 8 1491-3 (Pubitemid 41573660)
44. Hyvärinen A and Hoyer P 1999 Emergence of topography and complex cell properties from natural images using extensions of ICA Advances in Neural Information Processing Systems (Cambridge, MA: MIT Press) pp 827-33
45. Ivashko D G, Prilutsky B I, Markin S N, Chapin J K and Rybak I A 2003 Modeling the spinal cord neural circuitry controlling cat hindlimb movement during locomotion Neurocomputing 52-4 621-9 (Pubitemid 36700564)
46. Ting L and Lucas Mckay J 2010 Muscle synergies produce expensive behavioral biases during postural control Neural Control of Movement 2010 pp 14-5
47. Jung R, Kiemel T and Cohen A H 1996 Dynamic behavior of a neural network model of locomotor control in the lamprey J. Neurophysiol. 75 1074-86 (Pubitemid 26087475)
48. Kalaska J F, Cohen D A, Hyde M L and Prud'homme M 1989 A comparison of movement direction-related versus load direction-related activity in primate motor cortex, using a two-dimensional reaching task J. Neurosci. 9 2080-102 (Pubitemid 19156995)
49. Kawato M 1999 Internal models for motor control and trajectory planning Curr. Opin. Neurobiol. 9 718-27
50. Kleinman D L 1970 An optimal control model of human response: part I. Theory and validation Automatica 3 357-69
51. Kuroda S, Yamamoto K, Miyamoto H, Doya K and Kawat M 2001 Statistical characteristics of climbing fiber spikes necessary for efficient cerebellar learning Biol. Cybern. 84 183-92 (Pubitemid 33605172)
52. Lee H, Ekanadham C and Ng A 2007 Sparse deep belief net model for visual area v2 Energy 19 1-8
53. Lemay M, Chouinard S, Richer F and Lesperance P 2008 Huntington's disease affects movement termination Behav. Brain Res. 187 153-8 (Pubitemid 350216202)
54. Li W, Todorov E and Pan X 2005 Hierarchical feedback and learning for multi-joint arm movement control IEEE Eng. Med. Biol. Soc. 4 4400-3 (Pubitemid 46229915)
55. Liu D and Todorov E 2009 Hierarchical optimal control of a 7-dof arm model 2009 IEEE Symp. on Adaptive Dynamic Programming and Reinforcement Learning pp 50-7
56. Maier M A, Olivier E, Baker S N, Kirkwood P A, Morris T and Lemon R N 1997 Direct and indirect corticospinal control of arm and hand motoneurons in the squirrel monkey (saimiri sciureus) J. Neurophysiol. 78 721-33 (Pubitemid 27353538)
57. McKay N 1985 Minimum cost-control of robotic manipulators with geometric path constraints PhD Thesis University of Michigan
58. Meltzoff A and Keith Moore M 1997 Explaining facial imitation: a theoretical model Early Dev. Parenting 6 179-92 (Pubitemid 127527083)
59. Morari M, Garcia C E and Prett D M 1989 Model predictive control: theory and practice - a survey Automatica 25 335-48 (Pubitemid 20612627)
60. Morton S M and Bastian A J 2004 Cerebellar control of balance and locomotion Neuroscientist 10 247-59 (Pubitemid 38657091)
61. Mulliken G H, Musallam S and Andersen R A 2008 Decoding trajectories from posterior parietal cortex ensembles J. Neurosci. 28 12913-26
62. Nenadic Z, Anderson C H and Ghosh B 2000 Control of arm movement using population of neurons Proc. 39th IEEE Conf. on Decision and Control vol 35 pp 1776-81
63. Olshausen B A and Field D J 1996 Emergence of simple-cell receptive field properties by learning a sparse code for natural images Nature 381 607-9 (Pubitemid 26177476)
64. Olshausen B A 2002 Sparse Codes and Spikes (Cambridge, MA: MIT Press) pp 257-72
65. Penfield W and Rasmussen T 1950 The Cerebral Cortex of Man vol 40 (London: Macmillan)
66. Peters J and Schaal S 2007 Reinforcement learning by reward-weighted regression for operational space control Proc. 24th Int. Conf. on Machine Learning pp 745-50
67. Piaget J 1952 The Origins of Intelligence in Children (New York: International University Press)
68. Pouget A and Sejnowski T J 1997 Spatial transformations in the parietal cortex using basis functons J. Cogn. Neurosci. 9 222-37 (Pubitemid 27192694)
69. Riehle A and Vaadia E (ed) 2005 Motor Cortex in Voluntary Movements: A Distributed System for Distributed Functions (Boca Raton, FL: CRC Press)
70. Sanes J N and Donoghue J P 2000 Plasticity and primary motor cortex Annu. Rev. Neurosci. 23 393-415 (Pubitemid 30350048)
71. Schwartz A B 1994 Direct cortical representation of drawing Science 265 540-2 (Pubitemid 24265003)
72. Schwartz A B, Tracy Cui X, Weber D J and Moran D W 2006 Brain-controlled interfaces: movement restoration with neural prosthetics Neuron 52 205-20 (Pubitemid 44466359)
73. Scott S H 2004 Optimal feedback control and the neural basis of volitional motor control Nat. Rev. Neurosci. 5 532-46
74. Shenoy K V, Meeker D and Cao S 2003 Neural prosthetic control signal from plan activity Neuroreport 14 591-6 (Pubitemid 36528996)
75. Smith M A, Brandt J and Shadmehr R 2000 Motor disorder in Huntington's disease begins as a dysfunction in error feedback control Nature 403 544-9 (Pubitemid 30082199)
76. Souza A D, Vijayakumar S and Schaal S 2001 Learning inverse kinematics Proc. IROS pp 298-303
77. Stewart T C, Choo X and Eliasmith C 2010 Dynamic behaviour of a spiking model of action selection in the basal ganglia Int. Conf. on Cognitive Modeling pp 235-40
78. Sutton R S and Barto A G 1998 Reinforcement Learning - An Introduction (Cambridge, MA: MIT Press)
79. Tahara K, Arimoto S, Sekimoto M and Luo Z-W 2009 On control of reaching movements for musculo-skeletal redundant arm model Appl. Bionics Biomech. 6 11-26
80. Taira M, Boline J, Smyrnis N, Georgopoulos A and Ashe J 1996 On the relations between single cell activity in the motor cortex and the direction and magnitude of three-dimensional static isometric force Exp. Brain Res. 109 367-76 (Pubitemid 26188627)
81. Todorov E 2008 Parallels between sensory and motor information processing The Cognitive Neurosciences 4th edn (Cambridge, MA: MIT Press)
82. Todorov E, Li W and Pan X 2005 From task parameters to motor synergies: a hierarchical framework for approximately-optimal feedback control of redundant manipulators J. Robot. Syst. 22 691-710 (Pubitemid 41684629)
83. Todorov E 2003 On the role of primary motor cortex in arm movement control Cogn. Sci. 3 125-66
84. Todorov E 2009 Efficient computation of optimal actions Proc. Natl Acad. Sci. USA 106 11478-83
85. Todorov E 2006 Optimal Control Theory (Cambridge, MA: MIT Press) chapter 12
86. Trinkle J C, Abel J M and Paul R P 1988 An investigation of frictionless enveloping grasping in the plane Int. J. Robot. Res. 7 33-51
87. Tripp B 2009 A search for principles of basal ganglia function PhD Thesis University of Waterloo
88. Turella L, Pierno A C, Tubaldi F and Castiello U 2009 Mirror neurons in humans: consisting or confounding evidence? Brain Lang. 108 10-21
89. Turner R S, Desmurget M, Turner C and S. R 2010 Basal ganglia contributions to motor control: a vigorous tutor Curr. Opin. Neurobiol. 20 704-16
90. Valero-Cuevas F J, Venkadesan M and Todorov E 2009 Structured variability of muscle activations supports the minimal intervention principle of motor control J. Neurophysiol. 102 59-68
91. Velliste M, Perel S, Spalding M C, Whitford A S and Schwartz A B 2008 Cortical control of a prosthetic arm for self-feeding Nature 453 1098-101 (Pubitemid 351871735)
92. Wing A M 2000 Motor control: mechanisms of motor equivalence in handwriting Curr. Biol. 10 R245-8 (Pubitemid 30186130)
93. Wolpert D M, Miall R C and Kawato M 1998 Internal models in the cerebellum Trends Cogn. Sci. 2 338-47 (Pubitemid 28396569)