A robot model of the basal ganglia: Behavior and intrinsic processing

Neural Networks

Volumn 19, Issue 1, 2006, Pages 31-61

  Prescott, Tony J ^a   Montes González, Fernando M ^b   Gurney, Kevin ^a   Humphries, Mark D ^a   Redgrave, Peter ^a  

^a UNIVERSITY OF SHEFFIELD   (United Kingdom)

^b UNIVERSIDAD VERACRUZANA   (Mexico)

Author keywords

Action selection; Basal ganglia; Behavior switching; Embodied computational neuroscience; Rat; Robot

Indexed keywords

AUTONOMOUS AGENTS; COMPETITION; COMPUTATIONAL METHODS; COMPUTER ARCHITECTURE; NEUROLOGY; ROBOTICS; SENSORS;

ACTION SELECTION; BASAL GANGLIA; COMPUTATIONAL MODELS;

ROBOTS;

ANIMAL BEHAVIOR; ARTICLE; BASAL GANGLION; BEHAVIOR; MATHEMATICAL MODEL; METHODOLOGY; MOTIVATION; NONHUMAN; PRIORITY JOURNAL; ROBOTICS; SENSORIMOTOR FUNCTION;

BASAL GANGLIA; BEHAVIOR; CEREBRAL CORTEX; MODELS, NEUROLOGICAL; MOVEMENT; PERCEPTION; ROBOTICS; SENSATION; SUBSTANTIA NIGRA; THALAMUS;

EID: 30744451448     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neunet.2005.06.049     Document Type: Article

References (130)

1. D. Akkal, P. Burbaud, J. Audin, and B. Bioulac Responses of substantia nigra pars reticulata neurons to intrastriatal D1 and D2 dopaminergic agonist injections in the rat Neuroscience Letters 213 1 1996 66 70
2. R.L. Albin, A.B. Young, and J.B. Penney The functional anatomy of basal ganglia disorders Trends in Neurosciences 12 10 1989 366 375
3. J.W. Aldridge, R.J. Anderson, and J.T. Murphy The role of the basal ganglia in controlling a movement initiated by a visually presented cue Brain Research 192 1 1980 3 16
4. J.W. Aldridge, and K.C. Berridge Coding of serial order by neostriatal neurons: a 'natural action' approach to movement sequence Journal of Neuroscience 18 7 1998 2777 2787
5. G.E. Alexander, and M.D. Crutcher Functional architecture of basal ganglia circuits: neural substrates of parallel processing Trends in Neurosciences 13 7 1990 266 271
6. J. Bar-Gad, G. Morris, and H. Bergman Information processing, dimensionality reduction and reinforcement learning in the basal ganglia Progress in Neurobiology 71 2003 439 473
7. G.W. Barlow Modal action patterns T.A. Sebeok How Animals Communicate 1977 Indiana University Press Bloomington, Indiana 98 134
8. M.A. Basso, and R.H. Wurtz Neuronal activity in substantia nigra pars reticulata during target selection Journal of Neuroscience 22 5 2002 1883 1894
9. D.G. Beiser, and J.C. Houk Model of cortical-basal ganglionic processing: Encoding the serial order of sensory events Journal of Neurophysiology 79 6 1998 3168 3188
10. D.E. Berlyne Conflict, Arousal, and Curiosity 1960 McGraw-Hill New York
11. D. Bindra An interpretation of the displacement phenomenon British Journal of Psychology 50 1969 263 268
12. S. Blomfield Arithmetical operations performed by nerve cells Brain Research 69 1 1974 115 124
13. J.W. Brown, D. Bullock, and S. Grossberg How laminar frontal cortex and basal ganglia circuits interact to control planned and reactive saccades Neural Networks 17 2004 471 510
14. J. Bryson Cross-paradigm analysis of autonomous agent architecture Journal of Experimental and Theoretical Artificial Intelligence 12 2 2000 165 189
15. C.V. Buhusi, and W.H. Meck Differential effects of methamphetamine and haloperidol on the control of an internal clock Behavioral Neuroscience 116 2 2002 291 297
16. R.M. Carelli, M. Wolske, and M.O. West Loss of lever press-related firing of rat striatal forelimb neurons after repeated sessions in a lever pressing task Journal Of Neuroscience 17 5 1997 1804 1814
17. J.H. Casseday, and E. Covey A neuroethological theory of the operation of the inferior colliculus Brain, Behavior, and Evolution 47 6 1996 311 336
18. G. Chevalier, and J.M. Deniau Spatio-temporal organization of a branched tecto-spinal/tecto-diencephalic neuronal system Neuroscience 12 2 1984 427 439
19. P. Colgan Animal Motivation 1989 Chapman and Hall London
20. J.L. Contrerasvidal, and G.E. Stelmach A neural model of basal ganglia-thalamocortical relations in normal and Parkinsonian movement Biological Cybernetics 73 5 1995 467 476
21. A.R. Cools Role of the neostriatal dopaminergic activity in sequencing and selecting behavioural strategies: facilitation of processes involved in selecting the best strategy in a stressful situation Behavioural Brain Research 1 1980 361 378
22. R.L. Cowan, and C.J. Wilson Spontaneous firing patterns and axonal projections of single corticostriatal neurons in the rat medial agranular cortex Journal of Neurophysiology 71 1 1994 17 32
23. H.C. Cromwell, and K.C. Berridge Implementation of action sequences by a neostriatal site - a lesion mapping study of grooming syntax Journal Of Neuroscience 16 10 1996 3444 3458
24. P. Dayan, and B.W. Balleine Reward, motivation, and reinforcement learning Neuron 36 2 2002 285 298
25. M.R. DeLong, G.E. Alexander, A.P. Georgopoulos, M.D. Crutcher, S.J. Mitchell, and R.T. Richardson Role of basal ganglia in limb movements Human Neurobiology 2 4 1984 235 244
26. D. Denny-Brown, and N. Yanagisawa The role of the basal ganglia in the initiation of movement M.D. Yahr The basal ganglia 1976 Raven Press New York 115 149
27. P.F. Dominey, and M.A. Arbib A cortico-subcortical model for generation of spatially accurate sequential saccades Cerebral Cortex 2 1992 153 175
28. P.F. Dominey, and D. Boussaoud Encoding behavioral context in recurrent networks of the fronto-striatal system: a simulation study Cognitive Brain Research 6 1997 53 65
29. K. Doya, P. Dayan, and M.E. Hasselmo Special Issue: Computational Models of Neuromodulation Neural Networks 15 2002 475 774
30. C.L. Elsinger, S.M. Rao, J.L. Zimbelman, N.C. Reynolds, K.A. Blindauer, and R.G. Hoffmann Neural basis for impaired time reproduction in Parkinson's disease: an fMRI study Journal of the International Neuropsychological Society 9 7 2003 1088 1098
31. J.-P. Ewert Neuroethology of releasing mechanisms: Prey-catching in toads Behavioral and Brain Sciences 10 1987 337 389
32. J.P. Ewert, H. Buxbaum-Conradi, F. Dreisvogt, M. Glagow, C. Merkel-Harff, and A. Rottgen Neural modulation of visuomotor functions underlying prey-catching behaviour in anurans: perception, attention, motor performance, learning Comparative Biochemistry and Physiology A-Molecular & Integrative Physiology 128 3 2001 417 461
33. J.C. Fentress Specific and nonspecific factors in the causation of behavior P.P.G. Bateson P.H. Klopfer Perspectives in Ethology 1973 Plenum Press New York
34. M.J. Frank, B. Loughry, and R.C. O'Reilly Interactions between frontal cortex and basal ganglia in working memory: a computational model Cognitive, Affective, and Behavioral Neuroscience 1 2 2001 137 160
35. T. Fukai Sequence generation in arbitrary temporal patterns from theta- nested gamma oscillations: a model of the basal ganglia- thalamo-cortical loops Neural Networks 12 7-8 1999 975 987
36. J. Ganssle, and M. Barr Embedded Systems Dictionary 2003 CMP Books Lawrence, KS
37. C.R. Gerfen, T.M. Engber, L.C. Mahan, Z. Susel, T.N. Chase, and F.J. Monsma D1 and D2 dopamine receptor regulated gene-expression of striatonigral and striatopallidal neurons Science 250 4986 1990 1429 1432
38. C.R. Gerfen, and C.J. Wilson The basal ganglia L.W. Swanson A. Bjorklund T. Hokfelt Handbook of Chemical Neuroanatomy, Vol 12: Integrated Systems of the CNS, Part III 1996 Elsevier Amsterdam 371 468
39. J. Gibbon, R.M. Church, and W.H. Meck Scalar timing in memory Annals of the New York Academy of Sciences 423 1984 52 77
40. A. Gillies, and G. Arbuthnott Computational models of the basal ganglia Movement Disorders 15 5 2000 762 770
41. B. Girard, V. Cuzin, A. Guillot, K.N. Gurney, and T.J. Prescott A basal ganglia inspired model of action selection evaluated in a robotic survival task Journal of Integrative Neuroscience 2 2 2003 179 200
42. A.A. Grace Phasic versus tonic dopamine release and the modulation of dopamine system responsivity - a hypothesis for the etiology of schizophrenia Neuroscience 41 1 1991 1 24
43. A.M. Graybiel The basal ganglia and chunking of action repertoires Neurobiology of Learning and Memory 70 1998 119 136
44. J.M. Gulley, A.E. Kosobud, and G.V. Rebec Behavior-related modulation of substantia nigra pars reticulata neurons in rats performing a conditioned reinforcement task Neuroscience 111 2 2002 337 349
45. J.M. Gulley, M. Kuwajima, E. Mayhill, and G.V. Rebec Behavior-related changes in the activity of substantia nigra pars reticulata neurons in freely moving rats Brain Research 845 1 1999 68 76
46. K. Gurney, T.J. Prescott, and P. Redgrave A computational model of action selection in the basal ganglia I. A new functional anatomy Biological Cybernetics 84 6 2001 401 410
47. K. Gurney, T.J. Prescott, and P. Redgrave A computational model of action selection in the basal ganglia II. Analysis and simulation of behaviour Biological Cybernetics 84 6 2001 411 423
48. K. Gurney, T.J. Prescott, J. Wickens, and P. Redgrave Computational models of the basal ganglia: from membranes to robots Trends in Neurosciences 27 2004 453 459
49. K.N. Gurney, M. Humphries, R. Wood, T.J. Prescott, and P. Redgrave Testing computational hypotheses of brain systems function: a case study with the basal ganglia Network 15 4 2004 263 290
50. A. Handel, and P.W. Glimcher Quantitative analysis of substantia nigra pars reticulata activity during a visually guided saccade task Journal of Neurophysiology 82 6 1999 3458 3475
51. L.G. Harsing, and M.J. Zigmond Influence of dopamine on GABA release in striatum: Evidence for D-1-D-2 interactions and non-synaptic influences Neuroscience 77 2 1997 419 429
52. O. Hikosaka Role of basal ganglia in control of innate movements, learned behavior and cognition - a hypothesis G. Percheron J.S. McKenzie J. Feger The Basal Ganglia IV 1994 Plenum New York
53. O. Hikosaka, and R.H. Wurtz The basal ganglia E.H. Wurtz M.E. Goldberg The Neurobiology of Saccadic Eye Movements 1989 Elsevier Amsterdam 257 281
54. R.A. Hinde The conflict between drives in the courtship and copulation of the Chaffinch Behaviour 5 1953 1 31
55. R.A. Hinde Animal Behaviour: a Synthesis of Ethology and Comparative Psychology 1966 McGraw-Hill London
56. J.R. Hollerman, L. Tremblay, and W. Schultz Involvement of basal ganglia and orbitofrontal cortex in goal-directed behavior Progress in Brain Research 126 2000 193 215
57. J.C. Houk, J.L. Adams, and A.G. Barto A model of how the basal ganglia generate and use neural signals that predict reinforcement J.C. Houk J.L. Davis D.G. Beiser Models of Information Processing in the Basal Ganglia 1995 MIT Press Cambridge, MA
58. J.C. Houk, J.L. Davis, and D.G. Beiser Models of Information Processing in the Basal Ganglia 1995 MIT Press Cambridge, MA
59. J.C. Houk, and S.P. Wise Distributed modular architectures linking basal ganglia, cerebellum, and cerebral-cortex - their role in planning and controlling action Cerebral Cortex 5 2 1995 95 110
60. A. Houston, and B. Sumida A positive feedback model for switching between 2 activities Animal Behaviour 33 FEB 1985 315 325
61. G. Hoyle The scope of neuroethology The Behavioural and Brain Sciences 7 3 1984 367 412
62. Humphries, M. D. (2002). The Basal Ganglia and Action: A Computational Study at Multiple Levels of Abstraction. Unpublished PhD, University of Sheffield, Sheffield.
63. M.D. Humphries, and K.N. Gurney A pulsed neural network model of bursting in the basal ganglia Neural Networks 14 2001 845 863
64. M.D. Humphries, and K.N. Gurney The role of intra-thalamic and thalamocortical circuits in action selection Network: Computation in Neural Systems 13 2002 131 156
65. M.D. Humphries, T.J. Prescott, and K.N. Gurney The interaction of recurrent axon collateral networks in the basal ganglia E.A.O. Kaynak E. Oja L. Xu Artificial Neural Networks and Neural Information Processing - ICANN/ICONIP 2003 (Lecture Notes in Computer Science 2714) 2003 Springer Berlin 797 804
66. E.M. Izhikevich Simple model of spiking neurons IEEE Transactions on Neural Networks 14 6 2003 1569 1572
67. D. Jaeger, S. Gilman, and J.W. Aldridge Neuronal activity in the striatum and pallidum of primates related to the execution of externally cued reaching movements Brain Research 694 1995 111 127
68. D. Joel, Y. Niv, and E. Ruppin Actor-critic models of the basal ganglia: new anatomical and computational perspectives Neural Networks 15 4-6 2002 535 547
69. D. Joel, and I. Weiner The organization of the basal ganglia- thalamocortical circuits - open interconnected rather than closed segregated Neuroscience 63 2 1994 363 379
70. J.P. Joseph, D. Boussaoud, and B. Biguer Activity of neurons in the cat substantia nigra pars reticulata during drinking Experimental Brain Research 60 2 1985 375 379
71. I. Kermadi, and D. Boussaoud Role of the primate striatum in attention and sensorimotor processes: comparison with premotor cortex Neuroreport 6 8 1995 1177 1181
72. I. Kermadi, and J.P. Joseph Activity in the caudate nucleus of monkey during spatial sequencing Journal of Neurophysiology 74 3 1995 911 933
73. M. Kimura Role of basal ganglia in behavioral learning Neuroscience Research 22 4 1995 353 358
74. C. Koch, T. Poggio, and V. Torre Nonlinear interactions in a dendritic tree: localization, timing, and role in information processing Proceedings of the National Academy of Science USA 80 9 1983 2799 2802
75. K.E. Krout, A.D. Loewy, G.W. Westby, and P. Redgrave Superior colliculus projections to midline and intralaminar thalamic nuclei of the rat Journal of Comparative Neurology 431 2 2001 198 216
76. C. Latimer Computer modelling of cognitive processes Noetica: Open Forum 1 1 1995 (http://psy.uq.edu.au/CogPsych/Noetica/ )
77. P.N. Lehner Handbook of Ethological Methods 2nd ed 1996 Cambridge University Press Cambridge, UK
78. M. Levesque, A. Charara, S. Gagnon, A. Parent, and M. Deschenes Corticostriatal projections from layer V cells in rat are collaterals of long-range corticofugal axons Brain Research 709 2 1996 311 315
79. K. Lorenz Der kumpan in der umwelt des vogels Journal of Ornithology 83 1935 137 213 (pp. 289-413)
80. P. Maes Modelling adaptive autonomous agents C.G. Langton Artificial Life: An Overview 1995 MIT Press Cambridge, MA
81. R. Malmo Activation: a neuropsychological dimension Psychological Review 66 1959 367 386
82. O. Marin, A. Gonzalez, and W. Smeets Anatomical substrate of amphibian basal ganglia involvement in visuomotor behaviour European Journal of Neuroscience 9 10 1997 2100 2109
83. C.D. Marsden, and J.A. Obeso The function of the basal ganglia and the paradox of stereotaxic surgery in Parkinson's disease Brain 117 1994 877 897
84. D.W. Massaro Some criticisms of connectionist models of human performance Journal of Memory and Language 27 1988 213 234
85. M.S. Matell, and W.H. Meck Neuropsychological mechanisms of interval timing behavior Bioessays 22 1 2000 94 103
86. D. McFarland Problems of Animal Behaviour 1989 Longman Harlow, UK
87. D. McFarland, and T. Bosser Intelligent Behaviour in Animals and Robots 1993 MIT Press Cambridge, MA
88. W.H. Meck Selective adjustment of the speed of internal clock and memory processes Journal of Experimental Psychology: Animal Behavior Processes 9 2 1983 171 201
89. W.H. Meck, and A.M. Benson Dissecting the brain's internal clock: how frontal-striatal circuitry keeps time and shifts attention Brain and Cognition 48 1 2002 195 211
90. M. Meyer-Luehmann, J.F. Thompson, K.C. Berridge, and J.W. Aldridge Substantia nigra pars reticulata neurons code initiation of a serial pattern: implications for natural action sequences and sequential disorders European Journal of Neuroscience 16 8 2002 1599 1608
91. J.W. Mink The basal ganglia: Focused selection and inhibition of competing motor programs Progress In Neurobiology 50 4 1996 381 425
92. J.W. Mink, and W.T. Thach Basal ganglia motor control. II. Late pallidal timing relative to movement onset and inconsistent pallidal coding of movement parameters Journal of Neurophysiology 65 2 1991 301 329
93. P.R. Montague, P. Dayan, and T.J. Sejnowski A framework for mesencephalic dopamine systems based on predictive hebbian learning Journal Of Neuroscience 16 5 1996 1936 1947
94. P.R. Montague, S.E. Hyman, and J.D. Cohen Computational roles for dopamine in behavioural control Nature 431 7010 2004 760 767
95. F. Montes Gonzalez, T.J. Prescott, K. Gurney, M. Humphries, and P. Redgrave An embodied model of action selection mechanisms in the vertebrate brain J.A. Meyer From Animals to Animats 6: Proceedings of the 6th International Conference on the Simulation of Adaptive Behavior 2000 MIT Press Cambridge, MA 157 166
96. A.B. Mulder, E. Tabuchi, and S.I. Wiener Neurons in hippocampal afferent zones of rat striatum parse routes into multi-pace segments during maze navigation European Journal of Neuroscience 19 7 2004 1923 1932
97. A. Parent, and L.N. Hazrati Functional anatomy of the basal ganglia. II. The place of subthalamic nucleus and external pallidum in basal ganglia circuitry Brain Research Reviews 20 1 1995 128 154
98. S. Pinker, and A. Prince On language and connectionism: analysis of a parallel distributed processing model of language acquisition Cognition 28 1988 73 193
99. T.J. Prescott, K. Gurney, and P. Redgrave Basal ganglia M.A. Arbib The Handbook of Brain Theory and Neural Networks 2nd ed 2002 MIT Press Cambridge, MA 147 151
100. T.J. Prescott, P. Redgrave, and K.N. Gurney Layered control architectures in robots and vertebrates Adaptive Behavior 7 1 1999 99 127
101. P. Redgrave, T. Prescott, and K.N. Gurney The basal ganglia: a vertebrate solution to the selection problem? Neuroscience 89 1999 1009 1023
102. P. Redgrave, T.J. Prescott, and K. Gurney Is the short latency dopamine burst too short to signal reward error? Trends in Neurosciences 22 1999 146 150
103. Reeves, G. (1998). Re: What really happened on Mars. Retrieved 54, 19, from http://catless.ncl.ac.uk/Risks/19.54.html.
104. J. Rinzel, and E. Ermentrout Analysis of neural excitability and oscillations C. Koch I. Segev Methods in Neuronal Modelling 1998 MIT Press Cambridge, MA
105. T.W. Robbins, and V.J. Brown The role of the striatum in the mental chronometry of action: a theoretical review Reviews in the Neurosciences 2 1990 181 213
106. K.D. Roeder Feedback, spontaneous activity, and behaviour G. Baerends C. Beer A. Manning Function and Evolution in Behaviour 1975 Clarendon Press Oxford
107. L.J. Ryan, and K.B. Clark The role of the subthalamic nucleus in the response of globus- pallidus neurons to stimulation of the prelimbic and agranular frontal cortices in rats Experimental Brain Research 86 3 1991 641 651
108. N. Schmitzer-Torbert, and A.D. Redish Neuronal activity in the rodent dorsal striatum in sequential navigation: separation of spatial and reward responses on the multiple T task Journal of Neurophysiology 91 5 2004 2259 2272
109. W. Schultz Activity of pars reticulata neurons of monkey substantia nigra in relation to motor, sensory, and complex events Journal of Neurophysiology 55 4 1986 660 677
110. W. Schultz Dopamine neurons and their role in reward mechanisms Current Opinion in Neurobiology 7 1997 191 197
111. W. Schultz, P. Apicella, and T. Ljungberg Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task Journal Of Neuroscience 13 3 1993 900 913
112. W. Schultz, P. Dayan, and P.R. Montague A neural substrate for prediction and reward Science 275 1997 1593 1599
113. L. Sha, R. Rajkumar, and J.P. Lehoczky Priority inheritance protocols: An approach to real-time synchronization IEEE Transactions on Computers 39 1990 1175 1185
114. R. Shibata, A.B. Mulder, O. Trullier, and S.I. Wiener Position sensitivity in phasically discharging nucleus accumbens neurons of rats alternating between tasks requiring complementary types of spatial cues Neuroscience 108 3 2001 391 411
115. E. Shink, M. Sidibe, and Y. Smith Efferent connections of the internal globus pallidus in the squirrel monkey: II. Topography and synaptic organization of pallidal efferents to the pedunculopontine nucleus Journal of Comparative Neurology 382 3 1997 348 363
116. Y. Smith, M.D. Bevan, E. Shink, and J.P. Bolam Microcircuitry of the direct and indirect pathways of the basal ganglia Neuroscience 86 1998 353 387
117. O. Sporns, and W.H. Alexander Neuromodulation and plasticity in an autonomous robot Neural Networks 15 4-6 2002 761 774
118. R.S. Sutton, and A.G. Barto Reinforcement learning: an introduction 1998 MIT Press Cambridge, MA
119. J.G. Taylor, and N.R. Taylor Analysis of recurrent cortico-basal ganglia-thalamic loops for working memory Biological Cybernetics 82 5 2000 415 432
120. N. Tinbergen The Study of Instinct 1951 Clarendon Press Oxford
121. F. Toates The interaction of cognitive and stimulus-response processes in the control of behaviour Neuroscience and Biobehavioral Reviews 22 1 1998 59 83
122. T. Tsumori, and Y. Yasui Organization of the nigro-tecto-bulbar pathway to the parvicellular reticular formation: a light- and electron-microscopic study in the rat Experimental Brain Research 116 2 1997 341 350
123. D. Ulrich Differential arithmetic of shunting inhibition for voltage and spike rate in neocortical pyramidal cells European Journal of Neuroscience 18 8 2003 2159 2165
124. J.R. Walters, D.N. Ruskin, K.A. Allers, and D.A. Bergstrom Pre- and postsynaptic aspects of dopamine-mediated transmission Trends in Neurosciences 23 10 2000 S41 S47
125. J. Wickens Basal ganglia: structure and computations Network-Computation in Neural Systems 8 4 1997 R77 R109
126. C.J. Wilson, and Y. Kawaguchi The origins of two-state spontaneous membrane potential fluctuations of neostriatal spiny neurons Journal of Neuroscience 16 7 1996 2397 2410
127. R. Wood, K.N. Gurney, and C.J. Wilson A novel parameter optimisation technique for compartmental models applied to a model of a striatal medium spiny neuron Neurocomputing 58-60 2004 1109 1116
128. F. Worgotter, and B. Porr Temporal sequence learning, prediction, and control: a review of different models and their relation to biological mechanisms Neural Computation 17 2 2005 245 319
129. R.M. Yerkes, and J.D. Dodson The relation of strength of stimulus to rapidity of habit formation Journal of Comparative Neurology and Psychology 18 1908 459 482
130. C.F. Zink, G. Pagnoni, M.E. Martin, M. Dhamala, and G.S. Berns Human striatal response to salient nonrewarding stimuli Journal of Neuroscience 23 22 2003 8092 8097