On the neural substrates for exploratory dynamics in basal ganglia: A model

Neural Networks

Volumn 32, Issue , 2012, Pages 65-73

  Kalva, Sanjeeva K ^a   Rengaswamy, Maithreye ^a   Chakravarthy, V S ^a   Gupte, Neelima ^a  

^a INDIAN INSTITUTE OF TECHNOLOGY MADRAS   (India)

Author keywords

Basal ganglia; Exploration; Network model; Reinforcement learning

Indexed keywords

ACTION SELECTION; BASAL GANGLIA; CHOICE EXPERIMENTS; DOHERTY; EXTREME VALUE; GLOBUS PALLIDUS; GO/NO-GO; HUMAN SUBJECTS; NETWORK MODELS; NEURAL NETWORK MODEL; NEURAL SUBSTRATES; SUBTHALAMIC NUCLEUS; SYSTEM DISPLAYS; TWO-REGIME;

BEHAVIORAL RESEARCH; DYNAMICS; NATURAL RESOURCES EXPLORATION; NEUROPHYSIOLOGY; REINFORCEMENT LEARNING;

BRAIN;

ACCURACY; ALGORITHM; ARTICLE; BEHAVIORAL RESEARCH; BRAIN FUNCTION; CHAOTIC DYNAMICS; CONTROLLED STUDY; DOPAMINERGIC TRANSMISSION; ELECTROENCEPHALOGRAM; EXPLORATORY BEHAVIOR; GLOBUS PALLIDUS; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; NERVE CELL NETWORK; PREDICTION; PRIORITY JOURNAL; SENSITIVITY AND SPECIFICITY; SIMULATION; STRIATE CORTEX; SUBTHALAMIC NUCLEUS;

ALGORITHMS; BASAL GANGLIA; CHOICE BEHAVIOR; CORPUS STRIATUM; DOPAMINE; EXPLORATORY BEHAVIOR; GLOBUS PALLIDUS; HUMANS; MODELS, NEUROLOGICAL; MOVEMENT; NEURAL NETWORKS (COMPUTER); NEURAL PATHWAYS; NONLINEAR DYNAMICS; NORMAL DISTRIBUTION; SUBTHALAMIC NUCLEUS;

EID: 84861790286     PISSN: 08936080     EISSN: 18792782     Source Type: Journal    
DOI: 10.1016/j.neunet.2012.02.031     Document Type: Article

References (46)

1. Akaike A., Ohno Y., Sasa M., Takaori S. Excitatory and inhibitory effects of dopamine on neuronal activity of the caudate nucleus neurons in vitro. Brain Research 1987, 418:262-272.
2. Albin R.L., Young A.B., Penney J.B. The functional anatomy of basal ganglia disorders. Trends Neurosciences 1989, 12:366-375.
3. Alexander G.E., Crutcher M.D. Functional architecture of basal ganglia circuits: neural substrates of parallel processing. Trends in Neurosciences 1990, 13:266-271.
4. Bamford N.S., Zhang H., Schmitz Y., Wu N.P., Cepeda C., Levine M.S., et al. Heterosynaptic dopamine neurotransmission selects sets of corticostriatal terminals. Neuron 2004, 42:653-663.
5. Baunez C., Humby T., Eagle D.M., Ryan L.J., Dunnett S.B., Robbins T.W. Effects of STN lesions on simple vs choice reaction time tasks in the rat: preserved motor readiness, but impaired response selection. European Journal of Neuroscience 2001, 13:1609-1616.
6. Best J., Park C., Terman D., Wilson C. Transitions between irregular and rhythmic firing patterns in excitatory-inhibitory neuronal networks. Journal of Computational Neuroscience 2007, 23:217-235.
7. Bevan M.D., Magill P.J., Terman D., Bolam J.P., Wilson C.J. Move to the rhythm: oscillations in the subthalamic nucleus-external globus pallidus network. Trends in Neurosciences 2002, 25:525-531.
8. Bogacz R., Larsen T. Integration of reinforcement learning and optimal decision-making theories of the basal ganglia. Neural Computation 2011, 23:817-851.
9. Bourdaud N., Chavarriaga R., Galan F., Millan J.del R. Characterizing the EEG correlates of exploratory behavior. IEEE Transactions of Neural Systems and Rehabilitation Engineering 2008, 16:549-556.
10. Chakravarthy V.S., Denny Joseph, Bapi Raju S. What do the basal ganglia do? A modeling perspective. Biological Cybernetics 2010, 103:237-253.
11. Chen C.C., Hsu Y.T., Chan H.L., Chiou S.M., Tu P.H., Lee S.T., et al. Complexity of subthalamic 13-35 Hz oscillatory activity directly correlates with clinical impairment in patients with Parkinson's disease. Experimental Neurology 2010, 224:234-240.
12. Cools R., Frank M.J., Gibbs S.E., Miyakawa A., Jagust W., D'Esposito M. Striatal dopamine predicts outcome-specific reversal learning and its sensitivity to dopaminergic drug administration. Journal of Neuroscience 2009, 29:1538-1543.
13. Cruz A.V., Mallet N., Magill P.J., Brown P., Averbeck B.B. Effects of dopamine depletion on network entropy in the external globus pallidus. Journal of Neurophysiology 2009, 102:1092-1102.
14. Daw N.D., O'Doherty J.P., Dayan P., Seymour B., Dolan R.J. Cortical substrates for exploratory decisions in humans. Nature 2006, 441:876-879.
15. El-Ghundi M., Fletcher P.J., Drago J., Sibley D.R., O'Dowd B.F., George S.R. Spatial learning deficit in dopamine D(1) receptor knockout mice. European Journal of Pharmacology 1999, 383:95-106.
16. Fowler S.C., Zarcone T.J., Vorontsova E., Chen R. Motor and associative deficits in D2 dopamine receptor knockout mice. International Journal of Developmental Neuroscience 2002, 20:309-321.
17. Frank M.J., Doll B.B., Oas-Terpstra J., Moreno F. Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nature Neuroscience 2009, 12:1062-1068.
18. Frank M.J., Moustafa A.A., Haughey H.M., Curran T., Hutchison K.E. Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proceeding of the National Academy of Sciences of the United States of America 2007, 104:16311-16316.
19. Frank M.J. Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. Journal of Cognitive Neuroscience 2005, 17:51-72.
20. Gerfen C.R. Molecular effects of dopamine on striatal projection pathways. Trends in Neuroscience 2000, 23:S64-S70.
21. Gillies A., Willshaw D., Li Z. Subthalamic-pallidal interactions are critical in determining normal and abnormal functioning of the basal ganglia. Proceedings, Biological Sciences 2002, 269:545-551.
22. Gonon F. Prolonged and extrasynaptic excitatory action of dopamine mediated by D1 receptors in the rat striatum in vivo. Journal of Neuroscience 1997, 17:5972-5978.
23. Gruber A.J., Solla S.A., Surmeier D.J., Houk J.C. Modulation of striatal single units by expected reward: a spiny neuron model displaying dopamine-induced bistability. Journal of Neurophysiology 2003, 90:1095-1114.
24. Hauptmann C., Tass P.A. Therapeutic rewiring by means of desynchronizing brain stimulation. Biosystems 2007, 89:173-181.
25. Hikosaka O., Takikawa Y., Kawagoe R. Role of the basal ganglia in the control of purposive saccadic eye movements. Physiological Reviews 2000, 80:953-978.
26. Hurtado J.M., Gray C.M., Tamas L.B., Sigvardt K.A. Dynamics of tremor-related oscillations in the human globus pallidus: a single case study. Proceeding of the National Academy of Sciences of the United States of America 1999, 96:1674-1679.
27. Joel D., Niv Y., Ruppin E. Actor-critic models of the basal ganglia: new anatomical and computational perspectives. Neural Networks 2002, 15:535-547.
28. Kauffman S. Self organization and selection in evolution 1993, Oxford University Press, Oxford.
29. Kitzbichler M.G., Smith M.L., Christensen S.R., Bullmore E. Broadband criticality of human brain network synchronization. PLoS Computational Biology 2009, 5:e1000314.
30. Krishnan R., Ratnadurai S., Subramanian D., Chakravarthy V.S., Rengaswamy M. Modeling the role of basal ganglia in saccade generation: is the indirect pathway the explorer?. Neural Networks 2011, 8:801-813.
31. Langton C. Computation at the edge of chaos: phase transitions and emergent computation. Physica D: Nonlinear Phenomena 1990, 42:12-37.
32. Magdoom K.N., Subramanian D., Chakravarthy V.S., Ravindran B., Amari S., Meenakshisundaram N. Modeling basal ganglia for understanding Parkinsonian reaching movements. Neural Computation 2011, 23:477-516.
33. Magill P.J., Bolam J.P., Bevan M.D. Dopamine regulates the impact of the cerebral cortex on the subthalamic nucleus-globus pallidus network. Neuroscience 2001, 106:313-330.
34. Maia T.V. Reinforcement learning, conditioning, and the brain: successes and challenges. Cognitive Affective and Behavioral Neuroscience 2009, 9:343-364.
35. Mallet N., Pogosyan A., Márton L.F., Bolam J.P., Brown P., Magill P.J. Parkinsonian beta oscillations in the external globus pallidus and their relationship with subthalamic nucleus activity. Journal of Neuroscience 2008, 28:14245-14258.
36. Modolo J., Henry J., Beuter A. Dynamics of the subthalamo-pallidal complex in Parkinson's disease during deep brain stimulation. Journal Biological Physics 2008, 34:251-266.
37. O'Doherty J., Dayan P., Schultz J., Deichmann R., Friston K., Dolan R.J. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 2004, 304:452-454.
38. Plenz D., Kitai S.T. A basal ganglia pacemaker formed by the subthalamic nucleus and external globus pallidus. Nature 1999, 400:677-682.
39. Rubchinsky L.L., Kopell N., Sigvardt K.A. Modeling facilitation and inhibition of competing motor programs in basal ganglia subthalamic nucleus-pallidal circuits. Proceeding of the National Academy of Sciences of the United States of America 2003, 100:14427-14432.
40. Schultz W. Predictive reward signal of dopamine neurons. Journal of Neurophysiology 1998, 80:1-27.
41. Shen K.Z., Johnson S.W. Presynaptic dopamine D2 and muscarine M3 receptors inhibit excitatory and inhibitory transmission to rat subthalamic neurones in vitro. Journal of Physiology (London) 2000, 525:331-341.
42. Sridharan D., Prashanth P.S., Chakravarthy V.S. The role of the basal ganglia in exploration in a neural model based on reinforcement learning. International Journal of Neural Systems 2006, 16:111-124.
43. Standage D.I., Trappenberg T.P., Klein R.M. Modelling divided visual attention with a winner-take-all network. Neural Networks 2005, 18:620-627.
44. Steigerwald F., Pötter M., Herzog J., Pinsker M., Kopper F., Mehdorn H., et al. Neuronal activity of the human subthalamic nucleus in the parkinsonian and non parkinsonian state. Journal of Neurophysiology 2008, 100:2515-2524.
45. Terman D., Rubin J.E., Yew A.C., Wilson C.J. Activity patterns in a model for the subthalamopallidal network of the basal ganglia. Journal of Neuroscience 2002, 22:2963-2976.
46. Yin H.H., Knowlton B.J. The role of the basal ganglia in habit formation. Nature Reviews Neuroscience 2006, 7:464-476.