Computational implications of microcircuit specializations in forebrain circuits for motivated action selection

Proceedings of the International Joint Conference on Neural Networks

Volumn , Issue , 2009, Pages 401-407

  Bullock, Daniel ^a   Tan, Can Ozan ^b  

^a BOSTON UNIVERSITY   (United States)

^b and Red Sox Foundation and Massachusetts General Hospital Home Base Program   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACTION SELECTION; BASAL GANGLIA; BRAIN ACTIVITY; BRAIN SCIENCE; CONTROL PROBLEMS; CORTICAL MICROCIRCUITS; FLEXIBLE CONTROL; FRONTAL CORTEX; FRONTAL LOBES; INDIVIDUAL DIFFERENCES; LANGUAGE GENERATION; MASSIVELY PARALLEL SYSTEMS; NONINVASIVE TECHNIQUE; VOLUNTARY ACTION;

LINGUISTICS; MAMMALS; NEURAL NETWORKS; REINFORCEMENT;

BRAIN;

EID: 70449434797     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2009.5178946     Document Type: Conference Paper

References (46)

1. D. Bullock, "Adaptive neural models of queuing and timing in fluent action, " Trends Cogn Sci, vol. 8, no. 9, pp. 426-433, 2004.
2. P. Redgrave, T. J. Prescott, and K. Gurney, "The basal ganglia: a vertebrate solution to the selection problem?, " Neuroscience, vol. 89, no. 4, pp. 1009-1023, 1999.
3. J. Houk, J. L. Adams, and A. G. Barto, "A model of how the basal ganglia generate and use signals that predic reinforcement, " in Models of information processing in the basal ganglia. J. C. Houk, J. L. Davis, and D. G. Beiser, Eds. Cambridge, MA: MIT Press, 1995, pp. 249-270.
4. R. S. Sutton and A. G. Barto, "Toward a modern theory of adaptive networks: expectation and prediction, " Psychol Rev, vol. 88, no. 2, pp. 135-170, 1981.
5. L. W. Swanson, "Cerebral hemisphere regulation of motivated behavior, " BrainRes, vol. 886, no. 1-2, pp. 113-164, 2000.
6. C. O. Tan, E. Anderson, M. Dranias, and D. Bullock, "Can the apparent adaptation of dopamine neurons' mismatch sensitivities be reconciled with their computation of reward prediction errors?, " Neurosci Lett, vol. 438, no. 1, pp. 14-16, 2008.
7. P. N. Tobler, C. D. Fiorillo, and W. Schultz, "Adaptive coding of reward value by dopamine neurons, " Science, vol. 307, no. 5715, pp. 1642-1645, 2005.
8. M. A. Ungless, P. J. Magill, and J. P. Bolam, "Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli, " Science, vol. 303, no. 5666, pp. 2040-2042, 2004.
9. C. O. Tan and D. Bullock, "A local circuit model of learned striatal and dopamine cell responses under probabilistic schedules of reward, " J Neurosci, vol. 28, no. 40, pp. 10062-10074, 2008.
10. S. Kakade and P. Dayan, "Dopamine: generalization and bonuses, " Neural Netw., vol. 15, no. 4-6, pp. 549-559, 2002.
11. P. A. Laurent, "The emergence of saliency and novelty responses from Reinforcement Learning principles, " Neural Netw., vol. 21, no. 10, pp. 1493-1499, 2008.
12. P. Redgrave, T. J. Prescott, and K. Gurney, "Is the shortlatency dopamine response too short to signal reward error?, " Trends Neurosci, vol. 22, no. 4, pp. 146-151, 1999.
13. C. D. Fiorillo, P. N. Tobler, and W. Schultz, "Discrete coding of reward probability and uncertainty by dopamine neurons, " Science, vol. 299, no. 5614, pp. 1898-1902, 2003.
14. C. D. Fiorillo, P. N. Tobler, and W. Schultz, "Evidence that the delay-period activity of dopamine neurons corresponds to reward uncertainty rather than backpropagating TD errors, " Behav. Brain Funct., vol. 1, no. 1, p. 7, 2005.
15. Y. Niv, M. O. Duff, and P. Dayan, "Dopamine, uncertainty and TD learning, " Behav. Brain Fund., vol. 1, p. 6, 2005.
16. C. O. Tan and D. Bullock, "Neuropeptide co-release with GABA may explain functional non-monotonic uncertainty responses in dopamine neurons, " Neurosci Lett, vol. 430, no. 3, pp. 218-223, 2008.
17. A. E. Kelley, M. Cador, L. Stinus, and M. M. Le, "Neurotensin, substance P, neurokinin-alpha, and enkephalin: injection into ventral tegmental area in the rat produces differential effects on operant responding, " Psychopharmacology (Berl), vol. 97, no. 2, pp. 243-252, 1989.
18. F. M. Placenza, P. J. Fletcher, S. Rotzinger, and F. J. Vaccarino, "Infusion of the substance P analogue, DiMe-C7, into the ventral tegmental area induces reinstatement of cocaine-seeking behaviour in rats, " Psychopharmacology (Berl), vol. 177, no. 1-2, pp. 111-120, 2004.
19. C. O. Tan and D. Bullock, "A dopamine-acetylcholine cascade: simulating learned and lesion-induced behavior of striatal cholinergic interneurons, " J Neurophysiol., vol. 100, no. 4, pp. 2409-2421, 2008.
20. M. P. Kilgard and M. M. Merzenich, "Cortical map reorganization enabled by nucleus basalis activity, " Science, vol. 279, no. 5357, pp. 1714-1718, 1998.
21. M. E. Hasselmo, "The role of acetylcholine in learning and memory, " Curr Opin. Neurobiol., vol. 16, no. 6, pp. 710-715, 2006.
22. V. Pawlak and J. N. Kerr, "Dopamine receptor activation is required for corticostriatal spike-timing-dependent plasticity, " J Neurosci, vol. 28, no. 10, pp. 2435-2446, 2008.
23. Z. Wang, L. Kai, M. Day, J. Ronesi, H. H. Yin, J. Ding, T. Tkatch, D. M. Lovinger, and D. J. Surmeier, "Dopaminergic control of corticostriatal long-term synaptic depression in medium spiny neurons is mediated by cholinergic interneurons, " Neuron, vol. 50, no. 3, pp. 443-452, 2006.
24. J. Q. Wang and J. F. McGinty, "The full D1 dopamine receptor agonist SKF-82958 induces neuropeptide mRNA in the normosensitive striatum of rats: regulation of D1/D2 interactions by muscarinic receptors, " J Pharmacol Exp Ther, vol. 281, no. 2, pp. 972-982, 1997.
25. A. A. Alcantara, L. Mrzljak, R. L. Jakab, A. I. Levey, S. M. Hersch, and P. S. Goldman-Rakic, "Muscarinic ml and m2 receptor proteins in local circuit and projection neurons of the primate striatum: anatomical evidence for cholinergic modulation of glutamatergic prefronto-striatal pathways, " J Comp Neurol, vol. 434, no. 4, pp. 445-460, 2001.
26. M. J. Frank, "Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism, " J Cogn Neurosci, vol. 17, no. 1, pp. 51-72, 2005.
27. M. J. Frank and R. C. O'Reilly, "A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol, " Behav. Neurosci, vol. 120, no. 3, pp. 497-517, 2006.
28. B. DeMartino, D. Kumaran, B. Seymour, and R. J. Dolan, "Frames, biases, and rational decision-making in the human brain, " Science, vol. 313, no. 5787, pp. 684-687, 2006.
29. M. R. Dranias, S. Grossberg, and D. Bullock, "Dopaminergic and non-dopaminergic value systems in conditioning and outcome-specific revaluation, " Brain Res, vol. 1238, pp. 239-287, 2008.
30. M. J. Frank and E. D. Claus, "Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal, " Psychol. Rev, vol. 113, no. 2, pp. 300-326, 2006.
31. J. W. Brown, D. Bullock, and S. Grossberg, "How laminar frontal cortex and basal ganglia circuits interact to control planned and reactive saccades, " Neural Netw., vol. 17, no. 4, pp. 471-510, 2004.
32. K. Srihasam, D. Bullock, and S. Grossberg, "Target selection by frontal cortex during coordinated saccadic and smooth pursuit eye movements, " J Cogn Neurosci, vol. 21, in Press, 2009.
33. R. S. Turner and M. R. DeLong, "Corticostriatal activity in primary motor cortex of the macaque, " J Neurosci, vol. 20, no. 18, pp. 7096-7108, 2000.
34. M. D. Bevan, P. A. Booth, S. A. Eaton, and J. P. Bolam, "Selective innervation of neostriatal interneurons by a subclass of neuron in the globus pallidus of the rat, " J Neurosci, vol. 18, no. 22, pp. 9438-9452, 1998.
35. S. Taverna, E. Ilijic, and D. J. Surmeier, "Recurrent collateral connections of striatal medium spiny neurons are disrupted in models of Parkinson's disease, " J Neurosci, vol. 28, no. 21, pp. 5504-5512, 2008.
36. L. Venance, J. Glowinski, and C. Giaume, "Electrical and chemical transmission between striatal GABAergic output neurones in rat brain slices, " J Physiol, vol. 559, no. 1, pp. 215-230, 2004.
37. D. Bullock and C. O. Tan, "Role of basal ganglia in decision sequences: A computational model of dopamine and acetylcholine regulation of action selection, interruption, resumptions, and switching, " Soc Neurosci Abs, S514, San Diego, CA, 2007.
38. M. T. Ullman, S. Corkin, M. Coppola, G. Hickok, J. H. Growdon, W. J. Koroshetz, and S. Pinker, "A neural dissociation within language: evidence that the mental dictionary is part of declarative memory, and that grammatical rules are processed by the procedural system, " J Cogn Neurosci, vol. 9, pp. 266-276, 1997.
39. J. W. Bohland, D. Bullock, and F. H. Guenther, "Neural representations and mechanisms for the performance of simple speech sequences, ", Submitted 2009.
40. S. Farrell and S. Lewandowsky, "Modeling transposition latencies: Constraints for theories of serial order memory, " J Mem Lang, vol. 51, pp. 115-135, 2004.
41. B. B. Averbeck, M. V. Chafee, D. A. Crowe, and A. P. Georgopoulos, "Parallel processing of serial movements in prefrontal cortex, " Proc. Natl. Acad. Sci U. S A, vol. 99, no. 20, pp. 13172-13177, 2002.
42. B. J. Rhodes, D. Bullock, W. B. Verwey, B. B. Averbeck, and M. P. Page, "Learning and production of movement sequences: behavioral, neurophysiological, and modeling perspectives, " Hum. Mov Sci, vol. 23, no. 5, pp. 699-746, 2004.
43. D. Bullock, "From parallel sequence representations to calligraphic control: a conspiracy of neural circuits, " Motor Control, vol. 8, no. 4, pp. 371-391, 2004.
44. R. Ivey, D. Bullock, and S. Grossberg, "A neuromorphic model of spatial lookahead planning, " Proceedings of the 12th International Conference on Cognitive and Neural Systems, Boston, MA, 2008.
45. S. Grossberg, "A theory of human memory: Selforganization and performance of sensory-motor codes, maps, and plans, " in Progress in Theoretical Biology. R. Rosen and F. Snell, Eds. New York: Academic Press, 1978, pp. 233-374.
46. I. Bornkessel and M. Schlesewsky, "The extended argument dependency model: a neurocognitive approach to sentence comprehension across languages, " Psychol. Rev, vol. 113, no. 4, pp. 787-821, 2006.