Self-organization in the basal ganglia with modulation of reinforcement signals

Neural Computation

Volumn 14, Issue 4, 2002, Pages 819-844

  Nakahara, Hiroyuki ^a   Amari, Shun Ichi ^a   Hikosaka, Okihide ^b  

^a RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

^b JUNTENDO UNIVERSITY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; ASSOCIATION; BASAL GANGLION; BRAIN CORTEX; CAUDATE NUCLEUS; CYTOLOGY; HUMAN; LEARNING; MENTAL FUNCTION; NEOSTRIATUM; NERVE CELL; NERVE TRACT; PHYSIOLOGY; REINFORCEMENT; SYNAPSE;

ALGORITHMS; BASAL GANGLIA; CAUDATE NUCLEUS; CEREBRAL CORTEX; CUES; HUMANS; LEARNING; MENTAL PROCESSES; NEOSTRIATUM; NEURAL PATHWAYS; NEURONS; REINFORCEMENT (PSYCHOLOGY); SYNAPSES;

EID: 0036546950     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/089976602317318974     Document Type: Article

References (77)

1. Alexander, G. E., Crutcher, M. D., & DeLong, M. R. (1990). Basal ganglia-thalamocortical circuits: Parallel substrates for motor, oculomotor, "prefrontal" and "limbic" functions. In H. B. M. Uylings, C. G. Van Eden, J. P. C. De Bruin, M. A. Corner, & M. G. P. Feenstra (Eds.), Progress in brain research (Vol. 85, pp. 119-146). Amsterdam: Elsevier.
2. Amari, S. (1977). Neural theory of association and concept-formation. Biological Cybernetics, 26, 175-185.
3. Amari, S. (1980). Topographic organization of nerve fields. Bulletin of Mathematical Biology, 42, 339-364.
4. Amari, S. (1983). Field theory of self-organizing neural nets. IEEE Transaction on Systems, Man and Cybernetics, SMC-13(9 & 10), 741-748.
5. Amari, S., & Takeuchi, A. (1978). Mathematical theory on formation of category detecting nerve cells. Biological Cybernetics, 29, 127-136.
6. Aosaki, T., Tsubokawa, H., Ishida, A., Watanabe, K., Graybiel, A. M., & Kimura, M. (1994). Responses of tonically active neurons in the primate's striatum undergo systematic changes during behavioral sensorimotor conditioning. J. Neurosci., 6, 3969-3984.
7. Bakin, J. S., & Weinberger, N. M. (1996). Induction of a physiological memory in the cerebral cortex by stimulation of the nucleus basalis. Proceedings of the National Academy of Sciences, 93, 11219-11224.
8. Barto, A. G. (1985). Learning by statistical cooperation of self-interested neuron-like computing elements. Human Neurobiology, 4, 229-256.
9. Barto, A. G. (1995). Adaptive critics and the basal ganglia. In J. C. Houk, J. L. Davis, & D. G. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 215-232). Cambridge, MA: MIT Press.
10. Barto, A. G., Sutton, R. S., & Brouwer, P. S. (1981). Associative search network: A reinforcement learning associative memory. Biological Cybernetics, 40, 201-211.
11. Bennett, B. D., & Bolam, J. P. (1994). Synaptic input and output of parbalbumin-immunoreactive neurons in the neostriatum of the rat. Neuroscience, 62, 707-719.
12. Berns, G. S., & Sejnowski, T. J. (1998). A computational model of how the basal ganglia produce sequences. Journal of Cognitive Neuroscience, 10(1), 108-121.
13. Bienenstock, E. L., Cooper, L. N., & Munro, P. W. (1982). Theory for the development of neuron selectivity: Orientation specificity and binocular interaction in visual cortex. Journal of Neuroscience, 2, 32-48.
14. Buonomano, D. V., & Merzenich, M. M. (1998). Cortical plasticity: From synapses to maps. Annual Review of Neuroscience, 21, 149-186.
15. Calabresi, P., Maj, R., Pisani, A., Mercuri, N. B., & Bernardi, G. (1992). Long-term synaptic depression in the striatum: Physiological and pharmacological characterization. Journal of Neuroscience, 12, 4224-4233.
16. Cepeda, C., Buchwald, N. A., & Levine, M. S. (1993). Neuromodulatory actions of dopamine in the neostriatum are dependent upon the excitatory amino acid receptor subtypes activated. Proceedings of the National Academy of Sciences of the United States of America, 90, 9576-9580.
17. Dayan, P. (1991). Navigating through temporal difference. In R. P. Lippmann, J. E. Moody, & D. S. Touretzky (Eds.), Advances in neural information processing systems, 3 (pp. 464-470). San Mateo, CA: Morgan Kaufmann.
18. Durstewitz, D., Seamans, J. K., & Sejnowski, T. J. (2000). Dopamine-mediated stabilization of delay-period activity in a network model of prefrontal cortex. Journal of Neurophysiology, 83, 1733-1750.
19. Fagiolini, M., & Hensch, T. K. (2000). Inhibitory threshold for critical-period activation in primary visual cortex. Nature, 404(6774), 183-186.
20. Flaherty, A. W., & Graybiel, A. M. (1994). Input-output organization of the sensorimotor striatum in the squirrel monkey. J. Neurosci., 2, 599-610.
21. Földiák, P. (1991). Learning invariance from transformation sequences. Neural Computation, 3, 194-200.
22. Gerfen, C. R. (1992). The neostriatal mosaic: Multiple levels of compartmental organization. Trends in Neurosciences, 15(4), 133-139.
23. Gonon, F. (1997). Prolonged and extrasynaptic excitatory action of dopamine mediated by D1 receptors in the rat striatum in vivo. Journal of Neuroscience, 17, 5972-5978.
24. Graybiel, A. M. (1995). Building action repertoires: Memory and learning functions of the basal ganglia. Current Opinion in Neurobiology, 5, 733-741.
25. Graybiel, A. M., Aosaki, T., Flaherty, A., & Kimura, M. (1994). The basal ganglia and adaptive motor control. Science, 265, 1826-1831.
26. Groves, P. M. (1983). A theory of the functional organization of the neostriatum and the neostriatal control of voluntary movement. Brain Research, 286(2), 109-132.
27. Gruber, A. (2000). A computational study of D1 induced modulation of medium spiny neuron response properties. Unpublished master's thesis, Northwestern University.
28. Hensch, T. K., Fagiolini, M., Mataga, N., Stryker, M. P., Baekkeskov, S., & Kash, S. F. (1998). Local GABA circuit control of experience-dependent plasticity in developing visual cortex. Science, 282(5393), 1504-1508.
29. Hikosaka, O., Nakahara, H., Rand, M. K., Sakai, K., Lu, X., Nakamura, K., Miyachi, S., & Doya, K. (1999). Parallel neural networks for learning sequential procedures. Trends in Neuroscience, 22(10), 464-471.
30. Hikosaka, O., Sakamoto, M., & Usui, S. (1989). Functional properties of monkey caudate neurons. II. visual and auditory responses. Journal of Neurophysiology, 61, 799-813.
31. Houk, J. C., Adams, J. L., & Barto, A. G. (1995). A model of how the basal ganglia generate and use neural signals that predict reinforcement. In J. C. Houk, J. L. Davis, & D. G. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 249-270). Cambridge, MA: MIT Press.
32. Jaeger, D., Kita, H., & Wilson, C. J. (1994). Surround inhibition among projection neurons is weak or nonexistent in the rat neostriatum. Journal of Neurophysiology, 72(5), 2555-2558.
33. Kano, M. (1995). Plasticity of inhibitory synapses in the brain: A possible memory mechanism that has been overlooked. Neuroscience Research, 21, 177-182.
34. Kano, M., Rexhausen, U., Dreessen, J., & Konnerth, A. (1992). Synaptic excitation produces a long-lasting rebound potentiation of inhibitory synaptic signals in cerebellar Purkinje cells. Nature, 356, 601-604.
35. Kawagoe, R., Takikawa, Y., & Hikosaka, O. (1998). Expectation of reward modulates cognitive signals in the basal ganglia. Nature Neuroscience, 1(5), 411-416.
36. Kawagoe, R., Takikawa, Y., & Hikosaka, O. (1999). Change in reward-predicting activity of monkey dopamine neurons: Short-term plasticity. Society for Neuroscience Abstracts, 25, 1162.
37. Kawaguchi, Y., Wilson, C. J., Augood, S. J., & Emson, P. C. (1995). Striatal interneurones: Chemical, physiological and morphological characterization. Trends in Neurosciences, 18(12), 527-535.
38. Kilgard, M. P., & Merzenich, M. M. (1998). Cortical map reorganization enabled by nucleus basalis activity. Science, 279, 1714-1718.
39. Kincaid, A. E., Zheng, T., & Wilson, C. J. (1998). Connectivity and convergence of single corticostriatal axons. Journal of Neuroscience, 18(12), 4722-4731.
40. Kirkwood, A., Rioult, M. C., & Bear, M. F. (1996). Experience-dependent modification of synaptic plasticity in visual cortex. Nature, 381(6582), 526-528.
41. Kita, K. (1993). GABAergic circuits of the striatum. In G. W. Arbuthnott & P. C. Emson (Eds.), Chemical signalling in the basal ganglia (pp. 51-72). Amsterdam: Elsevier.
42. Knopman, D., & Nissen, M. J. (1991). Procedural learning is impaired in Huntington's disease: Evidence from the serial reaction time task. Neuropsychologia, 29(3), 245-254.
43. Knowlton, B. J., Mangels, J. A., & Squire, L. R. (1996). A neostriatal habit learning system in humans. Science, 273, 1399-1402.
44. b receptors, monoamine receptors, and postsynaptic inositol trisphosphate-induced CA2+ release are involved in the induction of long-term potentiation at visual cortical inhibitory synapses. Journal of Neuroscience, 16(20), 6342-6352.
45. Komatsu, Y., & Iwakiri, M. (1993). Long-term modification of inhibitory synaptic transmission in developing visual cortex. NeuroReport, 4(7), 907-910.
46. Koós, T., & Tepper, J. M. (1999). Inhibitory control of neostriatal projection neurons by GABAergic interneurons. Nature Neuroscience, 2(5), 467-472.
47. LeVay, S., Stryker, M. P., & Shatz, C. J. (1978). Ocular dominance columns and their development in layer IV of the cat's visual cortex: A quantitative study. Journal of Comparative Neurology, 179(1), 223-244.
48. Marsden, C. D. (1980). The enigma of the basal ganglia and movement. Trends in Neuroscience, pp. 284-287.
49. Monchi, O., & Taylor, J. G. (1999). A hard wired model of coupled frontal working memories for various tasks. Information Sciences, 113(3), 221-243.
50. Montague, R., Dayan, P., & Sejnowski, T. J. (1996). Framework for mesencephalic dopamine systems based on predictive Hebbian learning. Journal of Neuroscience, 16, 1936-1947.
51. Nakahara, H., Doya, K., & Hikosaka, O. (2001). Parallel cortico-basal ganglia mechanisms for acquisition and execution of visuo-motor sequences - a computational approach. Journal of Cognitive Neuroscience, 13: 5, 626-647.
52. Nakahara, H., Trappenberg, T., Hikosaka, O., Kawagoe, R., & Takikawa, Y. (1998). Computational analysis on reward-modulated activities of caudate neurons. Society for Neuroscience Abstracts, 28, 1651.
53. Nicola, S. M., Surmeier, J., & Malenka, R. C. (2000). Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. Annual Review of Neuroscience, 23, 185-215.
54. Nusser, Z., Hajos, N., Somogyi, P., & Mody, I. (1998). Increased number of synaptic GABA(A) receptors underlies potentiation at hippocampal inhibitory synapses. Nature, 395(6698), 172-177.
55. Obermayer, K., Ritter, H., & Schulten, K. (1990). A principle for the formation of the spatial structure of cortical feature maps. Proceedings of the National Academy of Sciences, 87(21), 8345-8349.
56. Oorschot, D. E. (1996). Total number of neurons in the neostriatal, pallidal, subthalamic, and substantia nigral nuclei of the rat basal ganglia: A stereological study using the cavalieri and optical disector methods. Journal of Comparative Neurology, 366, 580-599.
57. Parthasarathy, H. B., Schall, J. D., & Graybiel, A. M. (1992). Distributed but convergent ordering of corticostriatal projections: Analysis of the frontal eye field and the supplementary eye field in the macaque monkey. Journal of Neuroscience, 12, 4468-4488.
58. Redgrave, P., Prescott, T. J., & Gurney, K. (1999). Is the short-latency dopamine response too short to signal reward error? Trends in Neuroscience, 22(4), 146-151.
59. Reynolds, J. N. J., & Wickens, J. R. (2000). Substantia nigra dopamine regulates synaptic plasticity and membrane potential fluctuations in the rat neostriatum, in vivo. Neuroscience, 99, 199-203.
60. Sachdev, R. N. S., Lu, S.-M., Wiley, R. G., & Ebner, F. F. (1998). Role of the basal forebrain cholinergic projection in somatosensory cortical plasticity. Journal of Neurophysiology, 79, 3216-3228.
61. Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of Neurophysiology, 80, 1-27.
62. Schultz, W., Apicella, P., & Ljungberg, T. (1993). Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. Journal of Neuroscience, 13(3), 900-913.
63. Schultz, W., Apicella, P., Romo, R., & Scarnati, E. (1995). Context-dependent activity in primate striatum reflecting past and future behavioral events. In J. C. Houk, J. L. Davis, & D. G. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 11-27). Cambridge, MA: MIT Press.
64. Schultz, W., Dayan, P., & Montague, R. (1997). A neural substrate of prediction and reward. Science, 275, 1593-1599.
65. Schultz, W., Romo, R., Ljungberg, T., Mirenowicz, J., Hollerman, J. R., & Dickinson, A. (1995). Reward-related signals carried by dopamine neurons. In J. C. Houk, J. L. Davis, & D. G. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 231-248). Cambridge, MA: MIT Press.
66. Surmeier, D. J., Song, W.-J., & Yan, Z. (1996). Coordinated expression of dopamine receptors in neostriatal medium spiny neurons. Journal of Neuroscience, 16, 6579-6591.
67. Takeuchi, A., & Amari, S. (1979). Formation of topographic maps and columnar microstructures. Biological Cybernetics, 35, 63-72.
68. Takikawa, Y., Kawagoe, R., & Hikosaka (2001). Reward-dependent spatial selection of anticipatory activity in monkey caudate neurons. Manuscript submitted for publication.
69. Trappenberg, T., Nakahara, H., & Hikosaka, H. (1998). Modeling reward dependent activity pattern of caudate neurons. In International Conference on Artificial Neural Network (ICANN98) (pp. 973-978). Skövde, Sweden.
70. Tunstall, M. J., Kean, A., Wickens, J. R., &Oorschot, D. (2001). Inhibitory interaction between spiny projection neurons of the striatum: a physiological and anatomical study. In Abstracts for International Basal Ganglia Society VIIth International Triennial Meeting (p. 38). Waitangi, New Zealand.
71. von der Malsburg, C. (1973). Self-organization of orientation selective cells in the striate cortex. Kybernetik, 14, 85-100.
72. Wickens, J. (1993). A theory of the striatum. Oxford: Pergamon Press.
73. Wickens, J. (1997). Basal ganglia: Structure and computations. Network: Computation in Neural Systems, 8, R77-R109.
74. Wickens, J., & Kötter, R. (1995). Cellular models of reinforcement. In J. C. Houk, J. L. Davis, & D. G. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 187-214). Cambridge, MA: MIT Press.
75. Wiesel, T. N., & Hubel, D. H. (1963). Single-cell responses in striate cortex of kittens deprived of vision in one eye. Journal of Neurophysiology, 26, 1003-1017.
76. Willshaw, D. J., & von der Malsburg, C. (1979). A marker induction mechanism for the establishment of ordered neural mappings: Its application to the retinotectal problem. Philos. Trans. R. Soc. Lond. B. Biol. Sci., 287(1021), 203-243.
77. Wilson, C. J. (1998). Basal ganglia. In G. M. Shepherd (Ed.), The synaptic organization of the brain (4th ed., pp. 279-316). Oxford: Oxford University Press.