Shifting responsibly: The importance of striatal modularity to reinforcement learning in uncertain environments

Frontiers in Human Neuroscience

Volumn , Issue MAY, 2011, Pages 1-

  Amemori, Ken Ichi ^a,b   Gibb, Leif G ^a,b   Graybiel, Ann M ^a,b  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Acetylcholine; Basal ganglia; Direct and indirect pathways; Mixture of experts; Modular reinforcement learning; Responsibility signal; Striatum; Striosome and matrix compartments

Indexed keywords

ARTICLE; BASAL GANGLION; BEHAVIOR; CORPUS STRIATUM; INTERNEURON; MATHEMATICAL MODEL; NERVOUS SYSTEM; PREDICTION; REINFORCEMENT; SPIKE; STIMULATION;

EID: 82955167945     PISSN: 16625161     EISSN: 16625161     Source Type: Journal    
DOI: 10.3389/fnhum.2011.00047     Document Type: Article

References (131)

1. Alexander, G. E., DeLong, M. R., and Strick, P. L. (1986). Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 9, 357-381.
2. Amemori, K., and Graybiel, A. M. (2009). "Stimulation of the macaque rostral anterior cingulate cortex alters decision in approach-avoidance conflict," in Program No. 194.1, 2009 Neuroscience Meeting Planner (Chicago, IL: Society for Neuroscience).
3. Amemori, K., and Graybiel, A. M. (2010). "Localized microstimulation of macaque pregenual anterior cingulate cortex increases rejection of cued outcomes in approach-avoidance decision-making," in Program No. 306.4, 2010 Neuroscience Meeting Planner (San Diego, CA: Society for Neuroscience).
4. Amemori, K., and Ishii, S. (2001). Gaussian process approach to spiking neurons for inhomogeneous Poisson inputs. Neural Comput. 13, 2763-2797.
5. Aosaki, T, Graybiel, A. M., and Kimura, M. (1994a). Effect of the nigrostriatal dopamine system on acquired neural responses in the striatum of behaving monkeys. Science 265, 412-415.
6. Aosaki, T., Tsubokawa, H., Ishida, A., Watanabe, K., Graybiel, A. M., and Kimura, M. (1994b). Responses of tonically active neurons in the primate's striatum undergo systematic changes during behavioral sensorimotor conditioning. J. Neurosci. 14, 3969-3984.
7. Aosaki, T., Kimura, M., and Graybiel, A. M. (1995). Temporal and spatial characteristics of tonically active neurons of the primate's striatum. J. Neurophysiol. 73, 1234-1252.
8. Apicella, P. (2007). Leading tonically active neurons of the striatum from reward detection to context recognition. Trends Neurosci. 30, 299-306.
9. Ashby, F. G., and Crossley, M. J. (2011). A computational model of how cholinergic interneurons protect striatal-dependent learning. J. Cogn. Neurosci. 23, 1549-1566.
10. Aubry, J. M., Lundström, K., Kawashima, E., Ayala, G., Schulz, P., Bartanusz, V., and Kiss, J. Z. (1994). NK1 receptor expression by cholinergic interneurones in human striatum. Neuroreport 5, 1597-1600.
11. Bar-Gad, I., Morris, G., and Bergman, H. (2003). Information processing, dimensionality reduction and reinforcement learning in the basal ganglia. Prog. Neurobiol. 71, 439-473.
12. Behrens, T. E., Woolrich, M. W., Walton, M. E., and Rushworth, M. F. (2007). Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214-1221.
13. Bernacchia, A., Seo, H., Lee, D., and Wang, X. J. (2011). A reservoir of time constants for memory traces in cortical neurons. Nat. Neurosci. 14, 366-372.
14. Bertin, M., Schweighofer, N., and Doya, K. (2007). Multiple model-based reinforcement learning explains dopamine neuronal activity. Neural Netw. 20, 668-675.
15. Bevan, M. D., Booth, P. A., Eaton, S. A., and Bolam, J. P. (1998). Selective innervation of neostriatal interneurons by a subclass of neuron in the globus pallidus of the rat. J. Neurosci. 18, 9438-9452.
16. Bevan, M. D., Clarke, N. P., and Bolam, J. P. (1997). Synaptic integration of functionally diverse pallidal information in the entopeduncular nucleus and subthalamic nucleus in the rat. J. Neurosci. 17, 308-324.
17. Bevan, M. D., Smith, A. D., and Bolam, J. P. (1996). The substantia nigra as a site of synaptic integration of functionally diverse information arising from the ventral pallidum and the globus pallidus in the rat. Neuroscience 75, 5-12.
18. Bolam, J. P., Hanley, J. J., Booth, P. A., and Bevan, M. D. (2000). Synaptic organisation of the basal ganglia. J. Anat. 196(Pt 4), 527-542.
19. Bolam, J. P., Ingham, C. A., Izzo, P. N., Levey, A. I., Rye, D. B., Smith, A. D., and Wainer, B. H. (1986). Substance P-containing terminals in synaptic contact with cholinergic neurons in the neostriatum and basal forebrain: a double immunocytochemical study in the rat. Brain Res. 397, 279-289.
20. Bolam, J. P., Izzo, P. N., and Graybiel, A. M. (1988). Cellular substrate of the histochemically defined striosome/matrix system of the caudate nucleus: a combined golgi and immunocytochemical study in cat and ferret. Neuroscience 24, 853-875.
21. Bolam, J. P., Smith, Y., Ingham, C. A., von Krosigk, M., and Smith, A. D. (1993). Convergence of synaptic terminals from the striatum and the globus pallidus onto single neurones in the substantia nigra and the entopeduncular nucleus. Prog. Brain Res. 99, 73-88.
22. Calabresi, P., Centonze, D., Gubellini, P., and Bernardi, G. (1999). Activation of M1-like muscarinic receptors is required for the induction of corticostriatal LTP. Neuropharmacology 38, 323-326.
23. Calabresi, P., Centonze, D., Gubellini, P., Pisani, A., and Bernardi, G. (1998). Blockade of M2-like muscarinic receptors enhances long-term potentiation at corticostriatal synapses. Eur. J. Neurosci. 10, 3020-3023.
24. Canales, J. J. (2005). Stimulant-induced adaptations in neostriatal matrix and striosome systems: transiting from instrumental responding to habitual behavior in drug addiction. Neurobiol. Learn. Mem. 83, 93-103.
25. Canales, J. J., and Graybiel, A. M. (2000). A measure of striatal function predicts motor stereotypy. Nat. Neurosci. 3, 377-383.
26. Chesselet, M. F., and Graybiel, A. M. (1986). Striatal neurons expressing somatostatin-like immunoreactivity: evidence for a peptidergic interneuronal system in the cat. Neuroscience 17, 547-571.
27. Courtemanche, R., Fujii, N., and Graybiel, A. M. (2003). Synchronous, focally modulated beta-band oscillations characterize local field potential activity in the striatum of awake behaving monkeys. J. Neurosci. 23, 11741-11752.
28. Cragg, S. J. (2006). Meaningful silences: how dopamine listens to the ACh pause. Trends Neurosci. 29, 125-131.
29. Degos, B, Deniau, J. M., Le Cam, J., Mailly, P., and Maurice, N. (2008). Evidence for a direct subthalamo-cortical loop circuit in the rat. Eur. J. Neurosci. 27, 2599-2610.
30. Ding, J. B., Guzman, J. N., Peterson, J. D., Goldberg, J. A., and Surmeier, D. J. (2010). Thalamic gating of corticostriatal signaling by cholinergic interneurons. Neuron 67, 294-307.
31. Doya, K. (2000). Complementary roles of basal ganglia and cerebellum in learning and motor control. Curr. Opin. Neurobiol. 10, 732-739.
32. Doya, K. (2002). Metalearning and neuromodulation. Neural Netw. 15, 495-506.
33. Doya, K., Samejima, K., Katagiri, K., and Kawato, M. (2002). Multiple model-based reinforcement learning. Neural Comput. 14, 1347-1369.
34. Eblen, F., and Graybiel, A. M. (1995). Highly restricted origin of prefrontal cortical inputs to striosomes in the macaque monkey. J. Neurosci. 15, 5999-6013.
35. Flaherty, A. W., and Graybiel, A. M. (1991). Corticostriatal transformations in the primate somatosensory system. Projections from physiologically mapped body-part representations. J. Neurophysiol. 66, 1249-1263.
36. Flaherty, A. W., and Graybiel, A. M. (1993). Two input systems for body representations in the primate striatal matrix: experimental evidence in the squirrel monkey. J. Neurosci. 13, 1120-1137.
37. Flaherty, A. W., and Graybiel, A. M. (1994). Input-output organization of the sensorimotor striatum in the squirrel monkey. J. Neurosci. 14, 599-610.
38. Fujiyama, F., Sohn, J., Nakano, T., Furuta, T., Nakamura, K. C., Matsuda, W., and Kaneko, T. (2011). Exclusive and common targets of neostriatofugal projections of rat striosome neurons: a single neuron-tracing study using a viral vector. Eur. J. Neurosci. 33, 668-677.
39. Galarraga, E., Hernández-López, S., Reyes, A., Miranda, I., Bermudez-Rattoni, F., Vilchis, C., and Bargas, J. (1999). Cholinergic modulation of neostriatal output: a functional antagonism between different types of muscarinic receptors. J. Neurosci. 19, 3629-3638.
40. Gerfen, C. R. (1984). The neostriatal mosaic: compartmentalization of corticostriatal input and striatonigral output systems. Nature 311, 461-464.
41. Gerfen, C. R., Herkenham, M., and Thibault, J. (1987). The neostriatal mosaic: II. Patch- and matrix-directed mesostriatal dopaminergic and non-dopaminergic systems. Neurosci. J. 7, 3915-3934.
42. Gimenez-Amaya, J. M., and Graybiel, A. M. (1990). Compartmental origins of the striatopallidal projection in the primate. Neuroscience 34, 111-126.
43. Gimenez-Amaya, J. M., and Graybiel, A. M. (1991). Modular organization of projection neurons in the matrix compartment of the primate striatum. J. Neurosci. 11, 779-791.
44. Goto, S., Lee, L. V., Munoz, E. L., Tooyama, I., Tamiya, G., Makino, S., Ando, S., Dantes, M. B., Yamada, K., Matsumoto, S., Shimazu, H., Kuratsu, J., Hirano, A., and Kaji, R. (2005). Functional anatomy of the basal ganglia in X-linked recessive dystonia-parkinsonism. Ann. Neurol. 58, 7-17.
45. Graybiel, A. M. (1984). Correspondence between the dopamine islands and striosomes of the mammalian striatum. Neuroscience 13, 1157-1187.
46. Graybiel, A. M. (1991). Basal ganglia - input, neural activity, and relation to the cortex. Curr. Opin. Neurobiol. 1, 644-651.
47. Graybiel, A. M. (1995). The basal ganglia. Trends Neurosci. 18, 60-62.
48. Graybiel, A. M. (1997). The basal ganglia and cognitive pattern generators. Schizophr. Bull. 23, 459-469.
49. Graybiel, A. M. (1998). The basal ganglia and chunking of action repertoires. Neurobiol. Learn. Mem. 70, 119-136.
50. Graybiel, A. M. (2008). Habits, rituals, and the evaluative brain. Annu. Rev. Neurosci. 31, 359-387.
51. Graybiel, A. M., Aosaki, T., Flaherty, A. W., and Kimura, M. (1994). The basal ganglia and adaptive motor control. Science 265, 1826-1831.
52. Graybiel, A. M., Baughman, R. W., and Eckenstein, F. (1986). Cholinergic neuropil of the striatum observes striosomal boundaries. Nature 323, 625-627.
53. Graybiel, A. M., Canales, J. J., and Capper-Loup, C. (2000). Levodopa-induced dyskinesias and dopamine-dependent stereotypies: a new hypothesis. Trends Neurosci. 23(Suppl. 10), S71-S77.
54. Graybiel, A. M., and Ragsdale, C. W. (1978). Histochemically distinct compartments in the striatum of human, monkeys, and cat demonstrated by acetylthiocholinesterase staining. Proc. Natl. Acad. Sci. U.S.A. 75, 5723-5726.
55. Graybiel, A. M., Ragsdale, C. W. Jr., Yoneoka, E. S., and Elde, R. P. (1981). An immunohistochemical study of enkephalins and other neuropeptides in the striatum of the cat with evidence that the opiate peptides are arranged to form mosaic patterns in register with the striosomal compartments visible by acetylcholinesterase staining. Neuroscience 6, 377-397.
56. Groves, P. M., Martone, M., Young, S. J., and Armstrong, D. M. (1988). Three-dimensional pattern of enkephalin-like immunoreactivity in the caudate nucleus of the cat. J. Neurosci. 8, 892-900.
57. Haruno, M., Wolpert, D. M., and Kawato, M. (2001). Mosaic model for sensorimotor learning and control. Neural Comput. 13, 2201-2220.
58. Herkenham, M., and Pert, C. B. (1981). Mosaic distribution of opiate receptors, parafascicular projections and acetylcholinesterase in rat striatum. Nature 291, 415-418.
59. Holt, D. J., Graybiel, A. M., and Saper, C. B. (1997). Neurochemical architecture of the human striatum. J. Comp. Neurol. 384, 1-25.
60. Houk, J. C., Adams, J. L., and Barto, A. G. (1995). "A model of how the basal ganglia generate and use neural signals that predict reinforcement," in Models of Information Processing in the Basal Ganglia, eds J. C. Houk, J. D. Davis, and D. G. Beiser (Cambridge, MA: MIT Press), 249-270.
61. Hsu, K. S., Yang, C. H., Huang, C. C., and Gean, P. W. (1996). Carbachol induces inward current in neostriatal neurons through M1-like muscarinic receptors. Neuroscience 73, 751-760.
62. Imamizu, H., Kuroda, T., Miyauchi, S., Yoshioka, T., and Kawato, M. (2003). Modular organization of internal models of tools in the human cerebellum. Proc. Natl. Acad. Sci. U.S.A. 100, 5461-5466.
63. Imamizu, H., Kuroda, T., Yoshioka, T., and Kawato, M. (2004). Functional magnetic resonance imaging examination of two modular architectures for switching multiple internal models. J. Neurosci. 24, 1173-1181.
64. Jabourian, M, Venance, L., Bourgoin, S., Ozon, S., Pérez, S., Godeheu, G., Glowinski, J., and Kemel, M. L. (2005). Functional mu opioid receptors are expressed in cholinergic interneurons of the rat dorsal striatum: territorial specificity and diurnal variation. Eur. J. Neurosci. 21, 3301-3309.
65. Jackson, A., and Crossman, A. R. (1981). Subthalamic nucleus efferent projection to the cerebral cortex. Neuroscience 6, 2367-2377.
66. Jacobs, R. A., Jordan, M. I., Nowlan, S. J., and Hinton, G. E. (1991). Adaptive mixtures of local experts. Neural Comput. 3, 79-87.
67. Jakab, R. L., Hazrati, L. N., and Goldman-Rakic, P. (1996). Distribution and neurochemical character of substance P receptor (SPR)-immunoreactive striatal neurons of the macaque monkey: accumulation of SP fibers and SPR neurons and dendrites in "striocapsules" encircling striosomes. J. Comp. Neurol. 369, 137-149.
68. Jiménez-Castellanos, J., and Graybiel, A. M. (1989). Compartmental origins of striatal efferent projections in the cat. Neuroscience 32, 297-321.
69. Joel, D., and Weiner, I. (2000). The connections of the dopaminergic system with the striatum in rats and primates: an analysis with respect to the functional and compartmental organization of the striatum. Neuroscience 96, 451-474.
70. Jordan, M. I., and Jacobs, R. A. (1994). Hierarchical mixtures of experts and the EM algorithm. Neural Comput. 6, 181-214.
71. Kaneda, K., Nambu, A., Tokuno, H., and Takada, M. (2002). Differential processing patterns of motor information via striatopallidal and striatonigral projections. J. Neurophysiol. 88, 1420-1432.
72. Kaneko, S., Hikida, T., Watanabe, D., Ichinose, H., Nagatsu, T., Kreitman, R. J., Pastan, I., and Nakanishi, S. (2000). Synaptic integration mediated by striatal cholinergic interneurons in basal ganglia function. Science 289, 633-637.
73. Kaneko, T., Shigemoto, R., Nakanishi, S., and Mizuno, N. (1993). Substance P receptor-immunoreactive neurons in the rat neostriatum are segregated into somatostatinergic and cholinergic aspiny neurons. Brain Res. 631, 297-303.
74. Kawaguchi, Y. (1992). Large aspiny cells in the matrix of the rat neostriatum in vitro: physiological identification, relation to the compartments and excitatory postsynaptic currents. J. Neurophysiol. 67, 1669-1682.
75. Kawaguchi, Y., Wilson, C. J., Augood, S. J., and Emson, P. C. (1995). Striatal interneurones: chemical, physiological and morphological characterization. Trends Neurosci. 18, 527-535.
76. Kawaguchi, Y., Wilson, C. J., and Emson, P. C. (1989). Intracellular recording ofidentified neostriatal patch and matrix spiny cells in a slice preparation preserving cortical inputs. J. Neurophysiol. 62, 1052-1068.
77. Kimura, M., Rajkowski, J., and Evarts, E. (1984). Tonically discharging putamen neurons exhibit set-dependent responses. Proc. Natl. Acad. Sci. U.S.A. 81, 4998-5001.
78. Kincaid, A. E., and Wilson, C. J. (1996). Corticostriatal innervation of the patch and matrix in the rat neostriatum. J. Comp. Neurol. 374, 578-592.
79. Kita, H., and Kitai, S. T. (1987). Efferent projections of the subthalamic nucleus in the rat: light and electron microscopic analysis with the PHA-L method. J. Comp. Neurol. 260, 435-452.
80. Koós, T., and Tepper, J. M. (2002). Dual cholinergic control of fast-spiking interneurons in the neostriatum. J. Neurosci. 22, 529-535.
81. Kreitzer, A. C. (2009). Physiology and pharmacology of striatal neurons. Annu. Rev. Neurosci. 32, 127-147.
82. Langer, L. F., and Graybiel, A. M. (1989). Distinct nigrostriatal projection systems innervate striosomes and matrix in the primate striatum. Brain Res. 498, 344-350.
83. Lapper, S. R., and Bolam, J. P. (1992). Input from the frontal cortex and the parafascicular nucleus to cholinergic interneurons in the dorsal striatum of the rat. Neuroscience 51, 533-545.
84. Le Moine, C., Kieffer, B., Gaveriaux-Ruff, C., Befort, K., and Bloch, B. (1994). Delta-opioid receptor gene expression in the mouse forebrain: localization in cholinergic neurons of the striatum. Neuroscience 62, 635-640.
85. Lee, T, Kaneko, T., Shigemoto, R., Nomura, S., and Mizuno, N. (1997). Collateral projections from striatonigral neurons to substance P receptor-expressing intrinsic neurons in the striatum of the rat. J. Comp. Neurol. 388, 250-264.
86. Lei, W., Jiao, Y., Del Mar, N., and Reiner, A. (2004). Evidence for differential cortical input to direct pathway versus indirect pathway striatal projection neurons in rats. J. Neurosci. 24, 8289-8299.
87. Matsuda, W., Furuta, T., Nakamura, K. C., Hioki, H., Fujiyama, F., Arai, R., and Kaneko, T. (2009). Single nigrostriatal dopaminergic neurons form widely spread and highly dense axonal arborizations in the neostriatum. J. Neurosci. 29, 444-453.
88. Matsumoto, M., and Hikosaka, O. (2009). Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459, 837-841.
89. Matsumoto, M., Kenji, M., Hiroshi, A., and Keiji, T. (2007). Medial prefron-tal cell activity signaling prediction errors of action values. Nat. Neurosci. 10, 647-656.
90. Matsumoto, N., Minamimoto, T., Graybiel, A. M., and Kimura, M. (2001). Neurons in the thalamic CM-Pf complex supply striatal neurons with information about behaviorally significant sensory events. J. Neurophysiol. 85, 960-976.
91. Mikula, S., Parrish, S. K., Trimmer, J. S., and Jones, E. G. (2009). Complete 3D visualization of primate striosomes by KChIP1 immunostaining. J. Comp. Neurol. 514, 507-517.
92. Mink, J. W. (1996). The basal ganglia: focused selection and inhibition of competing motor programs. Prog. Neurobiol. 50, 381-425.
93. Miura, M., Masuda, M., and Aosaki, T. (2008). Roles of micro-opioid receptors in GABAergic synaptic transmission in the striosome and matrix compartments of the striatum. Mol. Neurobiol. 37, 104-115.
94. Miura, M., Saino-Saito, S., Masuda, M., Kobayashi, K., and Aosaki, T. (2007). Compartment-specific modulation of GABAergic synaptic transmission by mu-opioid receptor in the mouse striatum with green fluorescent protein-expressing dopamine islands. J. Neurosci. 27, 9721-9728.
95. Montague, P. R., Dayan, P., and Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 16, 1936-1947.
96. Morris, G., Arkadir, D., Nevet, A., Vaadia, E., and Bergman, H. (2004). Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron 43, 133-143.
97. Niki, H., and Watanabe, M. (1979). Prefrontal and cingulate unit activity during timing behavior in the monkey. Brain Res. 171, 213-224.
98. Oorschot, D. E. (1996). Total number of neurons in the neostriatal, pallidal, subthalamic, and substantia nigral nuclei of the rat basal ganglia: a stereological study using the cavalieri and optical disector methods. J. Comp. Neurol. 366, 580-599.
99. Pakhotin, P., and Bracci, E. (2007). Cholinergic interneurons control the excitatory input to the striatum. J. Neurosci. 27, 391-400.
100. Parent, A., and Hazrati, L. N. (1995). Functional anatomy of the basal ganglia. I. The cortico-basal ganglia-thalamo-cortical loop. Brain Res. Brain Res. Rev. 20, 91-127.
101. Parthasarathy, H. B., and Graybiel, A. M. (1997). Cortically driven immediate-early gene expression reflects modular influence of sensorimotor cortex on identified striatal neurons in the squirrel monkey. J. Neurosci. 17, 2477-2491.
102. Penny, G. R., Wilson, C. J., and Kitai, S. T. (1988). Relationship of the axonal and dendritic geometry of spiny projection neurons to the compartmental organization of the neostriatum. J. Comp. Neurol. 269, 275-289.
103. Percheron, G., and Filion, M. (1991). Parallel processing in the basal ganglia: up to a point. Trends Neurosci. 14, 55-59.
104. Ragsdale, C. W. Jr., and Graybiel, A. M. (1988). Fibers from the basolateral nucleus of the amygdala selectively innervate striosomes in the caudate nucleus of the cat. J. Comp. Neurol. 269, 506-522.
105. Reiner, A., Hart, N. M., Lei, W., and Deng, Y. (2010). Corticostriatal projection neurons - dichotomous types and dichotomous functions. Front. Neuroanat. 4:142. doi: 10.3389/fnana.2010.00142
106. Reynolds, J. N., Hyland, B. I., and Wickens, J. R. (2004). Modulation of an afterhyperpolarization by the substantia nigra induces pauses in the tonic firing of striatal cholinergic interneurons. J. eurosci. 24, 9870-9877.
107. Reynolds, N. J., Hyland, B. I., and Wickens, J. R. (2001). A cellular mechanism of reward-related learning. Nature 413, 67-70.
108. Rice, M. E., and Cragg, S. J. (2004). Nicotine amplifies reward-related dopamine signals in striatum. Nat. Neurosci. 7, 583-584.
109. Rogers, R. D., Andrews, T. C., Grasby, P. M., Brooks, D. J., and Robbins, T. W. (2000). Contrasting cortical and subcortical activations produced by attentional-set shifting and reversal learning in humans. J. Cogn. Neurosci. 12, 142-162.
110. Romanelli, P., Esposito, V., Schaal, D. W., and Heit, G. (2005). Somatotopy in the basal ganglia: experimental and clinical evidence for segregated sensorimotor channels. Brain Res. Brain Res. Rev. 48, 112-128.
111. Saka, E., Goodrich, C., Harlan, P., Madras, B. K., and Graybiel, A. M. (2004). Repetitive behaviors in monkeys are linked to specific striatal activation patterns. J. Neurosci. 24, 7557-7565.
112. Saka, E., Iadarola, M., Fitzgerald, D. J., and Graybiel, A. M. (2002). Local circuit neurons in the striatum regulate neural and behavioral responses to dopaminergic stimulation. Proc. Natl. Acad. Sci. U.S.A. 99, 9004-9009.
113. Samejima, K., Doya, K., and Kawato, M. (2003). Inter-module credit assignment in modular reinforcement learning. Neural Netw. 16, 985-994.
114. Samejima, K., Ueda, Y., Doya, K., and Kimura, M. (2005). Representation of action-specific reward values in the striatum. Science 310, 1337-1340.
115. Schultz, W. (1998). Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1-27.
116. Schultz, W., Dayan, P., and Montague, P. R. (1997). A neural substrate of prediction and reward. Science 275, 1593-1599.
117. Shen, W., Flajolet, M., Greengard, P., and Surmeier, D. J. (2008). Dichotomous dopaminergic control of striatal synaptic plasticity. Science 321, 848-851.
118. Surmeier, D. J., Ding, J., Day, M., Wang, Z., and Shen, W. (2007). D1 and D2 dopamine-receptor modulation of striatal glutamatergic signaling in striatal medium spiny neurons. Trends Neurosci. 30, 228-235.
119. Surmeier, D. J., Shen, W., Day, M., Gertler, T., Chan, S., Tian, X., and Plotkin, J. L. (2010). The role of dopamine in modulating the structure and function of striatal circuits. Prog. Brain Res. 183, 149-167.
120. Sutton, R. S., and Barto, A. G. (1998). Reinforcement Learning: An Introduction, Adaptive Computation and Machine Learning. Cambridge, MA: MIT Press
121. Suzuki, T., Miura, M., Nishimura, K., and Aosaki, T. (2001). Dopamine-dependent synaptic plasticity in the striatal cholinergic interneurons. J. Neurosci. 21, 6492-6501.
122. Tippett, L. J., Waldvogel, H. J., Thomas, S. J., Hogg, V. M., van Roon-Mom, W., Synek, B. J., Graybiel, A. M., and Faull, R. L. (2007). Striosomes and mood dysfunction in Huntington's disease. Brain 130(Pt 1), 206-221.
123. Walker, R. H., Arbuthnott, G. W., Baughman, R. W., and Graybiel, A. M. (1993). Dendritic domains of medium spiny neurons in the primate striatum: relationships to striosomal borders. J. Comp. Neurol. 337, 614-628.
124. Wang, Z, Kai, L., Day, M., Ronesi, J., Yin, H. H., Ding, J., Tkatch, T., Lovinger, D. M., and Surmeier, D. J. (2006). Dopaminergic control of corticostriatal long-term synaptic depression in medium spiny neurons is mediated by cholinergic interneurons. Neuron 50, 443-452.
125. Watanabe, K., and Kimura, M. (1998). Dopamine receptor-mediated mechanisms involved in the expressionof learned activity of primate striatal neurons. J. Neurophysiol. 79, 2568-2580.
126. Wilson, C. J. (2004). "Basal ganglia," in The Synaptic Organization of the Brain, ed. G. M. Shepherd (New York: Oxford University Press), 361-413.
127. Wilson, C. J., and Goldberg, J. A. (2006). Origin of the slow afterhyperpolarization and slow rhythmic bursting in striatal cholinergic interneurons. J. Neurophysiol. 95, 196-204.
128. Wolpert, D. M., Doya, K., and Kawato, M. (2003). A unifying computational framework for motor control and social interaction. Philos. Trans. R. Soc. Lond. B Biol. Sci. 358, 593-602.
129. Wolpert, D. M., and Kawato, M. (1998). Multiple paired forward and inverse models for motor control. Neural Netw. 11, 1317-1329.
130. Yan, Z., Song, W. J., and Surmeier, J. (1997). D2 dopamine receptors reduce N-type Ca2+ currents in rat neostriatal cholinergic interneurons through a membrane-delimited, protein-kinase-C-insensitive pathway. J. Neurophysiol. 77, 1003-1015.
131. Zhou, F. M., Liang, Y., and Dani, J. A. (2001). Endogenous nicotinic cholinergic activity regulates dopamine release in the striatum. Nat. Neurosci. 4, 1224-1229.