Oculomotor learning revisited: A model of reinforcement learning in the basal ganglia incorporating an efference copy of motor actions

Frontiers in Neural Circuits

Volumn , Issue JUNE 2012, 2012, Pages

  Fee, Michale S ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Context; Corticostriatal; Efference copy; Motor learning; Songbird; Striatum; Thalamostriatal

Indexed keywords

ANIMAL STRUCTURES; ARTICLE; BASAL GANGLION; BIOLOGICAL MODEL; BRAIN CORTEX; BRAIN FUNCTION; BRAIN REGION; CORPUS STRIATUM; EFFERENT NERVE; EVOKED VISUAL RESPONSE; EVOLUTION; EYE MOVEMENT CONTROL; GUIDED IMAGERY; LEARNING; NERVE CELL; NERVE CELL MEMBRANE POTENTIAL; NERVE CELL NETWORK; NEUROANATOMY; NONHUMAN; REINFORCEMENT; REWARD; SACCADIC EYE MOVEMENT; SIGNAL DETECTION; SONGBIRD; STRUCTURE ACTIVITY RELATION; THALAMUS; ULTRASTRUCTURE; VISUAL DISCRIMINATION; VISUAL STIMULATION; VOCALIZATION;

EID: 84865104951     PISSN: 16625110     EISSN: None     Source Type: Journal    
DOI: 10.3389/fncir.2012.00038     Document Type: Article

References (226)

1. Abramson, B. P., and Chalupa, L. M. (1988). Multiple pathways from the superior colliculus to the extra-geniculate visual thalamus of the cat. J. Comp. Neurol. 271, 397-418.
2. Alexander, G. E., and Crutcher, M. D. (1990). Functional architecture of basal ganglia circuits: neural sub-strates of parallel processing. Trends Neurosci. 13, 266-271.
3. Alexander, G. E., DeLong, M. R., and Strick, P. L. (1986). Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 9, 357-381.
4. Andalman, A. S., and Fee, M. S. (2009). A basal ganglia-forebrain circuit in the songbird biases motor output to avoid vocal errors. Proc. Natl. Acad. Sci. U.S.A. 106, 12518-12523.
5. Aronov, D., Andalman, A. S., and Fee, M. S. (2008). A specialized fore-brain circuit for vocal babbling in the juvenile songbird. Science 320, 630-634.
6. Aronov, D., Veit, L., Goldberg, J. H., and Fee, M. S. (2011). Two distinct modes of forebrain circuit dynamics underlie temporal patterning in the vocalizations of young songbirds. J. Neurosci. 31, 16353-16368.
7. Atallah, H. E., Lopez-Paniagua, D., Rudy, J. W., and O'Reilly, R. C. (2007). Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nat. Neurosci. 10, 126-131.
8. Bar-Gad, I., Morris, G., and Bergman, H. (2003). Information processing,dimensionality reduction and reinforcement learning in the basal ganglia. Prog. Neurobiol. 71, 439-473.
9. Barto, A. G. (1995). "Adaptive critics and the basal ganglia," in Models of Information Processing in the Basal Ganglia, eds J. C. Houk, J. L. Davis, and D. G. Beiser (Cambridge, MA: The MIT Press), 215-232.
10. Basso, M. A., and Wurtz, R. H. (2002). Neuronal activity in substantia nigra pars reticulata during target selection. J. Neurosci. 22, 1883-1894.
11. Bauswein, E., Fromm, C., and Preuss, A. (1989). Corticostriatal cells in comparison with pyramidal tract neurons: contrasting properties in the behaving monkey. Brain Res. 493, 198-203.
12. Beloozerova, I. N., Sirota, M. G., Swadlow, H. A., Orlovsky, G. N., Popova, L. B., and Deliagina, T. G. (2003). Activity of different classes of neurons of the motor cortex during postural corrections. J. Neurosci. 23, 7844-7853.
13. Berns, G. S., and Sejnowski, T. J. (1998). A computational model of how the basal ganglia produce sequences. J. Cogn. Neurosci. 10, 108-121.
14. Berson, D. M., and Graybiel, A. M. (1991). Tectorecipient zone of cat lateral posterior nucleus: evidence that collicular afferents contain acetylcholinesterase. Exp. Brain Res. 84, 478-486.
15. Bottjer, S. W., Miesner, E. A., and Arnold, A. P. (1984). Forebrain lesions disrupt development but not maintenance of song in passerine birds. Science 224, 901-903.
16. Brainard, M. S., and Doupe, A. J. (2002). What songbirds teach us about learning. Nature 417, 351-358.
17. Brown, J. W., Bullock, D., and Grossberg, S. (2004). How laminar frontal cortex and basal ganglia circuits interact to control planned and reactive saccades. Neural Netw. 17, 471-510.
18. Bruce, C. J., and Goldberg, M. E. (1985). Primate frontal eye fields. I.Single neurons discharging before saccades. J. Neurophysiol. 53, 603-635.
19. Butler, A. B. (1994). The evolution of the dorsal thalamus of jawed vertebrates, including mammals: cladistic analysis and a new hypothesis. Brain Res. Brain Res. Rev. 19, 29-65.
20. Cai, X., Kim, S., and Lee, D. (2011). Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice. Neuron 69, 170-182.
21. Carter, A. G., and Sabatini, B. L. (2004). State-dependent calcium signaling in dendritic spines of striatal medium spiny neurons. Neuron 44, 483-493.
22. Charpier, S., and Deniau, J. M. (1997). In vivo activity-dependent plasticity at cortico-striatal connections: evidence for physiological long-term potentiation. Proc. Natl. Acad. Sci. U.S.A. 94, 7036-7040.
23. Chevalier, G., Vacher, S., and Deniau, J. M. (1984). Inhibitory nigral influence on tectospinal neurons, a possible implication of basal ganglia in orienting behavior. Exp. Brain Res. 53, 320-326.
24. Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., and Uchida, N. (2012). Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482, 85-88.
25. Cools, A. R. (1980). Role of the neostriatal dopaminergic activity in sequencing and selecting behavioural strategies: facilitation of processes involved in selecting the best strategy in a stressful situation. Behav. Brain Res. 1, 361-378.
26. Cowan, R. L., and Wilson, C. J. (1994). Spontaneous firing patterns and axonal projections of single corticostriatal neurons in the rat medial agranular cortex. J. Neurophysiol. 71, 17-32.
27. Daw, N. D., and Doya, K. (2006). The computational neurobiology of learning and reward. Curr. Opin. Neurobiol. 16, 199-204.
28. Deschenes, M., and Bourassa, J. (1996). Striatal and cortical projections of single neurons from the central lateral thalamic nucleus in the rat. Neuroscience 72, 679-687.
29. Deschenes, M., Bourassa, J., and Parent, A. (1995). Two different types of thalamic fibers innervate the rat striatum. Brain Res. 701, 288-292.
30. Ding, J., Peterson, J. D., and Surmeier, D. J. (2008). Corticostriatal and thalamostriatal synapses have distinctive properties. J. Neurosci. 28, 6483-6492.
31. Ding, L., and Hikosaka, O. (2006). Comparison of reward modulation in the frontal eye field and caudate of the macaque. J. Neurosci. 26, 6695-6703.
32. Ding, L., Perkel, D. J., and Farries, M. A. (2003). Presynaptic depression of glutamatergic synaptic transmission by D1-like dopamine receptor activation in the avian basal ganglia. J. Neurosci. 23, 6086-6095.
33. Doig, N. M., Moss, J., and Bolam, J. P. (2010). Cortical and thalamic innervation of direct and indirect pathway medium-sized spiny neurons in mouse striatum. J. Neurosci. 30, 14610-14618.
34. Doupe, A. J., Perkel, D. J., Reiner, A., and Stern, E. A. (2005). Birdbrains could teach basal ganglia research a new song. Trends Neurosci. 28, 353-363.
35. Doya, K. (2000). Complementary roles of basal ganglia and cerebellum in learning and motor control. Curr. Opin. Neurobiol. 10, 732-739.
36. Doya, K., and Sejnowski, T. (1995). A novel reinforcement model of birdsong vocalization learning. Adv. Neural Inf. Process. Syst. 7, 101-108.
37. Doya, K., and Sejnowski, T. J. (2000). "A computational model of avian song learning," in The New Cognitive Neurosciences, ed M. Gazzaniga (Cambridge, MA: MIT Press), 469-482.
38. Erro, E., Lanciego, J. L., and Gimenez-Amaya, J. M. (1999). Relationships between thalamostriatal neurons and pedunculopontine projections to the thalamus: a neuroanatomical tract-tracing study in the rat. Exp. Brain Res. 127, 162-170.
39. Farries, M. A., Ding, L., and Perkel, D. J. (2005). Evidence for "direct" and "indirect" pathways through the song system basal ganglia. J. Comp. Neurol. 484, 93-104.
40. Farries, M. A., and Fairhall, A. L. (2007). Reinforcement learning with modulated spike timing dependent synaptic plasticity. J. Neurophysiol. 98, 3648-3665.
41. Farries, M. A., and Perkel, D. J. (2002). A telencephalic nucleus essential for song learning contains neurons with physiological characteristics of both striatum and globus pallidus. J. Neurosci. 22, 3776-3787.
42. Featherstone, R. E., and McDonald, R. J. (2004). Dorsal striatum and stimulus-response learning: lesions of the dorsolateral, but not dorsomedial, striatum impair acquisition of a stimulus-response-based instrumental discrimination task, while sparing conditioned place preference learning. Neuroscience 124, 23-31.
43. Fee, M. S., and Goldberg, J. H. (2011). A hypothesis for basal ganglia-dependent reinforcement learning in the songbird. Neuroscience 198, 152-170.
44. Fee, M. S., and Scharff, C. (2010). The songbird as a model for the generation and learning of complex sequential behaviors. ILAR J. 5, 362-377.
45. Fiete, I. R., Fee, M. S., and Seung, H. S. (2007). Model of birdsong learning based on gradient estimation by dynamic perturbation of neural conductances. J. Neurophysiol. 98, 2038-2057.
46. Fiete, I. R., Hahnloser, R. H. R., Fee, M. S., and Seung, H. S. (2004). Temporal sparseness of the premotor drive is important for rapid learning in a neural network model of birdsong. J. Neurophysiol. 92, 2274-2282.
47. Fitzpatrick, D. C., Kanwal, J. S., Butman, J. A., and Suga, N. (1993). Combination-sensitiveneurons in the primary auditory cortex of the mustached bat. J. Neurosci. 13, 931-940.
48. Flaherty, A. W., and Graybiel, A. M. (1991). Corticostriatal transformations in the primate somatosensory system. Projections from physiologically mapped body-partrepresentations.J. Neurophysiol. 66, 1249-1263.
49. Flaherty, A. W., and Graybiel, A. M. (1993). Two input systems for body representations in the primate striatal matrix: experimental evidence in the squirrel monkey. J. Neurosci. 13, 1120-1137.
50. Flaherty, A. W., and Graybiel, A. M. (1994). Input-output organization of the sensorimotor striatum in the squirrel monkey. J. Neurosci. 14, 599-610.
51. Frank, M. J., and O'Reilly, R. C. (2006). A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol. Behav. Neurosci. 120, 497-517.
52. Freiwald, W. A., and Tsao, D. Y. (2010). Functional compartmentalization and viewpoint generalization within the macaque face-processing system. Science 330, 845-851.
53. Freund, T. F., Powell, J. F., and Smith, A. D. (1984). Tyrosine hydroxylase-immunoreactiveboutonsin synaptic contact with identified striatonigral neurons, with particular reference to dendritic spines. Neuroscience 13, 1189-1215.
54. Fujimoto, H., Hasegawa, T., and Watanabe, D. (2011). Neural coding of syntactic structure in learned vocalizations in the songbird. J. Neurosci. 31, 10023-10033.
55. Funahashi, S., Bruce, C. J., and Goldman-Rakic, P. S. (1991). Neuronal activity related to saccadic eye movements in the monkey's dorsolateral prefrontal cortex. J. Neurophysiol. 65, 1464-1483.
56. Gale, S. D., and Perkel, D. J. (2005). Properties of dopamine release and uptake in the songbird basal ganglia. J. Neurophysiol. 93, 1871-1879.
57. Gale, S. D., and Perkel, D. J. (2010). A basal ganglia pathway drives selective auditory responses in songbird dopaminergic neurons via disinhibition. J. Neurosci. 30, 1027-1037.
58. Gale, S. D., Person, A. L., and Perkel, D. J. (2008). A novel basal ganglia pathway forms a loop linking a vocal learning circuit with its dopaminergic input. J. Comp. Neurol. 508, 824-839.
59. Ganz, J., Kaslin, J., Freudenreich, D., Machate, A., Geffarth, M., and Brand, M. (2012). Subdivisions of the adult zebrafish subpallium by molecular marker analysis. J. Comp. Neurol. 520, 633-655.
60. Garcia-Rill, E. (1986). The basal ganglia and the locomotor regions. Brain Res. 11, 47-63.
61. Gerfen, C. R. (1985). The neostriatal mosaic. I. Compartmental organization of projections from the striatum to the substantia nigra in the rat. J. Comp. Neurol. 236, 454-476.
62. Gerfen, C. R., Engber, T. M., Mahan, L. C., Susel, Z., Chase, T. N., Monsma, F. J., and Sibley, D. R. (1990). D1 and D2 dopamine receptor-regulated gene expression of striatonigral and striatopallidal neurons. Science 250, 1429-1432.
63. Gerfen, C. R., Herkenham, M., and Thibault, J. (1987). The neostriatal mosaic: II. Patch-and matrix-directed mesostriatal dopaminergic and non-dopaminergic systems. J. Neurosci. 7, 3915-3934.
64. Gerfen, T. S., and Wilson, C. J. (1996). "The basal ganglia," in Handbook of Chemical Anatomy. Integrated System of the CNS, Part III, eds L. W. Swanson, A. Bjorklund, and T. Hokfelt (Amsterdam: Elsevier), 360-466.
65. Goldberg, J. H., Adler, A., Bergman, H., and Fee, M. S. (2010). Singing-related neural activity distinguishes two putative pallidal cell types in the songbird basal ganglia: comparison to the primate internal and external pallidal segments. J. Neurosci. 30, 7088-7098.
66. Goldberg, J. H., and Fee, M. S. (2010). Singing-related neural activity distinguishes four classes of putative striatal neurons in the songbird basal ganglia. J. Neurophysiol. 103, 2002-2014.
67. Goldberg, J. H., and Fee, M. S. (2011). Vocal babbling in songbirds requires the basal ganglia-recipient motor thalamus but not the basal ganglia. J. Neurophysiol. 105, 2729-2739.
68. Goldman-Rakic, P. S., and Selemon, L. D. (1986). Topography of corti-costriatal projections in nonhuman primates and implications for functional parcellation of the neostriatum. Cereb. Cortex 5, 447-466.
69. Graybiel, A. M. (1972). Some ascending connections of the pulvinar and nucleus lateralis posterior of the thalamus in the cat. Brain Res. 44, 99-125.
70. Graybiel, A. M. (1978). Organization of the nigrotectal connection: an experimental tracer study in the cat. Brain Res. 143, 339-348.
71. Graybiel, A. M. (1998). The basal ganglia and chunking of action repertoires. Neurobiol. Learn. Mem. 70, 119-136.
72. Graybiel, A. M. (2005). The basal ganglia: learning new tricks and loving it. Curr. Opin. Neurobiol. 15, 638-644.
73. Graybiel, A. M. (2008). Habits, rituals, and the evaluative brain. Annu. Rev. Neurosci. 31, 359-387.
74. Graybiel, A. M., Aosaki, T., Flaherty, A. W., and Kimura, M. (1994). The basal ganglia and adaptive motor control. Science 265, 1826-1831.
75. Graybiel, A. M., and Ragsdale, C. W. J. (1978). Histochemically distinct compartments in the striatum of human, monkeys, and cat demonstrated by acetylthiocholinesterase staining. Proc. Natl. Acad. Sci. U.S.A. 75, 5723-5726.
76. Grillner, S. (2003). The motor infrastructure: from ion channels to neuronal networks. Nat. Rev. Neurosci. 4, 573-586.
77. Grillner, S., Hellgren, J., Menard, A., Saitoh, K., and Wikstrom, M. A. (2005). Mechanisms for selection of basic motor programs-roles for the striatum and pallidum. Trends Neurosci. 28, 364-370.
78. Gurney, K., Prescott, T. J., and Redgrave, P. (2001). A computational model of action selection in the basal ganglia. I. A new functional anatomy. Biol. Cybern. 84, 401-410.
79. Hahnloser, R. H. R., Kozhevnikov, A. A., and Fee, M. S. (2002). An ultra-sparse code underlies the generation of neural sequences in a songbird. Nature 419, 65-70.
80. Hampton, C. M., Sakata, J. T., and Brainard, M. S. (2009). An avian basalganglia-forebraincircuit contributes differentially to syllable versus sequence variability of adult Bengalese finch song. J. Neurophysiol. 101, 3235-3245.
81. Harding, C. F. (2004). Brief alteration in dopaminergic function during development causes deficits in adult reproductive behavior. J. Neurobiol. 61, 301-308.
82. Harting, J. K., Huerta, M. F., Frankfurter, J., Strominger, N. L., and Royce, G. J. (1980). Ascending pathways from the monkey superior colliculus: an autoradiographic analysis. J. Comp. Neurol. 192, 853-882.
83. Hessler, N. A., and Doupe, A. J. (1999). Singing-related neural activity in a dorsal forebrain-basal ganglia circuit of adult zebra finches. J. Neurosci. 19, 10461-10481.
84. Hikosaka, O. (2007). GABAergic output of the basal ganglia. Prog. Brain Res. 160, 209-226.
85. Hikosaka, O., Nakahara, H., Rand, M. K., Sakai, K., Lu, X., Nakamura, K., Miyachi, S., and Doya, K. (1999). Parallel neural networks for learning sequential procedures. Trends Neurosci. 22, 464-471.
86. Hikosaka, O., Nakamura, K., and Nakahara, H. (2006). Basal ganglia orient eyes to reward. J. Neurophysiol. 95, 567-584.
87. Hikosaka, O., Rand, M. K., Nakamura, K., Miyachi, S., Kitaguchi, K., Sakai, K., Lu, X., and Shimo, Y. (2002). Long-term retention of motor skill in macaque monkeys and humans. Exp. Brain Res. 147, 494-504.
88. Hikosaka, O., Sakamoto, M., and Usui, S. (1989a). Functional properties of monkey caudate neurons. I. Activities related to saccadic eye movements. J. Neurophysiol. 61, 780-798.
89. Hikosaka, O., Sakamoto, M., and Usui, S. (1989b). Functional properties of monkey caudate neurons. II. Visual and auditory responses. J. Neurophysiol. 61, 799-813.
90. Hikosaka, O., Sakamoto, M., and Usui, S. (1989c). Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. J. Neurophysiol. 61, 814-832.
91. Hikosaka, O., Takikawa, Y., and Kawagoe, R. (2000). Role of the basal ganglia in the control of purposive saccadic eye movements. Physiol. Rev. 80, 953-978.
92. Hikosaka, O., and Wurtz, R. H. (1983a). Visual and oculomotor functions of monkey substantia nigra pars reticulata. I. Relation of visual and auditory responses to saccades. J. Neurophysiol. 49, 1230-1253.
93. Hikosaka, O., and Wurtz, R. H. (1983b). Visual and oculomotor functions of monkey substantia nigra pars reticulata. IV. Relation of substantia nigra to superior colliculus. J. Neurophysiol. 49, 1285-1301.
94. Hikosaka, O., and Wurtz, R. H. (1985). Modification of saccadic eye movements by GABA-related substances. II. Effects of muscimol in monkey substantia nigra pars reticulata. J. Neurophysiol. 53, 292-308.
95. Hoover, J. E., and Strick, P. L. (1993). Multiple output channels in the basal ganglia. Science 259, 819-821.
96. Houk, J. C. (1995). "Information processing in modular circuits linking basal ganglia and cerebral cortex," in Models of Information Processing in the Basal Ganglia, eds J. C. Houk, J. L. Davis, and D. G. Beiser (Cambridge, MA: The MIT Press), 3-10.
97. Houk, J. C., Adams, J. L., and Barto, A. G. (1995). "A model of how the basal ganglia generate and use neural signals that predict reinforcement," in Models of Information Processing in the Basal Ganglia, eds J. C. Houk, J. L. Davis, and D. G. Beiser (Cambridge, MA: The MIT Press), 249-270.
98. Houk, J. C., and Wise, S. P. (1995). Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: their role in planning and controlling action. Cereb. Cortex 5, 95-110.
99. Ichinohe, N., Iwatsuki, H., and Shoumura, K. (2001). Intrastriatal targets of projection fibers from the central lateral nucleus of the rat thalamus. Neurosci. Lett. 302, 105-108.
100. Ikeda, T., and Hikosaka, O. (2003). Reward-dependent gain and bias of visual responses in primate superior colliculus. Neuron 39, 693-700.
101. Ito, M., and Doya, K. (2011). Multiple representations and algorithms for reinforcement learning in the cortico-basal ganglia circuit. Curr. Opin. Neurobiol. 21, 368-373.
102. Iyengar, S., Viswanathan, S. S., and Bottjer, S. W. (1999). Development of topography within song control circuitry of zebra finches during the sensitive period for song learning. J. Neurosci. 19, 6037-6057.
103. Izhikevich, E. M. (2007). Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb. Cortex 17, 2443-2452.
104. Jarvis, E. D. (2004). Learned birdsong and the neurobiology of human language. Ann. N.Y. Acad. Sci. 1016, 749-777.
105. Jayaraman, A., Batton, R. R., and Carpenter, M. B. (1977). Nigrotectalprojectionsin the monkey: an autoradiographic study. Brain Res. 135, 147-152.
106. Johnson, F., Sablan, M. M., and Bottjer, S. W. (1995). Topographic organization of a forebrain pathway involved with vocal learning in zebra finches. J. Comp. Neurol. 358, 260-278.
107. Kao, M. H., and Brainard, M. S. (2006). Lesions of an avian basal ganglia circuit prevent context-dependent changes to song variability. J. Neurophysiol. 96, 1441-1455.
108. Kao, M. H., Doupe, A. J., and Brainard, M. S. (2005). Contributions of an avian basal ganglia-forebrain circuit to real-time modulation of song. Nature 433, 638-643.
109. Kao, M. H., Wright, B. D., and Doupe, A. J. (2008). Neurons in a forebrain nucleus required for vocal plasticity rapidly switch between precise firing and variable bursting depending on social context. J. Neurosci. 28, 13232-13247.
110. Kawagoe, R., Takikawa, Y., and Hikosaka, O. (1998). Expectation of reward modulates cognitive signals in the basal ganglia. Nat. Neurosci. 1, 411-416.
111. Kawagoe, R., Takikawa, Y., and Hikosaka, O. (2004). Reward-predicting activity of dopamine and caudate neuronsa possible mechanism of motivational control of saccadic eye movement. J. Neurophysiol. 91, 1013-1024.
112. Kemp, J. M., and Powell, T. P. (1971). The structure of the caudate nucleus of the cat: light and electron microscopy. Philos. Trans. R. Soc. Lond. B Biol. Sci. 262, 383-401.
113. Kincaid, A. E., and Wilson, C. J. (1996). Corticostriatal innervation of the patch and matrix in the rat neostriatum. J. Comp. Neurol. 374, 578-592.
114. Kincaid, A. E., Zheng, T., and Wilson, C. J. (1998). Connectivity and convergence of single corticostriatal axons. J. Neurosci. 18, 4722-4731.
115. Kobayashi, S., Kawagoe, R., Takikawa, Y., Koizumi, M., Sakagami, M., and Hikosaka, O. (2007). Functional differences between macaque prefrontal cortex and caudate nucleus during eye movements with and without reward. Exp. Brain Res. 176, 341-355.
116. Komatsu, H., and Suzuki, H. (1985). Projections from the functional subdivisions of the frontal eye field to the superior colliculus in the monkey. Brain Res. 327, 324-327.
117. Kozhevnikov, A. A., and Fee, M. S. (2007). Singing-related activity of identified HVC neurons in the zebra finch. J. Neurophysiol. 97, 4271-4283.
118. Kreitzer, A. C., and Malenka, R. C. (2008). Striatal plasticity and basal ganglia circuit function. Neuron 60, 543-554.
119. Krout, K. E., Loewy, A. D., Westby, G. W., and Redgrave, P. (2001). Superior colliculus projections to midline and intralaminar thalamic nuclei of the rat. J. Comp. Neurol. 431, 198-216.
120. Kubikova, L., and Kostal, L. (2010). Dopaminergic system in birdsong learning and maintenance. J. Chem. Neuroanat. 39, 112-123.
121. Kubikova, L., Wada, K., and Jarvis, E. D. (2010). Dopamine receptors in a songbird brain. J. Comp. Neurol. 518, 741-769.
122. Kunimatsu, J., and Tanaka, M. (2010). Roles of the primate motor thalamus in the generation of antisaccades. J. Neurosci. 30, 5108-5117.
123. Kunzle, H., and Akert, K. (1977). Efferent connections of cortical, area 8 (frontal eye field) in Macaca fascicularis. A reinvestigation using the autoradiographic technique. J. Comp. Neurol. 173, 147-163.
124. Las, L., Denissenko, N. I., MandelblatDerf, Y., and Fee, M. S. (2011). "A cortical projection to the ventral tegmental area (VTA) is necessary for vocal learning in the songbird," in Neuroscience Meeting Planner, Program (Washington, DC: Society for Neuroscience).
125. Lau, B., and Glimcher, P. W. (2008). Value representations in the primate striatum during matching behavior. Neuron 58, 451-463.
126. Lauwereyns, J., Watanabe, K., Coe, B., and Hikosaka, O. (2002). A neural correlate of response bias in monkey caudate nucleus. Nature 418, 413-417.
127. Levesque, M., Charara, A., Gagnon, S., Parent, A., and Deschenes, M. (1996a). Corticostriatal projections from layer V cells in rat are collaterals of long-range corticofugal axons. Brain Res. 709, 311-315.
128. Levesque, M., Gagnon, S., Parent, A., and Deschenes, M. (1996b). Axonal arborizations of corticostriatal and corticothalamic fibers arising from the second somatosensory area in the rat. Cereb. Cortex 6, 759-770.
129. Levesque, M., and Parent, A. (1998). Axonal arborization of corti-costriataland corticothalamic fibers arising from prelimbic cortex in the rat. Cereb. Cortex 8, 602-613.
130. Li, N., and DiCarlo, J. J. (2010). Unsupervised natural visual experience rapidly reshapes size-invariant object representation in inferior temporal cortex. Neuron 67, 1062-1075.
131. Long, M. A., and Fee, M. S. (2008). Using temperature to analyse temporal dynamics in the songbird motor pathway. Nature 456, 189-194.
132. Long, M. A., Jin, D. Z., and Fee, M. S. (2010). Support for a synaptic chain model of neuronal sequence generation. Nature 468, 394-399.
133. Luo, M., Ding, L., and Perkel, D. J. (2001). An avian basal ganglia pathway essential for vocal learning forms a closed topographic loop. J. Neurosci. 21, 6836-6845.
134. Luo, M., and Perkel, D. J. (1999). Longrange GABAergic projection in a circuit essential for vocal learning. J. Comp. Neurol. 403, 68-84.
135. Maass, W. (2000). On the computational power of winner-take-all. Neural Comput. 12, 2519-2535.
136. Margoliash, D., and Yu, A. C. (1996). Temporal hierarchical control of singing in birds. Science 273, 1871-1875.
137. May, P. J., and Hall, W. C. (1984). Relationships between the nigrotectal pathway and the cells of origin of the predorsal bundle. J. Comp. Neurol. 226, 357-376.
138. McDonald, R. J., and White, N. M. (1994). Parallel information processing in the water maze: evidence for independent memory systems involving dorsal striatum and hippocampus. Behav. Neural Biol. 61, 260-270.
139. McFarland, N. R., and Haber, S. N. (2001). Organization of thalamostriatal terminals from the ventral motor nuclei in the macaque. J. Comp. Neurol. 429, 321-336.
140. McHaffie, J. G., Stanford, T. R., Stein, B. E., Coizet, V., and Redgrave, P. (2005). Subcortical loops through the basal ganglia. Trends Neurosci. 28, 401-407.
141. Mengual, E., de las Heras, S., Erro, E., Lanciego, J. L., and Gimenez-Amaya, J. M. (1999). Thalamic interaction between the input and the output systems of the basal ganglia. J. Chem. Neuroanat. 16, 187-200.
142. Miller, E. K., and Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167-202.
143. Mink, J. W. (1996). The basal ganglia: focused selection and inhibition of competing motor programs. Prog. Neurobiol. 50, 381-425.
144. Montague, P. R., Dayan, P., and Sejnowski, T. J. (1996). A frame-work for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 16, 1936-1947.
145. Nambu, A. (2004). A new dynamic model of the cortico-basal ganglia loop. Prog. Brain Res. 143, 461-466.
146. Nieuwenhuys, R., Donkelaar, H. J., and Nicholson, C. (1998). The Central Nervous System of Vertebrates. Heidelberg: Springer Verlag.
147. Nottebohm, F., Kelley, D. B., and Paton, J. A. (1982). Connections of vocal control nuclei in the canary telencephalon. J. Comp. Neurol. 207, 344-357.
148. Olveczky, B. P., Andalman, A. S., and Fee, M. S. (2005). Vocal experimentation in the juvenile songbird requires a basal ganglia circuit. PLoS Biol. 3:e153. doi: 10.1371/journal.pbio.0030153
149. Olveczky, B. P., Otchy, T. M., Goldberg, J. H., Aronov, D., and Fee, M. S. (2011). Changes in the neural control of a complex motor sequence during learning. J. Neurophysiol. 106, 386-397.
150. Packard, M. G., Hirsh, R., and White, N. M. (1989). Differential effects of fornix and caudate nucleus lesions on two radial maze tasks: evidence for multiple memory systems. J. Neurosci. 9, 1465-1472.
151. Packard, M. G., and Knowlton, B. J. (2002). Learning and memory functions of the basal ganglia. Annu. Rev. Neurosci. 25, 563-593.
152. Packard, M. G., and McGaugh, J. L. (1996). Inactivation of hippocampus or caudate nucleus with lidocaine differentially affects expression of place and response learning. Neurobiol. Learn. Mem. 65, 65-72.
153. Parent, M., and Parent, A. (2005). Single-axon tracing and three-dimensional reconstruction of centre median-parafascicular thalamic neurons in primates. J. Comp. Neurol. 481, 127-144.
154. Parent, M., and Parent, A. (2006). Single-axon tracing study of corticostriatalprojectionsarising from primary motor cortex in primates. J. Comp. Neurol. 496, 202-213.
155. Pasupathy, A., and Miller, E. K. (2005). Different time courses of learning-related activity in the prefrontal cortex and striatum. Nature 433, 873-876.
156. Pawlak, V., and Kerr, J. N. (2008). Dopamine receptor activation isrequired for corticostriatal spike-timing-dependent plasticity. J. Neurosci. 28, 2435-2446.
157. Person, A. L., Gale, S. D., Farries, M. A., and Perkel, D. J. (2008). Organization of the songbird basal ganglia, including area X. J. Comp. Neurol. 508, 840-866.
158. Plenz, P., and Wickens, J. R. (2010). "The striatal skeleton: medium spiny projection neurons and their lateral connections," in Handbook of Basal Ganglia Structure and Function, eds H. Steiner and K. Y. Tseng (Amsterdam: Elsevier), 99-112.
159. Plotkin, J. L., Day, M., and Surmeier, D. J. (2011). Synaptically driven state transitions in distal dendrites of striatal spiny neurons. Nat. Neurosci. 14, 881-888.
160. Prather, J. F., Peters, S., Nowicki, S., and Mooney, R. (2008). Precise auditory-vocal mirroring in neurons for learned vocal communication. Nature 451, 305-310.
161. Rainer, G., Asaad, W. F., and Miller, E. K. (1998). Memory fields of neurons in the primate prefrontal cortex. Proc. Natl. Acad. Sci. U.S.A. 95, 15008.
162. Ramony C. S. (1911). Histologie du Systeme Nerveux de l Homme et des Vertebres. Paris: Maloine 1, 174-192.
163. Redgrave, P., Prescott, T. J., and Gurney, K. (1999). The basal ganglia: a vertebrate solution to the selection problem? Neuroscience 89, 1009-1023.
164. Redondo, R. L., and Morris, R. G. (2011). Making memories last: the synaptic tagging and capture hypothesis. Nat. Rev. Neurosci. 12, 17-30.
165. Reiner, A., Hart, N. M., Lei, W., and Deng, Y. (2010). Corticostriatal projection neurons-dichotomous types and dichotomous functions. Front. Neuroanat. 4:142. doi: 10.3389/fnana.2010.00142
166. Reiner, A., Jiao, Y., Del Mar, N., Laverghetta, A. V., and Lei, W. L. (2003). Differential morphology of pyramidal tract-type and intratelencephalically projecting-type corticostriatal neurons and their intrastriatal terminals in rats. J. Comp. Neurol. 457, 420-440.
167. Reiner, A., Medina, L., and Veenman, C. L. (1998). Structural and functional evolution of the basal ganglia in vertebrates. Brain Res. Brain Res. Rev. 28, 235-285.
168. Reiner, A., Perkel, D. J., Bruce, L. L., Butler, A. B., Csillag, A., Kuenzel, W., Medina, L., Paxinos, G., Shimizu, T., Striedter, G., Wild, M., Ball, G. F., Durand, S., Gunturkun, O., Lee, D. W., Mello, C. V., Powers, A., White, S. A., Hough, G., Kubikova, L., Smulders, T. V., Wada, K., Dugas-Ford, J., Husband, S., Yamamoto, K., Yu, J., Siang, C., Jarvis, E. D., Guturkun, O., and Forum, A. B. N. (2004). Revised nomenclature for avian telencephalon and some related brainstem nuclei. J. Comp. Neurol. 473, 377-414.
169. Reynolds, J. N. J., Hyland, B. I., and Wickens, J. R. (2001). A cellular mechanism of reward-related learning. Nature 413, 67-70.
170. Robinson, D. A. (1972). Eye movements evoked by collicular stimulation in the alert monkey. Vision Res. 12, 1795-1808.
171. Robinson, D. A., and Fuchs, A. F. (1969). Eye movements evoked by stimulation of frontal eye fields. J. Neurophysiol. 32, 637-648.
172. Romo, R., Brody, C. D., Hernandez, A., and Lemus, L. (1999). Neuronal correlates of parametric working memory in the prefrontal cortex. Nature 399, 470-473.
173. Roth, G., Laberge, F., Muhlenbrock-Lenter, S., and Grunwald, W. (2007). Organization of the pallium in the fire-bellied toad Bombina orientalis. I: morphology and axonal projection pattern of neurons revealed by intracellular biocytin labeling. J. Comp. Neurol. 501, 443-464
174. Royce, G. J. (1983). Single thalamic neurons which project to both the rostra1 cortex and caudate nucleus studied with the fluorescent double labeling method. Exp. Neurol. 79, 773-784.
175. Russell, J. A., and Bullock, M. (1985). Multidimensional scaling of emotional facial expressions: similarity from preschoolers to adults. J. Pers. Soc. Psychol. 48, 1290.
176. Sadikot, A. F., Parent, A., Smith, Y., and Bolam, J. P. (1992). Efferent connections of the centromedian and parafascicular thalamic nuclei in the squirrel monkey: a light and electron microscopic study of the thalamostriatal projection in relation to striatal heterogeneity. J. Comp. Neurol. 320, 228-242.
177. Samejima, K., Ueda, Y., Doya, K., and Kimura, M. (2005). Representation of action-specific reward values in the striatum. Science 310, 1337-1340.
178. Sato, M., and Hikosaka, O. (2002). Role of primate substantia nigra pars reticulata in reward-oriented saccadic eye movement. J. Neurosci. 22, 2363-2373.
179. Scharff, C., and Nottebohm, F. (1991). A comparative study of the behavioral deficits following lesions of various parts of the zebra finch song system: implications for vocal learning. J. Neurosci. 11, 2896-2913.
180. Schlag-Rey, M., Schlag, J., and Dassonville, P. (1992). How the frontal eye field can impose a saccade goal on superior colliculus neurons. J. Neurophysiol. 67, 1003-1005.
181. Schultz, W. (2002). Getting formal with dopamine and reward. Neuron 36, 241-263.
182. Schultz, W., Dayan, P., and Montague, P. R. (1997). A neural substrate of prediction and reward. Science 275, 1593-1599.
183. Shen, W., Flajolet, M., Greengard, P., and Surmeier, D. J. (2008). Dichotomous dopaminergic control of striatal synaptic plasticity. Science 321, 848-851.
184. Sidibe, M., and Smith, Y. (1996). Differential synaptic innervations of striatofugal neurones projecting to the internal or external segments of the globus pallidus by thalamic afferents in the squirrel monkey. J. Comp. Neurol. 365, 445-465.
185. Smith, A. D., and Bolam, J. P. (1990). The neural network of the basal ganglia as revealed by the study of synaptic connections of identified neurones. Trends Neurosci. 13, 1-7.
186. Smith, Y., Bennett, B. D., Bolam, J. P., Parent, A., and Sadikot, A. F. (1994). Synaptic relationships between dopaminergic afferents and cortical or thalamic input in the sensorimotor territory of the striatum in monkey. J. Comp. Neurol. 344, 1-19.
187. Smith, Y., Bevan, M. D., Shink, E., and Bolam, J. P. (1998). Microcircuitry of the direct and indirect pathways of the basal ganglia. Neuroscience 86, 353-387.
188. Smith, Y., Raju, D. V., Pare, J.-F., and Sidibe, M. (2004). The thalamostriatal system: a highly specific network of the basal ganglia circuitry. Trends Neurosci. 27, 520-527.
189. Smith, Y., Surmeier, D. J., Redgrave, P., and Kimura, M. (2011). Thalamic contributions to Basal Ganglia-related behavioral switching and reinforcement. J. Neurosci. 31, 16102-16106.
190. Sober, S. J., Wohlgemuth, M. J., and Brainard, M. S. (2008). Central contributions to acoustic variation in birdsong. J. Neurosci. 28, 10370-10379.
191. Sohrabji, F., Nordeen, E. J., and Nordeen, K. W. (1990). Selective impairment of song learning following lesions of a forebrain nucleus in the juvenile zebra finch. Behav. Neural Biol. 53, 51-63.
192. Sommer, M. A., and Wurtz, R. H. (2002). A pathway in primate brain for internal monitoring of movements. Science 296, 1480-1482.
193. Sommer, M. A., and Wurtz, R. H. (2008). Brain circuits for the internal monitoring of movements. Annu. Rev. Neurosci. 31, 317-338.
194. Stepanek, L., and Doupe, A. J. (2010). Activity in a cortical-basal ganglia circuit for song is required for social context-dependent vocal variability. J. Neurophysiol. 104, 2474-2486.
195. Stephenson-Jones, M., Ericsson, J., Robertson, B., Grillner, S. (2012). Evolution of the basal ganglia; dual output pathways conserved throughout vertebrate phylogeny. J. Comp. Neurol. doi: 10.1002/cne. 23087. [Epub ahead of print].
196. Stern, E. A., Jaeger, D., and Wilson, C. J. (1998). Membrane potential synchrony of simultaneously recorded striatal spiny neurons in vivo. Nature 394, 475-478.
197. Suri, R. E., and Schultz, W. (1999). A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience 91, 871-890.
198. Sutton, R. S., and Barto, A. G. (1998). Reinforcement Learning: An Introduction. Cambridge, MA: The MIT Press.
199. Swanson, L. W. (2000). Cerebral hemisphere regulation of motivated behavior. Brain Res. 886, 113-164.
200. Takada, M., Tokuno, H., Ikai, Y., and Mizuno, N. (1994). Direct projections from the entopeduncular nucleus to the lower brainstem in the rat. J. Comp. Neurol. 342, 409-429.
201. Takakusaki, K., Habaguchi, T., Ohtinata-Sugimoto, J., Saitoh, K., and Sakamoto, T. (2003). Basal ganglia efferents to the brain-stem centers controlling postural muscle tone and locomotion: a new concept for understanding motor disorders in basal ganglia dysfunction. Neuroscience 119, 293-308.
202. Takakusaki, K., Saitoh, K., Harada, H., and Kashiwayanagi, M. (2004). Role of basal ganglia-brainstem pathways in the control of motor behaviors. Neurosci. Res. 50, 137-151.
203. Tanaka, K. (1996). Inferotemporal cortex and object vision. Annu. Rev. Neurosci. 19, 109-139.
204. Thorndike, E. L.(1911). Animal Intelligence. Darien, CT: Hafner.
205. Tseng, K. Y., and Steiner, H. (2010). Handbook of Basal Ganglia Structure and Function. Amsterdam: Elsevier.
206. Tumer, E. C., and Brainard, M. S. (2007). Performance variability enables adaptive plasticity of 'crystallized' adult birdsong. Nature 450, 1240-1244.
207. Turner, R. S., and DeLong, M. R. (2000). Corticostriatal activity in primary motor cortex of the macaque. J. Neurosci. 20, 7096-7108.
208. Vates, G. E., and Nottebohm, F. (1995). Feedback circuitry within a song-learning pathway. Proc. Natl. Acad. Sci. U.S.A. 92, 5139-5143.
209. Vicario, D. S. (1991). Organization of the zebra finch song control system: functional organization of outputs from nucleus robustus archistriatalis. J. Comp. Neurol. 309, 486-494.
210. Warren, T. L., Tumer, E. C., Charlesworth, J. D., and Brainard, M. S. (2011). Mechanisms and time course of vocal learning and consolidation in the adult songbird. J. Neurophysiol. 106, 1806-1821.
211. Watanabe, K., Lauwereyns, J., and Hikosaka, O. (2003). Neural correlates of rewarded and unrewarded eye movements in the primate caudate nucleus. J. Neurosci. 23, 10052-10057.
212. Wickens, J., and Kötter, R. (1995). "Cellular models of reinforcement," in Models of Information Processing in the Basal Ganglia, eds J. C. Houk, J. L. Davis, and D. G. Beiser (Cambridge MA: The MIT Press), 187-214.
213. Wickens, J. R., Alexander, M. E., and Miller, R. (1991). Two dynamic modes of striatal function under dopaminergic-cholinergic control: simulation and analysis of a model. Synapse 8, 1-12.
214. Wickens, J. R., and Arbuthnott, G. W. (2010). "Gating of cortical input to the striatum," in Handbook of Basal Ganglia Structure and Function, a Decade of Progress, eds H. Steiner, and K. Y. Tseng (Amsterdam: Elsevier), 341-352.
215. Wilczynski, W., and Northcutt, R. G. (1983). Connections of the bull-frog striatum: afferent organization. J. Comp. Neurol. 214, 321-332.
216. Wilson, C. J. (1987). Morphology and synaptic connections of crossed corticostriatal neurons in the rat. J. Comp. Neurol. 263, 567-580.
217. Wilson, C. J. (2007). GABAergic inhibition in the neostriatum. Prog. Brain Res. 160, 91-110.
218. Wilson, C. J., Chang, H. T., and Kitai, S. T. (1983). Origins of post synaptic potentials evoked in spiny neostriatal projection neurons by thalamic stimulation in the rat. Exp. Brain Res. 51, 217-226.
219. Wilson, C. J., and Goves, P. M. (1980). Fine structure and synaptic connections of the common spiny neuron of the rat neostriatum: a study employing intracellular injection of horseradish peroxidase. J. Comp. Neurol. 194, 599-615.
220. Wilson, C. J., and Kawaguchi, Y. (1996). The origins of two-state spontaneous membrane potential fluctuations of neostriatal spiny neurons. J. Neurosci. 16, 2397-2410.
221. Winn, P., Wilson, D. I. G., and Redgrave, P. (2010). "Subcortical connections of the basal ganglia," in Handbook of Basal Ganglia Structure and Function, eds H. Steiner and K. Y. Tseng (Amsterdam: Elsevier), 397-408.
222. Wright, A. K., Ramanathan, S., and Arbuthnott, G. W. (2001). Identification of the source of the bilateral projection system from cortex to somatosensory neostriatum and an exploration of its physiological actions. Neuroscience 103, 87-96.
223. Wurtz, R. H., and Albano, J. E. (1980). Visual-motor function of the primate superior colliculus. Annu. Rev. Neurosci. 3, 189-226.
224. Yin, H. H., and Knowlton, B. J. (2006). The role of the basal ganglia in habit formation. Nat. Rev. Neurosci. 7, 464-476.
225. Yuste, R. (2011). Dendritic spines and distributed circuits. Neuron 71, 772-781.
226. Zheng, T., and Wilson, C. J. (2002). Corticostriatal combinatorics: the implications of corticostriatal axonal arborizations. J. Neurophysiol. 87, 1007-1017.