The role of efference copy in striatal learning

Current Opinion in Neurobiology

Volumn 25, Issue , 2014, Pages 194-200

  Fee, Michale S ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACTION GENERATING CIRCUIT; BASAL GANGLION; BIOLOGICAL MODEL; CONNECTOME; CORPUS STRIATUM; EFFERENCE COPY SIGNAL; EYE MOVEMENT CONTROL; FEEDBACK SYSTEM; HUMAN; MEDIUM SPINY NEURON; NERVE CELL; NERVE CELL PLASTICITY; NERVOUS SYSTEM PARAMETERS; NEURAL MODEL; NONHUMAN; PRIORITY JOURNAL; REINFORCEMENT; REVIEW; REWARD; SACCADIC EYE MOVEMENT; SIGNAL TRANSDUCTION; THALAMUS; ANIMAL; EFFERENT NERVE; NEOSTRIATUM; PHYSIOLOGY;

ANIMALS; EFFERENT PATHWAYS; HUMANS; NEOSTRIATUM; NEURONS; REINFORCEMENT (PSYCHOLOGY);

EID: 84896724329     PISSN: 09594388     EISSN: 18736882     Source Type: Journal    
DOI: 10.1016/j.conb.2014.01.012     Document Type: Review

References (63)

1. Thorndike E.L. Animal intelligence 1911, Hafner, Darien, CT.
2. Packard M.G., Knowlton B.J. Learning and memory functions of the basal ganglia. Annu Rev Neurosci 2002, 25:563-593.
3. Graybiel A.M. Habits, rituals, and the evaluative brain. Annu Rev Neurosci 2008, 31:359-387.
4. Sutton R.S., Barto A.G. Reinforcement learning: an introduction 1998, The MIT Press, Cambridge, MA.
5. Graybiel A.M., Aosaki T., Flaherty A.W., Kimura M. The basal ganglia and adaptive motor control. Science 1994, 265:1826-1831.
6. Graybiel A.M. The basal ganglia and chunking of action repertoires. Neurobiol Learn Mem 1998, 70:119-136.
7. Houk J.C. Information processing in modular circuits linking basal ganglia and cerebral cortex. Models of information processing in the basal ganglia 1995, 3-10. The MIT Press, Cambridge, MA. J.C. Houk, J.L. Davis, D.G. Beiser (Eds.).
8. Doya K. Complementary roles of basal ganglia and cerebellum in learning and motor control. Curr Opin Neurobiol 2000, 10:732-739.
9. Daw N.D., Doya K. The computational neurobiology of learning and reward. Curr Opin Neurobiol 2006, 16:199-204.
10. Frank M.J. Computational models of motivated action selection in corticostriatal circuits. Curr Opin Neurobiol 2011, 21:381-386.
11. Kawagoe R., Takikawa Y., Hikosaka O. Reward-predicting activity of dopamine and caudate neurons - a possible mechanism of motivational control of saccadic eye movement. J Neurophysiol 2004, 91:1013-1024.
12. Pasupathy A., Miller E.K. Different time courses of learning-related activity in the prefrontal cortex and striatum. Nature 2005, 433:873-876.
13. Hikosaka O. Basal ganglia mechanisms of reward-oriented eye movement. Ann N Y Acad Sci 2007, 1104:229-249.
14. Hikosaka O., Nakamura K., Nakahara H. Basal ganglia orient eyes to reward. J. Neurophysiol 2006, 95:567-584.
15. Grillner S., Hellgren J., Ménard A., Saitoh K., Wikström M.A. Mechanisms for selection of basic motor programs - roles for the striatum and pallidum. Trends Neurosci 2005, 28:364-370.
16. Alexander G.E., DeLong M.R., Strick P.L. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu Rev Neurosci 1986, 9:357-381.
17. Hoover J.E., Strick P.L. Multiple output channels in the basal ganglia. Science 1993, 259:819-821.
18. Berns G.S., Sejnowski T.J. A computational model of how the basal ganglia produce sequences. J Cogn Neurosci 1998, 10:108-121.
19. Hikosaka O., Nakahara H., Rand M.K., Sakai K., Lu X., Nakamura K., Miyachi S., Doya K. Parallel neural networks for learning sequential procedures. Trends Neurosci 1999, 22:464-471.
20. Ito M., Doya K. Multiple representations and algorithms for reinforcement learning in the cortico-basal ganglia circuit. Curr Opin Neurobiol 2011, 21:368-373.
21. Mirenowicz J., Schultz W. Importance of unpredictability for reward responses in primate dopamine neurons. J Neurophysiol 1994, 72:1024-1027.
22. Schultz W., Dayan P., Montague P.R. A neural substrate of prediction and reward. Science 1997, 275:1593-1599.
23. Schultz W. Predictive reward signal of dopamine neurons. J Neurophysiol 1998, 80:1.
24. Hollerman J.R., Schultz W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 1998, 1:304-309.
25. Farries M.A., Fairhall A.L. Reinforcement learning with modulated spike timing dependent synaptic plasticity. J Neurophysiol 2007, 98:3648-3665.
26. Florian R.V. Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity. Neural Comput 2007, 19:1468-1502.
27. Izhikevich E.M. Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb Cortex 2007, 17:2443-2452.
28. Roberts P.D., Santiago R.A., Lafferriere G. An implementation of reinforcement learning based on spike timing dependent plasticity. Biol Cybern 2008, 99:517-523.
29. Reynolds J.N.J., Hyland B.I., Wickens J.R. A cellular mechanism of reward-related learning. Nature 2001, 413:67-70.
30. Pawlak V., Kerr J.N. Dopamine receptor activation is required for corticostriatal spike-timing-dependent plasticity. J Neurosci 2008, 28:2435-2446.
31. Shen W., Flajolet M., Greengard P., Surmeier D.J. Dichotomous dopaminergic control of striatal synaptic plasticity. Science 2008, 321:848-851.
32. Fee M.S. Oculomotor learning revisited: a model of reinforcement learning in the basal ganglia incorporating an efference copy of motor actions. Front Neural Circuits 2012, 6:38.
33. Wurtz R.H., Albano J.E. Visual-motor function of the primate superior colliculus. Annu Rev Neurosci 1980, 3:189-226.
34. Swanson L.W. Cerebral hemisphere regulation of motivated behavior. Brain Res 2000, 886:113-164.
35. Goldberg J.H., Fee M.S. Vocal babbling in songbirds requires the basal ganglia-recipient motor thalamus but not the basal ganglia. J Neurophysiol 2011, 105:2729-2739.
36. Ölveczky B.P., Andalman A.S., Fee M.S. Vocal experimentation in the juvenile songbird requires a basal ganglia circuit. PLoS Biol 2005, 3:e153.
37. Kao M.H., Doupe A.J., Brainard M.S. Contributions of an avian basal ganglia-forebrain circuit to real-time modulation of song. Nature 2005, 433:638-643.
38. Aronov D., Andalman A.S., Fee M.S. A specialized forebrain circuit for vocal babbling in the juvenile songbird. Science 2008, 320:630-634.
39. Aronov D., Veit L., Goldberg J.H., Fee M.S. Two distinct modes of forebrain circuit dynamics underlie temporal patterning in the vocalizations of young songbirds. J Neurosci 2011, 31:16353-16368.
40. Choi G.B., Dong H.W., Murphy A.J., Valenzuela D.M., Yancopoulos G.D., Swanson L.W., Anderson D.J. Lhx6 delineates a pathway mediating innate reproductive behaviors from the amygdala to the hypothalamus. Neuron 2005, 46:647-660.
41. Mysore S.P., Knudsen E.I. A shared inhibitory circuit for both exogenous and endogenous control of stimulus selection. Nat Neurosci 2013, 16:473-478.
42. Mink J.W. The basal ganglia: focused selection and inhibition of competing motor programs. Prog Neurobiol 1996, 50:381-425.
43. Redgrave P., Prescott T.J., Gurney K. The basal ganglia: a vertebrate solution to the selection problem?. Neuroscience 1999, 89:1009-1023.
44. Frank M.J. Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. J Cogn Neurosci 2005, 17:51-72.
45. Samson R.D., Frank M.J., Fellous J.M. Computational models of reinforcement learning: the role of dopamine as a reward signal. Cogn Neurodyn 2010, 4:91-105.
46. Redgrave P., Gurney K. The short-latency dopamine signal: a role in discovering novel actions?. Nat Rev Neurosci 2006, 7:967-975.
47. Fee M.S., Goldberg J.H. A hypothesis for basal ganglia-dependent reinforcement learning in the songbird. Neuroscience 2011, 198:152-170.
48. Horvitz J.C. Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience 2000, 96:651-656.
49. Bromberg-Martin E.S., Matsumoto M., Hikosaka O. Dopamine in motivational control: rewarding, aversive, and alerting. Neuron 2010, 68:815-834.
50. Doya K., Sejnowski T. A novel reinforcement model of birdsong vocalization learning. Adv Neural Inform Process Syst 1995, 7:101-108.
51. Vates G.E., Nottebohm F. Feedback circuitry within a song-learning pathway. Proc Natl Acad Sci U S A 1995, 92:5139-5143.
52. Gale S.D., Perkel D.J. A basal ganglia pathway drives selective auditory responses in songbird dopaminergic neurons via disinhibition. J Neurosci 2010, 30:1027-1037.
53. Kozhevnikov A.A., Fee M.S. Singing-related activity of identified HVC neurons in the zebra finch. J Neurophysiol 2007, 97:4271-4283.
54. Prather J.F., Peters S., Nowicki S., Mooney R. Precise auditory-vocal mirroring in neurons for learned vocal communication. Nature 2008, 451:305-310.
55. Frey U., Morris R.G.M. Synaptic tagging and long-term potentiation. Nature 1997, 385:533-536.
56. Fiete I.R., Fee M.S., Seung H.S. Model of birdsong learning based on gradient estimation by dynamic perturbation of neural conductances. J Neurophysiol 2007, 98:2038-2057.
57. Andalman A.S., Fee M.S. A basal ganglia-forebrain circuit in the songbird biases motor output to avoid vocal errors. Proc Natl Acad Sci U S A 2009, 106:12518-12523.
58. Warren T.L., Tumer E.C., Charlesworth J.D., Brainard M.S. Mechanisms and time course of vocal learning and consolidation in the adult songbird. J Neurophysiol 2011, 106:1806-1821.
59. Sommer M.A., Wurtz R.H. Brain circuits for the internal monitoring of movements. Annu Rev Neurosci 2008, 31:317-338.
60. Bruce C.J., Goldberg M.E. Primate frontal eye fields. I. Single neurons discharging before saccades. J Neurophysiol 1985, 53:603-635.
61. Komatsu H., Suzuki H. Projections from the functional subdivisions of the frontal eye field to the superior colliculus in the monkey. Brain Res 1985, 327:324-327.
62. Künzle H., Akert K. Efferent connections of cortical, area 8 (frontal eye field) in Macaca fascicularis. A reinvestigation using the autoradiographic technique. J Comp Neurol 1977, 173:147-163.
63. Watkins C.J., Dayan P. Q-learning. Mach Learn 1992, 8:279-292.