Neural mechanisms of reward-related motor learning

Current Opinion in Neurobiology

Volumn 13, Issue 6, 2003, Pages 685-690

  Wickens, Jeffery R ^a   Reynolds, John N J ^a   Hyland, Brian I ^a  

^a UNIVERSITY OF OTAGO   (New Zealand)

Author keywords

[No Author keywords available]

Indexed keywords

DOPAMINE;

CELL ACTIVITY; CORPUS STRIATUM; HUMAN; LEARNING; LONG TERM POTENTIATION; MOTOR CONTROL; NERVE CELL; NERVE CELL PLASTICITY; NEUROTRANSMISSION; NONHUMAN; PRIORITY JOURNAL; REGULATORY MECHANISM; REVIEW; REWARD;

EID: 0345118165     PISSN: 09594388     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.conb.2003.10.013     Document Type: Review

References (57)

1. Thorndike E.L. A theory of the action of the after-effects of a connection upon it. Psychol Rev. 40:1933;434-439.
2. Mishkin M, Malamut B, Bachevalier J: Memories and Habits: Two Neural Systems. Edited by Lynch G, McGaugh JL, Weinberger NM. New York: The Guilford Press; 1984.
3. Packard M.G., Knowlton B.J. Learning and memory functions of the basal ganglia. Annu Rev Neurosci. 25:2002;563-593 The authors present an excellent review of neurobehavioral studies examining the role of the striatum in learning and memory, and support the hypothesis that this brain region mediates stimulus-response associations - or habits - acquired incrementally on the basis of reinforcement.
4. McDonald R.J., White N.M. A triple dissociation of memory system: hippocampus, amygdala, and dorsal striatum. Behav Neurosci. 107:1993;3-22.
5. Sage J.R., Knowlton B.J. Effects of US devaluation on win-stay and win-shift radial maze performance in rats. Behav Neurosci. 114:2000;295-306.
6. Swainson R., Rogers R.D., Sahakian B.J., Summers B.A., Polkey C.E., Robbins T.W. Probabilistic learning and reversal deficits in patients with Parkinson's disease or frontal or temporal lobe lesions: possible adverse effects of dopaminergic medication. Neuropsychologia. 38:2000;596-612.
7. Charbonneau D., Riopelle R.J., Beninger R.J. Impaired incentive learning in treated Parkinson's disease. Can J Neurol Sci. 23:1996;271-278.
8. Kunig G., Leenders K.L., Martin-Solch C., Missimer J., Magyar S., Schultz W. Reduced reward processing in the brains of Parkinsonian patients. Neuroreport. 11:2000;3681-3687.
9. Corbett D., Wise R.A. Intracranial self-stimulation in relation to the ascending dopaminergic systems of the midbrain: a moveable electrode mapping study. Brain Res. 185:1980;1-15.
10. Beninger R.J., Freedman N.L. The use of two operants to examine the nature of pimozide-induced decreases in responding for brain stimulation. Physiol Psychol. 10:1982;409-412.
11. Garris P.A., Kilpatrick M., Bunin M.A., Michael D., Walker Q.D., Wightman R.M. Dissociation of dopamine release in the nucleus accumbens from intracranial self-stimulation. Nature. 398:1999;67-69.
12. Kilpatrick M.R., Rooney M.B., Michael D.J., Wightman R.M. Extracellular dopamine dynamics in rat caudate-putamen during experimenter-delivered and intracranial self-stimulation. Neuroscience. 96:2000;697-706.
13. Schultz W. Predictive reward signal of dopamine neurons. J Neurophysiol. 80:1998;1-27.
14. Phillips P.E., Stuber G.D., Heien M.L., Wightman R.M., Carelli R.M. Subsecond dopamine release promotes cocaine seeking. Nature. 422:2003;614-618 This study measured dopamine concentration changes related to behavioral cues associated with cocaine reward. Previously neutral cues elicited rapid changes in dopamine concentration, but only if they had previously been paired to cocaine delivery.
15. DeLong M.R., Crutcher M.D., Georgopoulos A.P. Relations between movement and single cell discharge in the substantia nigra of the behaving monkey. J Neurosci. 3:1983;1599-1606.
16. Schultz W., Ruffieux A., Aebischer P. The activity of pars compacta neurons of the monkey substantia nigra in relation to motor activation. Exp Brain Res. 51:1983;377-387.
17. Schultz W. Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. J Neurophysiol. 56:1986;1439-1461.
18. Romo R., Schultz W. Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. J Neurophysiol. 63:1990;592-606.
19. Ljungberg T., Apicella P., Schultz W. Responses of monkey midbrain dopamine neurons during delayed alternation performance. Brain Res. 567:1991;337-341.
20. Hyland B.I., Reynolds J.N., Hay J., Perk C.G., Miller R. Firing modes of midbrain dopamine cells in the freely moving rat. Neuroscience. 114: 2002;475-492 This study provides a quantitative description of dopamine cell firing properties in the awake rat, and the effect of reward-related stimuli on burst firing. It shows that basic firing modes described in reduced or anaesthetized preparations occur naturally, although the detailed properties are influenced by behavioral stimuli.
21. Kiyatkin E.A., Rebec G.V. Impulse activity of ventral tegmental area neurons during heroin self-administration in rats. Neuroscience. 102:2001;565-580.
22. Freeman A.S., Bunney B.S. Activity of A9 and A10 dopaminergic neurons in unrestrained rats: further characterization and effects of apomorphine and cholecystokinin. Brain Res. 405:1987;46-55.
23. Steinfels G.F., Heym J., Strecker R.E., Jacobs B.L. Behavioral correlates of dopaminergic unit activity in freely moving cats. Brain Res. 258:1983;217-228.
24. Elliott R., Newman J.L., Longe O.A., Deakin J.F. Differential response patterns in the striatum and orbitofrontal cortex to financial reward in humans: a parametric functional magnetic resonance imaging study. J Neurosci. 23:2003;303-307.
25. Delgado M.R., Nystrom L.E., Fissell C., Noll D.C., Fiez J.A. Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol. 84:2000;3072-3077.
26. Knutson B., Fong G.W., Adams C.M., Varner J.L., Hommer D. Dissociation of reward anticipation and outcome with event-related fMRI. Neuroreport. 12:2001;3683-3687.
27. Schultz W., Tremblay L., Hollerman J.R. Changes in behavior-related neuronal activity in the striatum during learning. Trends Neurosci. 26:2003;321-328.
28. Miyachi S., Hikosaka O., Lu X. Differential activation of monkey striatal neurons in the early and late stages of procedural learning. Exp Brain Res. 146:2002;122-126.
29. Kimura M. Behaviorally contingent property of movement-related activity of the primate putamen. J Neurophysiol. 63:1990;1277-1296.
30. Tremblay L., Hollerman J.R., Schultz W. Modifications of reward expectation-related neuronal activity during learning in primate striatum. J Neurophysiol. 80:1998;964-977.
31. Jog M.S., Kubota Y., Connolly C.I., Hillegaart V., Graybiel A.M. Building neural representations of habits. Science. 286:1999;1745-1749.
32. Kawagoe R., Takikawa Y., Hikosaka O. Expectation of reward modulates cognitive signals in the basal ganglia. Nat Neurosci. 1:1998;411-416.
33. Lauwereyns J., Watanabe K., Coe B., Hikosaka O. A neural correlate of response bias in monkey caudate nucleus. Nature. 418:2002;413-417 The authors analyzed single unit activity in the caudate nucleus during visually guided eye movements. Rewards were given for a correct saccade to only one of two possible positions. Recordings indicated that there was an anticipatory increase in firing related to the predictably rewarded direction, which was associated with decreased latency for eye movements in that direction. This may indicate that anticipatory activity creates a bias that favors a spatial response in the direction associated with reward.
34. Lauwereyns J., Takikawa Y., Kawagoe R., Kobayashi S., Koizumi M., Coe B., Sakagami M., Hikosaka O. Feature-based anticipation of cues that predict reward in monkey caudate nucleus. Neuron. 33:2002;463-473.
35. Aosaki T., Tsubokawa H., Ishida A., Watanabe K., Graybiel A.M., Kimura M. Responses of tonically active neurons in the primate's striatum undergo systematic changes during behavioral sensorimotor conditioning. J Neurosci. 14:1994;3969-3984.
36. Kimura M., Rajowski J., Evarts E. Tonically discharging putamen neurons exhibit set dependent responses. Proc Natl Acad Sci USA. 81:1984;4998-5001.
37. Aosaki T., Graybiel A.M., Kimura M. Effect of the nigrostriatal dopamine system on acquired neural responses in the striatum of behaving monkeys. Science. 265:1994;412-415.
38. Schultz W., Apicella P., Ljungberg T. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci. 13:1993;900-913.
39. Brembs B., Lorenzetti F.D., Reyes F.D., Baxter D.A., Byrne J.H. Operant reward learning in Aplysia: neuronal correlates and mechanisms. Science. 296:2002;1706-1709 The authors of this study measured biophysical changes associated with operant conditioning in an identified neuron in Aplysia. Similar changes were induced by pulsatile dopamine application in a single-cell analog.
40. Wilson C.J., Chang H.T., Kitai S.T. Disfacilitation and long-lasting inhibition of neostriatal neurons in the rat. Exp Brain Res. 51:1983;227-235.
41. Wickens JR, Arbuthnott G: Structural and functional interactions in the striatum at the receptor level. In Handbook of Chemical Neuroanatomy, vol 20. Edited by Dunnett, Bentivoglio, Björklund, Hökfelt. Elsevier Science: Netherlands; In press.
42. Reynolds J.N., Wickens J.R. Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw. 15:2002;507-521.
43. Bonsi P., Pisani A., Bernardi G., Calabresi P. Stimulus frequency, calcium levels and striatal synaptic plasticity. Neuroreport. 14:2003;419-422.
44. Kerr J.N.D., Plenz D. Dendritic calcium encodes striatal neuron output during up-states. J Neurosci. 22:2002;1499-1512 The authors measured changes in intracellular calcium concentration in spiny projection neurons in the striatum. Dendritic calcium ion transients occurred in distal dendrites because of the active propagation of somatic action potentials along dendrites.
45. Calabresi P., Pisani A., Mercuri N.B., Bernardi G. Long-term potentiation in the striatum is unmasked by removing the voltage-dependent magnesium block of NMDA receptor channels. Eur J Neurosci. 4:1992;929-935.
46. Calabresi P., Pisani A., Mercuri N.B., Bernardi G. The corticostriatal projection: from synaptic plasticity to dysfunctions of the basal ganglia. Trends Neurosci. 19:1996;19-24.
47. Kerr J.N.D., Wickens J.R. Dopamine D-1/D-5 receptor activation is required for long-term potentiation in the rat neostriatum in vitro. J Neurophysiol. 85:2001;117-124.
48. Wickens J.R., Begg A.J., Arbuthnott G.W. Dopamine reverses the depression of rat cortico-striatal synapses which normally follows high frequency stimulation of cortex in vitro. Neuroscience. 70:1996;1-5.
49. Akopian G., Musleh W., Smith R., Walsh J.P. Functional state of corticostriatal synapses determines their expression of short- and long-term plasticity. Synapse. 38:2000;271-280.
50. Partridge J.G., Tang K.C., Lovinger D.M. Regional and postnatal heterogeneity of activity-dependent long-term changes in synaptic efficacy in the dorsal striatum. J Neurophysiol. 84:2000;1422-1429.
51. Smith R., Musleh W., Akopian G., Buckwalter G., Walsh J.P. Regional differences in the expression of corticostriatal synaptic plasticity. Neuroscience. 106:2001;95-101.
52. Reynolds J.N.J., Wickens J.R. Substantia nigra dopamine regulates synaptic plasticity and membrane potential fluctuations in the rat neostriatum, in vivo. Neuroscience. 99:2000;199-203.
53. Floresco S.B., Blaha C.D., Yang C.R., Phillips A.G. Modulation of hippocampal and amygdalar-evoked activity of nucleus accumbens neurons by dopamine: cellular mechanisms of input selection. J Neurosci. 21:2001;2851-2860 The authors combined extracellular single unit recordings of nucleus accumbens neurons with chronoamperometric measures of dopamine efflux. Potentiation of the hippocampal-evoked response was induced in accumbens cells by HFS of the fimbria. This dopamine D-1 dependent potentiation was associated with a transient increase in dopamine concentration in the accumbens. The subsequent release of dopamine after induction of this facilitation did not appear to play a role.
54. Reynolds J.N.J., Hyland B.I., Wickens J.R. A cellular mechanism of reward-related learning. Nature. 413:2001;67-70 This study showed that stimulation of the substantia nigra pars compacta, using stimuli that supported ICSS, led to potentiation of corticostriatal responses. The potentiation was mediated by dopamine D-1 receptors. Analysis of learning and synaptic plasticity in the same animals showed a significant relationship between the degree of potentiation and the time taken to learn ICSS behavior.
55. Suzuki T., Miura M., Nishimura K., Aosaki T. Dopamine-dependent synaptic plasticity in the striatal cholinergic interneurons. J Neurosci. 21:2001;6492-6501 This study demonstrates dopamine D1-receptor dependent synaptic plasticity in both monosynaptic excitatory and disynaptic inhibitory inputs to cholinergic interneurons. The excitatory inputs showed potentiation after high-frequency stimulation of corticostriatal and thalamostriatal afferents. The inhibitory inputs, thought to be from axon collaterals of spiny projection neurons, showed a decreased apparent failure rate. These changes may be relevant to the learning-related changes in reported in cholinergic interneurons.
56. Tunstall M.J., Oorschot D.E., Kean A., Wickens J.R. Inhibitory interactions between spiny projection neurons in the rat striatum. J Neurophysiol. 88:2002;1263-1269.
57. Wickens JR: Dopamine regulation of synaptic plasticity in the neostriatum: a cellular model of reinforcement. In Brain Dynamics and the Striatal Complex. Edited by Miller R, Wickens JR: Harwood Academic; 2000:65-76.