Different neural correlates of reward expectation and reward expectation error in the putamen and caudate nucleus during stimulus-action-reward association learning

Journal of Neurophysiology

Volumn 95, Issue 2, 2006, Pages 948-959

  Haruno, Masahiko ^a,b   Kawato, Mitsuo ^a  

^a ADVANCED TELECOMMUNICATIONS RESEARCH INSTITUTE INTERNATIONAL   (Japan)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; ARTICLE; ASSOCIATION CORTEX; BEHAVIOR; BRAIN CORTEX; CAUDATE NUCLEUS; CONTROLLED STUDY; CORPUS STRIATUM; EXPECTATION; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; LEARNING; MALE; NORMAL HUMAN; PRIORITY JOURNAL; PUTAMEN; REINFORCEMENT; REWARD; SENSORY STIMULATION; STIMULUS RESPONSE; TASK PERFORMANCE; BIOLOGICAL MODEL; BRAIN MAPPING; CLINICAL TRIAL; COMPUTER SIMULATION; CONTROLLED CLINICAL TRIAL; DECISION MAKING; EVOKED RESPONSE; PHYSIOLOGY; PSYCHOMOTOR PERFORMANCE; RANDOMIZED CONTROLLED TRIAL; STATISTICAL MODEL; STATISTICS;

ADULT; BRAIN MAPPING; CAUDATE NUCLEUS; COMPUTER SIMULATION; DECISION MAKING; EVOKED POTENTIALS; FEMALE; HUMANS; MALE; MODELS, NEUROLOGICAL; MODELS, STATISTICAL; PSYCHOMOTOR PERFORMANCE; PUTAMEN; REWARD; STATISTICS; STOCHASTIC PROCESSES;

EID: 33644858743     PISSN: 00223077     EISSN: None     Source Type: Journal    
DOI: 10.1152/jn.00382.2005     Document Type: Article

References (50)

1. Alexander GE, Crutcher MD, and Delong MR. Basal ganglia thalamocortical circuits: parallel substrates for motor, oculomotor, "prefrontal" and "limbic" functions. Prog Brain Res 85: 119-146, 1990.
2. Barraclough DJ, Conroy ML, and Lee D. Prefrontal cortex and decision making in a mixed-strategy game. Nat Neurosci 7: 404-410, 2004.
3. Barto AG, Sutton RS, and Anderson CW. Neuron-like elements that can solve difficult learning control problems. IEEE Trans Syst Man Cybern 13: 835-846, 1983.
4. Berns GS, McClure MS, Pagnoni G, and Montague PR. Predictability modulates human brain response to reward. J Neurosci 21: 2793-2798, 2001.
5. Bertsekas DP and Tsitsiklis JN. Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, 1996.
6. Breiter HC, Aharon I, Kahneman D, Dale A, and Shizgal P. Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 30: 619-639, 2001.
7. Brown J, Bullock D, and Grossberg S. How the basal ganglia use parallel excitatory and inhibitory learning pathways to selectively respond to unexpected rewarding cues. J Neurosci 19: 10502-10511, 1999.
8. Cromwell HC and Schultz W. Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. J Neurophysiol 89: 2823-2838, 2003.
9. Daw ND, Kakade S, and Dayan P. Opponent interactions between serotonin and dopamine. Neural Netw 15: 603-616, 2002.
10. Dayan P, Kakade S, and Montague RP. Learning and selective attention. Nat Neurosci 3: 1218-1223, 2000.
11. Delgado MR, Nystrom LE, Fissell C, Noll DC, and Fiez JA. Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol 84: 3072-3077, 2000.
12. Friston KJ, Holmes AP, Worsley K, Poline JB, Frith C, and Frackowiak RSJ. Statistical parametric maps in functional brain imaging: a general linear approach. Hum Brain Map 2: 189-210, 1995.
13. Gerardin E, Lehericy S, Pochon JB, Tezenas du Montcel S, Mangin JF, Poupon F, Agid Y, Le Bihan D, and Marsault C. Foot, hand, face and eye representation in the human striatum. Cereb Cortex 13: 162-169, 2003.
14. Haruno M, Kuroda T, Doya K, Toyama K, Kimura M, Samejima K, Imamizu H, and Kawato M. A neural correlate of reward-based behavioral learning in caudate nucleus: a functional magnetic resonance imaging study of a stochastic decision task. J Neurosci 24: 1660-1665, 2004.
15. Hikosaka O, Nakahara H, Rand MK, Sakai K, Lu X, Nakamura K, Miyachi S, and Doya K. Parallel neural networks for learning sequential procedures. Trends Neurosci 22: 464-471, 1999.
16. Hikosaka O, Nakamura K, Sakai K, and Nakahara H. Central mechanisms of motor skill learning. Curr Opin Neurobiol 12: 217-222, 2002.
17. Hollerman JR and Schultz W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 4: 304-309, 1998.
18. Houk JC, Adams JL, and Barto AG. A model of how the basal ganglia generate and use neural signals that predict reinforcement. In: Models of Information Processing in the Basal Ganglia, edited by Houk JC, Davis JL, and Beiser DG. Cambridge, MA: MIT Press, 1995, p. 249-270.
19. Jeuptner M, Frith CD, Brooks DJ, Frackowiak RSJ, and Passingham RE. Anatomy of motor learning. II. Subcortical structures and learning by trial and error. J Neurophysiol 77: 1325-1337, 1997.
20. Jeuptner M and Weiller C. A review of differences between basal ganglia and cerebellar control of movements as revealed by functional imaging studies. Brain 121: 1437-1449, 1998.
21. Kawagoe R, Takikawa Y, and Hikosaka O. Expectation of reward modulates cognitive signals in the basal ganglia. Nat Neurosci 1: 411-416, 2001.
22. Knutson B, Adams MC, Fong WG, and Hommer D. Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J Neurosci 159: 1-5, 2001.
23. Logothetis NK, Pauls J, Augath M, Trinath T, and Oeltermann A. Neurophysiological investigation of the basis of the fMRI signal. Nature 412: 128-130, 2001.
24. Matsumoto N, Hanakawa T, Maki S, Graybiel AM, and Kimura M. Role of nigrostriatal dopamine system in learning to perform sequential motor tasks in a predictive manner. J Neurophysiol 82: 978-998, 1999.
25. Matsumoto K, Suzuki W, and Tanaka K. Neuronal correlates of goal-based motor selection in the prefrontal cortex. Science 30: 229-232, 2003.
26. McClure SM, Berns GS, and Montague PR. Temporal prediction errors in a passive learning task activate human striatum. Neuron 38: 339-346, 2003.
27. Miyachi S, Hikosaka O, Miyashita K, Karadi Z, and Rand MK. Differential roles of monkey striatum in learning of sequential hand movement. Exp Brain Res 115: 1-5, 1997.
28. Miyachi S, Hikosaka O, and Lu X. Differential activation of monkey striatal neurons in the early and late stages of procedural learning. Exp Brain Res 146: 122-126, 2002.
29. Montague PR, Dayan P, and Sejnowski T. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16: 1936-1947, 1996.
30. O'Doherty J, Dayan P, Friston K, Critchley H, and Dolan RJ. Temporal difference models and reward-related learning in the human brain. Neuron 38: 329-337, 2003.
31. O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, and Dolan RJ. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304: 452-454, 2004.
32. Pagnoni G, Zink CF, Montague PR, and Berns GS. Activity in human ventral striatum locked to errors of reward prediction. Nat Neurosci 5: 97-98, 2002.
33. Parthsarathy HB, Schall JD, and Graybiel AM. Distributed but convergent ordering of corticostriatal projections: analysis of the frontal eye field and the supplementary eye field in the macaque monkey. J Neurosci 12: 4468-4488, 1992.
34. Schultz W, Apicella P, Scarnati E, and Ljungberg T. Neuronal activity in monkey ventral striatum related to the expectation of reward. J Neurosci 12: 4595-4610, 1992.
35. Schultz W and Dickinson A. Neuronal coding of prediction errors. Annu Rev Neurosci 23: 473-500, 2000.
36. Schultz W, Tremblay L, and Hollerman JR. Changes in behavior-related neuronal activity in the striatum during learning. Trends Neurosci 26: 321-328, 2003.
37. Selemon LD and Goldman-Rakic PS. Longitudinal topography and interdigitation of corticostriatal projections in the rhesus monkey. J Neurosci 5: 776-794, 1985.
38. Shidara M, Aigner TG, and Richmond BI. Neuronal signals in the monkey ventral striatum related to progress through a predictable series of trials. J Neurosci 18: 2613-2625, 1998.
39. Smith Y, Raju DV, Pare JF, and Sidibe M. The thalamostriatal system: a highly specific network of the basal ganglia circuitry. Trends Neurosci 27: 520-527, 2004.
40. Sugrue LP, Corrado GS, and Newsome WT. Matching behavior and the representation of value in the parietal cortex. Science 304: 1782-1787, 2004.
41. Sutton RS and Barto AG. Reinforcement Learning. Cambridge, MA: MIT Press, 1998.
42. Takada M, Tokuno H, Nambu A, and Inase M. Corticostriatal projections from the somatic motor areas of the frontal cortex in the macaque monkey: segregation versus overlap of input zones from the primary motor cortex, the supplementary motor area, and the premotor cortex. Exp Brain Res 120: 114-128, 1998.
43. Takikawa Y, Kawagoe R, and Hikosaka O. A possible role of midbrain dopamine neurons in short- and long-term adaptation of saccades to position-reward mapping. J Neurophysiol 92: 2520-2529, 2004.
44. Talairach J and Tournoux P. Co-Planar Stereotaxic Atlas of the Human Brain. New York: Thieme, 1998.
45. Tremblay I, Hollerman JR, and Schultz W. Modifications of reward expectation-related neuronal activity during learning in primate striatum. J Neurophysiol 80: 964-977, 1998.
46. Tricomi EM, Delgado MR, and Fiez JA. Modulation of caudate activity by action contingency. Neuron 41: 281-292, 2004.
47. Williams ZM, Bush G, Rauch SL, Cosgrove GR, and Eskandar EN. Human anterior cingulate neurons and the integration of monetary reward with motor responses. Nat Neurosci 12: 1376-1380, 2004.
48. Yamada H, Matsumoto N, and Kimura M. Tonically active neurons in the primate caudate nucleus and putamen differently encode instructed motivational outcomes of action. J Neurosci 24: 3500-3510, 2004.
49. Young P. Recursive Estimation and Time Series. New York: Springer-Verlag, 1984.
50. Zald DH, Boileau I, El-Deared W, Gunn R, McGlone F, Dichter GS, and Dagher A. Dopamine transmission in the human striatum during monetary reward tasks. J Neurosci 24: 4105-4112, 2004.