Learning and motivation in the human striatum

Current Opinion in Neurobiology

Volumn 21, Issue 3, 2011, Pages 408-414

  Shohamy, Daphna ^a  

^a Columbia University ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BEHAVIOR; COGNITION; CORPUS STRIATUM; HUMAN; LEARNING; MOTIVATION; NEUROIMAGING; PRIORITY JOURNAL; REVIEW; SIGNAL TRANSDUCTION; STIMULUS RESPONSE;

CORPUS STRIATUM; HUMANS; LEARNING; MOTIVATION;

EID: 79960251013     PISSN: 09594388     EISSN: 18736882     Source Type: Journal    
DOI: 10.1016/j.conb.2011.05.009     Document Type: Review

References (94)

1. Gabrieli J.D. Cognitive neuroscience of human memory. Annu Rev Psychol 1998, 49:87-115.
2. Knowlton B.J., Mangels J.A., Squire L.R. A neostriatal habit learning system in humans. Science 1996, 273:1399-1402.
3. Everitt B.J., Robbins T.W. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat Neurosci 2005, 8:1481-1489.
4. Jog M.S., Kubota Y., Connolly C.I., Hillegaart V., Graybiel A.M. Building neural representations of habits. Science 1999, 286:1745-1749.
5. Albin R.L., Young A.B., Penney J.B. The functional anatomy of basal ganglia disorders. Trends Neurosci 1989, 12:366-375.
6. DeLong M.R. Primate models of movement disorders of basal ganglia origin. Trends Neurosci 1990, 13:281-285.
7. Schultz W. Predictive reward signal of dopamine neurons. J Neurophysiol 1998, 80:1-27.
8. Frank M.J., Doll B.B., Oas-Terpstra J., Moreno F. Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nat Neurosci 2009, 12:1062-1068.
9. Shohamy D., Myers C.E., Grossman S., Sage J., Gluck M.A., Poldrack R.A. Cortico-striatal contributions to feedback-based learning: converging data from neuroimaging and neuropsychology. Brain 2004, 127:851-859.
10. Cools R., Clark L., Owen A.M., Robbins T.W. Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J Neurosci 2002, 22:4563-4567.
11. Daw N.D., O'Doherty J.P., Dayan P., Seymour B., Dolan R.J. Cortical substrates for exploratory decisions in humans. Nature 2006, 441:876-879.
12. Delgado M.R., Miller M.M., Inati S., Phelps E.A. An fMRI study of reward-related probability learning. Neuroimage 2005, 24:862-873.
13. Poldrack R.A., Clark J., Pare-Blagoev E.J., Shohamy D., Creso Moyano J., Myers C., Gluck M.A. Interactive memory systems in the human brain. Nature 2001, 414:546-550.
14. O'Doherty J., Dayan P., Schultz J., Deichmann R., Friston K., Dolan R.J. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 2004, 304:452-454.
15. Yin H.H., Knowlton B.J. The role of the basal ganglia in habit formation. Nat Rev Neurosci 2006, 7:464-476.
16. Tricomi E., Balleine B.W., O'Doherty J.P. A specific role for posterior dorsolateral striatum in human habit learning. Eur J Neurosci 2009, 29:2225-2232.
17. Foerde K., Knowlton B.J., Poldrack R.A. Modulation of competing memory systems by distraction. Proc Natl Acad Sci USA 2006, 103:11778-11783.
18. Rangel A., Hare T. Neural computations associated with goal-directed choice. Curr Opin Neurobiol 2010, 20:262-270.
19. Shohamy D., Wagner A.D. Integrating memories in the human brain: hippocampal-midbrain encoding of overlapping events. Neuron 2008, 60:378-389.
20. Daw N., Gershman S.J., Dayan P., Dolan R.J. Model-based influences on humans' choices and striatal prediction errors. Neuron 2011, 69:1204-1215.
21. Glascher J., Daw N., Dayan P., O'Doherty J.P. States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 2010, 66:585-595.
22. Hare T.A., O'Doherty J., Camerer C.F., Schultz W., Rangel A. Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors. J Neurosci 2008, 28:5623-5630.
23. McClure S.M., Berns G.S., Montague P.R. Temporal prediction errors in a passive learning task activate human striatum. Neuron 2003, 38:339-346.
24. O'Doherty J.P., Dayan P., Friston K., Critchley H., Dolan R.J. Temporal difference models and reward-related learning in the human brain. Neuron 2003, 38:329-337.
25. Daw N.D., O'Doherty J.P., Dayan P., Seymour B., Dolan R.J. Cortical substrates for exploratory decisions in humans. Nature 2006.
26. Delgado M.R., Nystrom L.E., Fissell C., Noll D.C., Fiez J.A. Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol 2000, 84:3072-3077.
27. Daw N.D., Doya K. The computational neurobiology of learning and reward. Curr Opin Neurobiol 2006, 16:199-204.
28. Dickerson K.C., Li J., Delgado M.R. Parallel contributions of distinct human memory systems during probabilistic learning. Neuroimage 2011, 55:266-276.
29. Lin A., Adolphs R., Rangel A. Social and monetary reward learning engage overlapping neural substrates. Soc Cogn Affect Neurosci 2011.
30. Stice E., Yokum S., Blum K., Bohon C. Weight gain is associated with reduced striatal response to palatable food. J Neurosci 2010, 30:13105-13109.
31. Aron A.R., Shohamy D., Clark J., Myers C., Gluck M.A., Poldrack R.A. Human midbrain sensitivity to cognitive feedback and uncertainty during classification learning. J Neurophysiol 2004, 92:1144-1152.
32. Salimpoor V.N., Benovoy M., Larcher K., Dagher A., Zatorre R.J. Anatomically distinct dopamine release during anticipation and experience of peak emotion to music. Nat Neurosci 2011, 14:257-262.
33. Haruno M., Kawato M. Different neural correlates of reward expectation and reward expectation error in the putamen and caudate nucleus during stimulus-action-reward association learning. J Neurophysiol 2006, 95:948-959.
34. Haruno M., Kuroda T., Doya K., Toyama K., Kimura M., Samejima K., Imamizu H., Kawato M. A neural correlate of reward-based behavioral learning in caudate nucleus: a functional magnetic resonance imaging study of a stochastic decision task. J Neurosci 2004, 24:1660-1665.
35. King-Casas B., Tomlin D., Anen C., Camerer C.F., Quartz S.R., Montague P.R. Getting to know you: reputation and trust in a two-person economic exchange. Science 2005, 308:78-83.
36. Tricomi E.M., Delgado M.R., Fiez J.A. Modulation of caudate activity by action contingency. Neuron 2004, 41:281-292.
37. Bromberg-Martin E.S., Matsumoto M., Hikosaka O. Dopamine in motivational control: rewarding, aversive, and alerting. Neuron 2010, 68:815-834.
38. Salas R., Baldwin P., de Biasi M., Montague P.R. BOLD responses to negative reward prediction errors in human habenula. Front Hum Neurosci 2010, 4:36.
39. Robinson O.J., Frank M.J., Sahakian B.J., Cools R. Dissociable responses to punishment in distinct striatal regions during reversal learning. Neuroimage 2010, 51:1459-1467.
40. Seymour B., O'Doherty J.P., Dayan P., Koltzenburg M., Jones A.K., Dolan R.J., Friston K.J., Frackowiak R.S. Temporal difference models describe higher-order learning in humans. Nature 2004, 429:664-667.
41. Delgado M.R., Schotter A., Ozbay E.Y., Phelps E.A. Understanding overbidding: using the neural circuitry of reward to design economic auctions. Science 2008, 321:1849-1852.
42. Balleine B.W., Delgado M.R., Hikosaka O. The role of the dorsal striatum in reward and decision-making. J Neurosci 2007, 27:8161-8165.
43. Phan K.L., Sripada C.S., Angstadt M., McCabe K. Reputation for reciprocity engages the brain reward center. Proc Natl Acad Sci USA 2010, 107:13099-13104.
44. Tricomi E., Rangel A., Camerer C.F., O'Doherty J.P. Neural evidence for inequality-averse social preferences. Nature 2010, 463:1089-1091.
45. Campbell-Meiklejohn D.K., Bach D.R., Roepstorff A., Dolan R.J., Frith C.D. How the opinion of others affects our valuation of objects. Curr Biol 2010, 20:1165-1170.
46. Mobbs D., Yu R., Meyer M., Passamonti L., Seymour B., Calder A.J., Schweizer S., Frith C.D., Dalgleish T. A key role for similarity in vicarious reward. Science 2009, 324:900.
47. Burke C.J., Tobler P.N., Baddeley M., Schultz W. Neural mechanisms of observational learning. Proc Natl Acad Sci USA 2010, 107:14431-14436.
48. Murayama K., Matsumoto M., Izuma K., Matsumoto K. Neural basis of the undermining effect of monetary reward on intrinsic motivation. Proc Natl Acad Sci USA 2010, 107:20911-20916.
49. Mobbs D., Hassabis D., Seymour B., Marchant J.L., Weiskopf N., Dolan R.J., Frith C.D. Choking on the money: reward-based performance decrements are associated with midbrain activity. Psychol Sci 2009, 20:955-962.
50. Adcock R.A., Thangavel A., Whitfield-Gabrieli S., Knutson B., Gabrieli J.D. Reward-motivated learning: mesolimbic activation precedes memory formation. Neuron 2006, 50:507-517.
51. Kang M.J., Hsu M., Krajbich I.M., Loewenstein G., McClure S.M., Wang J.T., Camerer C.F. The wick in the candle of learning: epistemic curiosity activates reward circuitry and enhances memory. Psychol Sci 2009, 20:963-973.
52. Fenker D.B., Schoenfeld M.A., Waldmann M.R., Schuetze H., Heinze H.J., Duezel E. "Virus and epidemic": causal knowledge activates prediction error circuitry. J Cogn Neurosci 2010, 22:2151-2163.
53. Wittmann B.C., Schott B.H., Guderian S., Frey J.U., Heinze H.J., Duzel E. Reward-related FMRI activation of dopaminergic midbrain is associated with enhanced hippocampus-dependent long-term memory formation. Neuron 2005, 45:459-467.
54. Han S., Huettel S.A., Raposo A., Adcock R.A., Dobbins I.G. Functional significance of striatal responses during episodic decisions: recovery or goal attainment?. J Neurosci 2010, 30:4767-4775.
55. Tricomi E., Fiez J.A. Feedback signals in the caudate reflect goal achievement on a declarative memory task. Neuroimage 2008, 41:1154-1167.
56. Turk-Browne N.B., Scholl B.J., Chun M.M., Johnson M.K. Neural evidence of statistical learning: efficient detection of visual regularities without awareness. J Cogn Neurosci 2009, 21:1934-1945.
57. Brainard M.S., Doupe A.J. Interruption of a basal ganglia-forebrain circuit prevents plasticity of learned vocalizations. Nature 2000, 404:762-766.
58. Pessiglione M., Petrovic P., Daunizeau J., Palminteri S., Dolan R.J., Frith C.D. Subliminal instrumental conditioning demonstrated in the human brain. Neuron 2008, 59:561-567.
59. Alexander G.E., DeLong M.R., Strick P.L. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu Rev Neurosci 1986, 9:357-381.
60. Schoenbaum G., Esber G.R. How do you (estimate you will) like them apples? Integration as a defining trait of orbitofrontal function. Curr Opin Neurobiol 2010, 20:205-211.
61. McDannald M.A., Lucantonio F., Burke K.A., Niv Y., Schoenbaum G. Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. J Neurosci 2011, 31:2700-2705.
62. den Ouden H.E., Daunizeau J., Roiser J., Friston K.J., Stephan K.E. Striatal prediction error modulates cortical coupling. J Neurosci 2010, 30:3210-3219.
63. Shohamy D., Adcock R.A. Dopamine and adaptive memory. Trends Cogn Sci 2010, 14:464-472.
64. van der Meer M.A., Johnson A., Schmitzer-Torbert N.C., Redish A.D. Triple dissociation of information processing in dorsal striatum, ventral striatum, and hippocampus on a learned spatial decision task. Neuron 2010, 67:25-32.
65. Johnson A., van der Meer M.A., Redish A.D. Integrating hippocampus and striatum in decision-making. Curr Opin Neurobiol 2007, 17:692-697.
66. Voermans N.C., Petersson K.M., Daudey L., Weber B., Van Spaendonck K.P., Kremer H.P., Fernandez G. Interaction between the human hippocampus and the caudate nucleus during route recognition. Neuron 2004, 43:427-435.
67. Mattfeld A.T., Stark C.E. Striatal and medial temporal lobe functional interactions during visuomotor associative learning. Cereb Cortex 2011, 21:647-658.
68. Hartley T., Burgess N. Complementary memory systems: competition, cooperation and compensation. Trends Neurosci 2005, 28:169-170.
69. Goldman-Rakic P.S., Selemon L.D., Schwartz M.L. Dual pathways connecting the dorsolateral prefrontal cortex with the hippocampal formation and parahippocampal cortex in the rhesus monkey. Neuroscience 1984, 12:719-743.
70. Fuster J.M. The Prefrontal Cortex: Anatomy, Physiology, and Neuropsychology of the Frontal Lobe 1997, Lippincott-Raven, Philadelphia.
71. Haber S.N. The primate basal ganglia: parallel and integrative networks. J Chem Neuroanat 2003, 26:317-330.
72. Suzuki W.A., Amaral D.G. Functional neuroanatomy of the medial temporal lobe memory system. Cortex 2004, 40:220-222.
73. Doeller C.F., King J.A., Burgess N. Parallel striatal and hippocampal systems for landmarks and boundaries in spatial memory. Proc Natl Acad Sci USA 2008, 105:5915-5920.
74. Sadeh T., Shohamy D., Levy D.R., Reggev N., Maril A. Cooperation between the hippocampus and the striatum during episodic encoding. J Cogn Neurosci 2011.
75. Levy I., Lazzaro S.C., Rutledge R.B., Glimcher P.W. Choice from non-choice: predicting consumer preferences from blood oxygenation level-dependent signals obtained during passive viewing. J Neurosci 2011, 31:118-125.
76. Axmacher N., Cohen M.X., Fell J., Haupt S., Dumpelmann M., Elger C.E., Schlaepfer T.E., Lenartz D., Sturm V., Ranganath C. Intracranial EEG correlates of expectancy and memory formation in the human hippocampus and nucleus accumbens. Neuron 2010, 65:541-549.
77. Lisman J.E., Grace A.A. The hippocampal-VTA loop: controlling the entry of information into long-term memory. Neuron 2005, 46:703-713.
78. Daw N.D., Shohamy D. The cognitive neuroscience of motivation and learning. Soc Cogn 2008, 26:593-620.
79. Peters J., Buchel C. Episodic future thinking reduces reward delay discounting through an enhancement of prefrontal-mediotemporal interactions. Neuron 2010, 66:138-148.
80. Shohamy D., Myers C.E., Kalanithi J., Gluck M.A. Basal ganglia and dopamine contributions to probabilistic category learning. Neurosci Biobehav Rev 2008, 32:219-236.
81. Grahn J.A., Parkinson J.A., Owen A.M. The role of the basal ganglia in learning and memory: neuropsychological studies. Behav Brain Res 2009, 199:53-60.
82. Rowe J.B., Hughes L., Ghosh B.C., Eckstein D., Williams-Gray C.H., Fallon S., Barker R.A., Owen A.M. Parkinson's disease and dopaminergic therapy - differential effects on movement, reward and cognition. Brain 2008, 131:2094-2105.
83. Frank M.J., Seeberger L.C., O'Reilly R.C. By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 2004, 306:1940-1943.
84. Cools R., Lewis S.J., Clark L., Barker R.A., Robbins T.W. L-DOPA disrupts activity in the nucleus accumbens during reversal learning in Parkinson's disease. Neuropsychopharmacology 2007, 32:180-189.
85. Schonberg T., O'Doherty J.P., Joel D., Inzelberg R., Segev Y., Daw N.D. Selective impairment of prediction error signaling in human dorsolateral but not ventral striatum in Parkinson's disease patients: evidence from a model-based fMRI study. Neuroimage 2010, 49:772-781.
86. Shohamy D., Myers C.E., Geghman K.D., Sage J., Gluck M.A. L-dopa impairs learning, but spares generalization, in Parkinson's disease. Neuropsychologia 2006, 44:774-784.
87. Lotze M., Reimold M., Heymans U., Laihinen A., Patt M., Halsband U. Reduced ventrolateral fMRI response during observation of emotional gestures related to the degree of dopaminergic impairment in Parkinson disease. J Cogn Neurosci 2009, 21:1321-1331.
88. Schott B.H., Minuzzi L., Krebs R.M., Elmenhorst D., Lang M., Winz O.H., Seidenbecher C.I., Coenen H.H., Heinze H.J., Zilles K., et al. Mesolimbic functional magnetic resonance imaging activations during reward anticipation correlate with reward-related ventral striatal dopamine release. J Neurosci 2008, 28:14311-14319.
89. Keri S., Moustafa A.A., Myers C.E., Benedek G., Gluck M.A. {alpha}-Synuclein gene duplication impairs reward learning. Proc Natl Acad Sci USA 2010, 107:15992-15994.
90. Krugel L.K., Biele G., Mohr P.N., Li S.C., Heekeren H.R. Genetic variation in dopaminergic neuromodulation influences the ability to rapidly and flexibly adapt decisions. Proc Natl Acad Sci USA 2009, 106:17951-17956.
91. Maia T.V., Frank M.J. From reinforcement learning models to psychiatric and neurological disorders. Nat Neurosci 2011, 14:154-162.
92. Van Leijenhorst L., Moor B.G., Op de Macks Z.A., Rombouts S.A., Westenberg P.M., Crone E.A. Adolescent risky decision-making: neurocognitive development of reward and control regions. Neuroimage 2010, 51:345-355.
93. Cohen J.R., Asarnow R.F., Sabb F.W., Bilder R.M., Bookheimer S.Y., Knowlton B.J., Poldrack R.A. A unique adolescent response to reward prediction errors. Nat Neurosci 2010, 13:669-671.
94. DeCouteau W.E., Thorn C., Gibson D.J., Courtemanche R., Mitra P., Kubota Y., Graybiel A.M. Learning-related coordination of striatal and hippocampal theta rhythms during acquisition of a procedural maze task. Proc Natl Acad Sci USA 2007, 104:5644-5649.