A Reinforcement Learning Mechanism Responsible for the Valuation of Free Choice

Neuron

Volumn 83, Issue 3, 2014, Pages 551-557

  Cockburn, Jeffrey ^a   Collins, Anne G E ^a   Frank, Michael J ^a  

^a BROWN UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DOPAMINE;

COGNITION; DECISION MAKING; HUMAN; INDIVIDUALITY; LEARNING; METABOLISM; PHYSIOLOGY; PSYCHOLOGICAL MODEL; PSYCHOMOTOR PERFORMANCE; REINFORCEMENT; REWARD;

CHOICE BEHAVIOR; COGNITION; DOPAMINE; HUMANS; INDIVIDUALITY; LEARNING; MODELS, PSYCHOLOGICAL; PSYCHOMOTOR PERFORMANCE; REINFORCEMENT (PSYCHOLOGY); REWARD;

EID: 84905569399     PISSN: 08966273     EISSN: 10974199     Source Type: Journal    
DOI: 10.1016/j.neuron.2014.06.035     Document Type: Article

References (38)

1. Alexander G.E., Crutcher M.D. Functional architecture of basal ganglia circuits: neural substrates of parallel processing. Trends Neurosci. 1990, 13:266-271.
2. Ashby F.G., Ennis J.M., Spiering B.J. A neurobiological theory of automaticity in perceptual categorization. Psychol. Rev. 2007, 114:632-656.
3. Bown N.J., Read D., Summers B. The lure of choice. J. Behav. Decis. Making 2003, 16:297-308.
4. Brown P., Marsden C.D. What do the basal ganglia do?. Lancet 1998, 351:1801-1804.
5. Collins A.G.E., Frank M.J. Opponent Actor Learning (OpAL): modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive. Psychol. Rev. 2014, 121:337-366.
6. Doll B.B., Hutchison K.E., Frank M.J. Dopaminergic genes predict individual differences in susceptibility to confirmation bias. J. Neurosci. 2011, 31:6188-6198.
7. Egan L.C., Santos L.R., Bloom P. The origins of cognitive dissonance: evidence from children and monkeys. Psychol. Sci. 2007, 18:978-983.
8. Festinger L. A Theory of Cognitive Dissonance 1962, Stanford University Press, Stanford.
9. François-Brosseau F.-E., Martinu K., Strafella A.P., Petrides M., Simard F., Monchi O. Basal ganglia and frontal involvement in self-generated and externally-triggered finger movements in the dominant and non-dominant hand. Eur. J. Neurosci. 2009, 29:1277-1286.
10. Frank M.J. Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. J. Cogn. Neurosci. 2005, 17:51-72.
11. Frank M.J. Hold your horses: a dynamic computational role for the subthalamic nucleus in decision making. Neural Netw. 2006, 19:1120-1136.
12. Frank M.J., Seeberger L.C., O'reilly R.C. By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 2004, 306:1940-1943.
13. Frank M.J., Moustafa A.A., Haughey H.M., Curran T., Hutchison K.E. Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc. Natl. Acad. Sci. USA 2007, 104:16311-16316.
14. Frank M.J., Doll B.B., Oas-Terpstra J., Moreno F. Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nat. Neurosci. 2009, 12:1062-1068.
15. Gershman S.J., Pesaran B., Daw N.D. Human reinforcement learning subdivides structured action spaces by learning effector-specific values. J. Neurosci. 2009, 29:13524-13531.
16. Hirvonen M.M., Laakso A., Någren K., Rinne J.O., Pohjalainen T., Hietala J. C957T polymorphism of dopamine D2 receptor gene affects striatal DRD2 in vivo availability by changing the receptor affinity. Synapse 2009, 63:907-912.
17. Huotari M., Gogos J.A., Karayiorgou M., Koponen O., Forsberg M., Raasmaja A., Hyttinen J., Männistö P.T. Brain catecholamine metabolism in catechol-O-methyltransferase (COMT)-deficient mice. Eur. J. Neurosci. 2002, 15:246-256.
18. Iyengar S.S., Lepper M.R. When choice is demotivating: can one desire too much of a good thing?. J. Pers. Soc. Psychol. 2000, 79:995-1006.
19. Joel D., Weiner I. The connections of the dopaminergic system with the striatum in rats and primates: an analysis with respect to the functional and compartmental organization of the striatum. Neuroscience 2000, 96:451-474.
20. Lee C.R., Abercrombie E.D., Tepper J.M. Pallidal control of substantia nigra dopaminergic neuron firing pattern and its relation to extracellular neostriatal dopamine levels. Neuroscience 2004, 129:481-489.
21. Leotti L.A., Delgado M.R. The inherent reward of choice. Psychol. Sci. 2011, 22:1310-1318.
22. Leotti L.A., Delgado M.R. The value of exercising control over monetary gains and losses. Psychol. Sci. 2014, 25:596-604.
23. Lieberman M.D., Ochsner K.N., Gilbert D.T., Schacter D.L. Do amnesics exhibit cognitive dissonance reduction? The role of explicit memory and attention in attitude change. Psychol. Sci. 2001, 12:135-140.
24. Lobb C.J., Troyer T.W., Wilson C.J., Paladini C.a. Disinhibition bursting of dopaminergic neurons. Front. Syst. Neurosci. 2011, 5:1-8.
25. Matsumoto M., Weickert C.S., Akil M., Lipska B.K., Hyde T.M., Herman M.M., Kleinman J.E., Weinberger D.R. Catechol O-methyltransferase mRNA expression in human and rat brain: evidence for a role in cortical neuronal function. Neuroscience 2003, 116:127-137.
26. Maunsell J.H.R. Neuronal representations of cognitive state: reward or attention?. Trends Cogn. Sci. 2004, 8:261-265.
27. Mink J.W. The basal ganglia: focused selection and inhibition of competing motor programs. Prog. Neurobiol. 1996, 50:381-425.
28. O'Doherty J., Dayan P., Schultz J., Deichmann R., Friston K., Dolan R.J. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 2004, 304:452-454.
29. O'Reilly R.C., Frank M.J. Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Comput. 2006, 18:283-328.
30. Orr H.A. Fitness and its role in evolutionary genetics. Nat. Rev. Genet. 2009, 10:531-539.
31. Schultz W. Getting formal with dopamine and reward. Neuron 2002, 36:241-263.
32. Sharot T., De Martino B., Dolan R.J. How choice reveals and shapes expected hedonic outcome. J. Neurosci. 2009, 29:3760-3765.
33. Sharot T., Velasquez C.M., Dolan R.J. Do decisions shape preference? Evidence from blind choice. Psychol. Sci. 2010, 21:1231-1235.
34. St Onge J.R., Ahn S., Phillips A.G., Floresco S.B. Dynamic fluctuations in dopamine efflux in the prefrontal cortex and nucleus accumbens during risk-based decision making. J. Neurosci. 2012, 32:16880-16891.
35. Stephan K.E., Penny W.D., Daunizeau J., Moran R.J., Friston K.J. Bayesian model selection for group studies. Neuroimage 2009, 46:1004-1017.
36. Stipanovich A., Valjent E., Matamales M., Nishi A., Ahn J.-H.H., Maroteaux M., Bertran-Gonzalez J., Brami-Cherrier K., Enslen H., Corbillé A.-G.G., et al. A phosphatase cascade by which rewarding stimuli control nucleosomal response. Nature 2008, 453:879-884.
37. Tversky A. Elimination by aspects: A theory of choice. Psychol. Rev. 1972, 79:281.
38. Wickens J.R., Begg A.J., Arbuthnott G.W. Dopamine reverses the depression of rat corticostriatal synapses which normally follows high-frequency stimulation of cortex in vitro. Neuroscience 1996, 70:1-5.