Neural coding of computational factors affecting decision making

Progress in Brain Research

Volumn 202, Issue , 2013, Pages 289-320

  Dreher, Jean Claude ^a  

^a INSTITUT DES SCIENCES COGNITIVES   (France)

Author keywords

Genetic variations; Prediction error; Reward uncertainty; Subjective value; Valuation systems; Value based decision making

Indexed keywords

DOPAMINE;

ANALYTICAL ERROR; ARTICLE; ASSOCIATION; BRAIN; BRAIN ELECTROPHYSIOLOGY; BRAIN FUNCTION; BRAIN REGION; CODING; COMPUTATIONAL FLUID DYNAMICS; CORPUS STRIATUM; COST; DECISION MAKING; DOPAMINERGIC NERVE CELL; ENTROPY; EPILEPSY; EVENT RELATED POTENTIAL; FORCED CHOICE METHOD; FUNCTIONAL MAGNETIC RESONANCE IMAGING; FUNCTIONAL NEUROIMAGING; GENE EXPRESSION; GENETIC VARIABILITY; HAPLORHINI; HELP SEEKING BEHAVIOR; HIPPOCAMPUS; HUMAN; LEARNING; MAGNITUDE ESTIMATION METHOD; MESENCEPHALON; NERVE CELL CODE; NERVE CELL NETWORK; NONHUMAN; ORBITAL CORTEX; PARAMETERS; PREDICTION ERROR; PRIORITY JOURNAL; PROBABILITY; PUNISHMENT; REWARD; SIGNAL NOISE RATIO; SIGNAL PROCESSING; UNCERTAINTY; VENTROMEDIAL PREFRONTAL CORTEX; VISUAL STIMULATION;

EID: 84872387882     PISSN: 00796123     EISSN: 18757855     Source Type: Book Series    
DOI: 10.1016/B978-0-444-62604-2.00016-2     Document Type: Chapter

References (102)

1. Abler B., Walter H., et al. Prediction error as a linear function of reward probability is coded in human nucleus accumbens. Neuroimage 2006, 31(2):790-795.
2. Bayer H.M., Lau B., et al. Statistics of midbrain dopamine neuron spike trains in the awake primate. J. Neurophysiol. 2007, 98(3):1428-1439.
3. Berns G.S., Mc Clure S.M., et al. Predictability modulates human brain response to reward. J. Neurosci. 2001, 21(8):2793-2798.
4. Bertolino A., Blasi G., et al. Additive effects of genetic variation in dopamine regulating genes on working memory cortical activity in human brain. J. Neurosci. 2006, 26(15):3918-3922.
5. Bray S., O'Doherty J. Neural coding of reward-prediction error signals during classical conditioning with attractive faces. J. Neurophysiol. 2007, 97(4):3036-3045.
6. Brischoux F., Chakraborty S., et al. Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc. Natl. Acad. Sci. U.S.A. 2009, 106(12):4894-4899.
7. Caldu X., Dreher J.C. Hormonal and genetic influences on processing reward and social information. Ann. N. Y. Acad. Sci. 2007, 1118:43-73.
8. Caldu X., Vendrell P., et al. Impact of the COMT Val108/158 Met and DAT genotypes on prefrontal function in healthy subjects. Neuroimage 2007, 37(4):1437-1444.
9. Chen Y.C., Galpern W.R., et al. Detection of dopaminergic neurotransmitter activity using pharmacologic MRI: correlation with PET, microdialysis, and behavioral data. Magn. Reson. Med. 1997, 38(3):389-398.
10. Chen J., Lipska B.K., et al. Functional analysis of genetic variation in catechol-O-methyltransferase (COMT): effects on mRNA, protein, and enzyme activity in postmortem human brain. Am. J. Hum. Genet. 2004, 75(5):807-821.
11. Deaner R.O., Khera A.V., et al. Monkeys pay per view: adaptive valuation of social images by rhesus macaques. Curr. Biol. 2005, 15(6):543-548.
12. Domenech P., Dreher J.-C. Distinguishing Two Brain Systems Involved in Choosing Between Different Types of Rewards 2008, Society for Neuroscience, Washington, DC.
13. Domenech P., Dreher J.C. Decision threshold modulation in the human brain. J. Neurosci. 2010, 30(43):14305-14317.
14. Dreher J.C., Kohn P., et al. Neural coding of distinct statistical properties of reward information in humans. Cereb. Cortex 2006, 16(4):561-573.
15. Dreher J.C., Meyer-Lindenberg A., et al. Age-related changes in midbrain dopaminergic regulation of the human reward system. Proc. Natl. Acad. Sci. U.S.A. 2008, 105(39):15106-15111.
16. Dreher J.-C., Kohn P., et al. Variation in dopamine genes influences responsivity of the human reward system. Proc. Natl. Acad. Sci. U.S.A. 2009, 106(2):617-622.
17. Fan J., Fossella J., et al. Mapping the genetic variation of executive attention onto brain activity. Proc. Natl. Acad. Sci. U.S.A. 2003, 100(12):7406-7411.
18. Fiorillo C.D., Tobler P.N., et al. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 2003, 299(5614):1898-1902.
19. Fletcher P.C., Anderson J.M., et al. Responses of human frontal cortex to surprising events are predicted by formal associative learning theory. Nat. Neurosci. 2001, 4(10):1043-1048.
20. Floresco S.B., Tse M.T., et al. Dopaminergic and glutamatergic regulation of effort- and delay-based decision making. Neuropsychopharmacology 2008, 33(8):1966-1979.
21. Frank M.J., Moustafa A.A., et al. Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc. Natl. Acad. Sci. U.S.A. 2007, 104(41):16311-16316.
22. Gan J.O., Walton M.E., et al. Dissociable cost and benefit encoding of future rewards by mesolimbic dopamine. Nat. Neurosci. 2010, 13(1):25-27.
23. Grinband J., Hirsch J., et al. A neural representation of categorization uncertainty in the human brain. Neuron 2006, 49(5):757-763.
24. Hayden B.Y., Platt M.L. Temporal discounting predicts risk sensitivity in rhesus macaques. Curr. Biol. 2007, 17(1):49-53.
25. Heinz A., Goldman D., et al. Genotype influences in vivo dopamine transporter availability in human striatum. Neuropsychopharmacology 2000, 22(2):133-139.
26. Hsu M., Bhatt M., et al. Neural systems responding to degrees of uncertainty in human decision-making. Science 2005, 310(5754):1680-1683.
27. Huettel S.A., Song A.W., et al. Decisions under uncertainty: probabilistic context influences activation of prefrontal and parietal cortices. J. Neurosci. 2005, 25(13):3304-3311.
28. Izuma K., Saito D.N., et al. Processing of social and monetary rewards in the human striatum. Neuron 2008, 58(2):284-294.
29. Jacobsen L.K., Staley J.K., et al. Prediction of dopamine transporter binding availability by genotype: a preliminary report. Am. J. Psychiatry 2000, 157(10):1700-1703.
30. Jones S.R., Gainetdinov R.R., et al. Profound neuronal plasticity in response to inactivation of the dopamine transporter. Proc. Natl. Acad. Sci. U.S.A. 1998, 95(7):4029-4034.
31. Kable J.W., Glimcher P.W. The neural correlates of subjective value during intertemporal choice. Nat. Neurosci. 2007, 10(12):1625-1633.
32. Karama S., Lecours A.R., et al. Areas of brain activation in males and females during viewing of erotic film excerpts. Hum. Brain Mapp. 2002, 16(1):1-13.
33. Karoum F., Chrapusta S.J., et al. 3-Methoxytyramine is the major metabolite of released dopamine in the rat frontal cortex: reassessment of the effects of antipsychotics on the dynamics of dopamine release and metabolism in the frontal cortex, nucleus accumbens, and striatum by a simple two pool model. J. Neurochem. 1994, 63(3):972-979.
34. Klein T.A., Neumann J., et al. Genetically determined differences in learning from errors. Science 2007, 318(5856):1642-1645.
35. Knutson B., Gibbs S.E. Linking nucleus accumbens dopamine and, blood oxygenation. Psychopharmacology (Berl) 2007, 191(3):813-822.
36. Knutson B., Adams C.M., et al. Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J. Neurosci. 2001, 21(16):RC159.
37. Knutson B., Fong G.W., et al. A region of mesial prefrontal cortex tracks monetarily rewarding outcomes: characterization with rapid event-related fMRI. Neuroimage 2003, 18(2):263-272.
38. Knutson B., Taylor J., et al. Distributed neural representation of expected value. J. Neurosci. 2005, 25(19):4806-4812.
39. Kobayashi S., Schultz W. Influence of reward delays on responses of dopamine neurons. J. Neurosci. 2008, 28(31):7837-7846.
40. Kobayashi S., Lauwereyns J., et al. Influence of reward expectation on visuospatial processing in macaque lateral prefrontal cortex. J. Neurophysiol. 2002, 87(3):1488-1498.
41. Kringelbach M.L. The human orbitofrontal cortex: linking reward to hedonic experience. Nat. Rev. Neurosci. 2005, 6(9):691-702.
42. Kufahl P., Li Z., et al. Expectation modulates human brain responses to acute cocaine: a functional magnetic resonance imaging study. Biol. Psychiatry 2008, 63(2):222-230.
43. Lachman H.M., Papolos D.F., et al. Human catechol-O-methyltransferase pharmacogenetics: description of a functional polymorphism and its potential application to neuropsychiatric disorders. Pharmacogenetics 1996, 6(3):243-250.
44. Matsumoto M., Hikosaka O. Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 2009, 459(7248):837-841.
45. Matsumoto M., Weickert C.S., et al. Catechol O-methyltransferase mRNA expression in human and rat brain: evidence for a role in cortical neuronal function. Neuroscience 2003, 116(1):127-137.
46. Matsumoto M., Weickert C.S., et al. Catechol O-methyltransferase (COMT) mRNA expression in the dorsolateral prefrontal cortex of patients with schizophrenia. Neuropsychopharmacology 2003, 28(8):1521-1530.
47. McClure S.M., Berns G.S., et al. Temporal prediction errors in a passive learning task activate human striatum. Neuron 2003, 38(2):339-346.
48. McClure S.M., Ericson K.M., et al. Time discounting for primary rewards. J. Neurosci. 2007, 27(21):5796-5804.
49. Metereau E., Dreher J.C. Cerebral Correlates of Salient Prediction Error for Different Rewards and Punishments. Cereb. Cortex 2012, [Epub ahead of print]. 10.1093/cercor/bhs037.
50. Mill J., Asherson P., et al. Expression of the dopamine transporter gene is regulated by the 3' UTR VNTR: evidence from brain and lymphocytes using quantitative RT-PCR. Am. J. Med. Genet. 2002, 114(8):975-979.
51. O'Doherty J. Can't learn without you: predictive value coding in orbitofrontal cortex requires the basolateral amygdala. Neuron 2003, 39(5):731-733.
52. O'Doherty J.P. Reward representations and reward-related learning in the human brain: insights from neuroimaging. Curr. Opin. Neurobiol. 2004, 14(6):769-776.
53. O'Doherty J.P., Deichmann R., et al. Neural responses during anticipation of a primary taste reward. Neuron 2002, 33(5):815-826.
54. O'Doherty J.P., Dayan P., et al. Temporal difference models and reward-related learning in the human brain. Neuron 2003, 38(2):329-337.
55. Padoa-Schioppa C., Assad J.A. Neurons in the orbitofrontal cortex encode economic value. Nature 2006, 441(7090):223-226.
56. Padoa-Schioppa C., Assad J.A. The representation of economic value in the orbitofrontal cortex is invariant for changes of menu. Nat. Neurosci. 2008, 11(1):95-102.
57. Paulus M.P. Decision-making dysfunctions in psychiatry-altered homeostatic processing?. Science 2007, 318(5850):602-606.
58. Pezze M.A., Feldon J. Mesolimbic dopaminergic pathways in fear conditioning. Prog. Neurobiol. 2004, 74(5):301-320.
59. Preuschoff K., Bossaerts P., et al. Neural differentiation of expected reward and risk in human subcortical structures. Neuron 2006, 51(3):381-390.
60. Preuschoff K., Quartz S.R., et al. Human insula activation reflects risk prediction errors as well as risk. J. Neurosci. 2008, 28(11):2745-2752.
61. Prevost C., Pessiglione M., et al. Separate valuation subsystems for delay and effort decision costs. J. Neurosci. 2010, 30(42):14080-14090.
62. Rangel A., Camerer C., et al. A framework for studying the neurobiology of value-based decision making. Nat. Rev. Neurosci. 2008, 9(7):545-556.
63. Redoute J., Stoleru S., et al. Brain processing of visual sexual stimuli in human males. Hum. Brain Mapp. 2000, 11(3):162-177.
64. Reynolds J.N., Hyland B.I., et al. A cellular mechanism of reward-related learning. Nature 2001, 413(6851):67-70.
65. Risinger R.C., Salmeron B.J., et al. Neural correlates of high and craving during cocaine self-administration using BOLD fMRI. Neuroimage 2005, 26(4):1097-1108.
66. Roesch M.R., Calu D.J., et al. Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat. Neurosci. 2007, 10(12):1615-1624.
67. Rudebeck P.H., Walton M.E., et al. Separate neural pathways process different decision costs. Nat. Neurosci. 2006, 9(9):1161-1168.
68. Rushworth M.F., Behrens T.E., et al. Contrasting roles for cingulate and orbitofrontal cortex in decisions and social behaviour. Trends Cogn. Sci. 2007, 11(4):168-176.
69. Schott B.H., Seidenbecher C.I., et al. The dopaminergic midbrain participates in human episodic memory formation: evidence from genetic imaging. J. Neurosci. 2006, 26(5):1407-1417.
70. Schultz W. Multiple reward signals in the brain. Nat. Rev. Neurosci. 2000, 1(3):199-207.
71. Schultz W., Dickinson A. Neuronal coding of prediction errors. Annu. Rev. Neurosci. 2000, 23:473-500.
72. Schultz W., Dayan P., et al. A neural substrate of prediction and reward. Science 1997, 275(5306):1593-1599.
73. Schweighofer N., Tanaka S.C., et al. Serotonin and the evaluation of future rewards: theory, experiments, and possible neural mechanisms. Ann. N. Y. Acad. Sci. 2007, 1104:289-300.
74. Schweighofer N., Bertin M., et al. Low-serotonin levels increase delayed reward discounting in humans. J. Neurosci. 2008, 28(17):4528-4532.
75. Sesack S.R., Hawrylak V.A., et al. Dopamine axon varicosities in the prelimbic division of the rat prefrontal cortex exhibit sparse immunoreactivity for the dopamine transporter. J. Neurosci. 1998, 18(7):2697-2708.
76. Sesack S.R., Carr D.B., et al. Anatomical substrates for glutamate-dopamine interactions: evidence for specificity of connections and extrasynaptic actions. Ann. N. Y. Acad. Sci. 2003, 1003:36-52.
77. Sescousse, G., Caldú, X., et al., Processing of primary and secondary rewards: a quantitative meta-analysis and review of human functional neuroimaging studies, Neuroscience & BioBehavioral Reviews, in press.
78. Sescousse G., Dreher J.-C. Coding of reward type along an antero-posterior gradient in the human orbitofrontal cortex. 2008, Exciting Biologies meeting, Chantilly, France, Biology of Cognition.
79. Sescousse G., Redoute J., et al. The architecture of reward value coding in the human orbitofrontal cortex. J. Neurosci. 2010, 30(39):13095-13104.
80. Seymour B., O'Doherty J.P., et al. Temporal difference models describe higher-order learning in humans. Nature 2004, 429(6992):664-667.
81. Seymour B., O'Doherty J.P., et al. Opponent appetitive-aversive neural processes underlie predictive learning of pain relief. Nat. Neurosci. 2005, 8(9):1234-1240.
82. Seymour B., Singer T., et al. The neurobiology of punishment. Nat. Rev. Neurosci. 2007, 8(4):300-311.
83. Stevens J.R., Rosati A.G., et al. Will travel for food: spatial discounting in two new world monkeys. Curr. Biol. 2005, 15(20):1855-1860.
84. Sugrue L.P., Corrado G.S., et al. Choosing the greater of two goods: neural currencies for valuation and decision making. Nat. Rev. Neurosci. 2005, 6(5):363-375.
85. Sutton R.S., Barto A.G. Reinforcement learning: an introduction. IEEE Trans. Neural Netw. 1998, 9(5):1054.
86. Suzuki W.A., Amaral D.G. Topographic organization of the reciprocal connections between the monkey entorhinal cortex and the perirhinal and parahippocampal cortices. J. Neurosci. 1994, 14(3Pt.2):1856-1877.
87. Tanaka S.C., Doya K., et al. Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat. Neurosci. 2004, 7(8):887-893.
88. Thomas J., Vanni-Mercier G. Temporal dynamics of reward probability coding: a magnetoencephalographic study in humans. 2008, Human Brain Mapping meeting, Melbourne, Australia.
89. Tobler P.N., Fiorillo C.D., et al. Adaptive coding of reward value by dopamine neurons. Science 2005, 307(5715):1642-1645.
90. Tobler P.N., O'Doherty J.P., et al. Reward value coding distinct from risk attitude-related uncertainty coding in human reward systems. J. Neurophysiol. 2007, 97(2):1621-1632.
91. Tremblay L., Schultz W. Relative reward preference in primate orbitofrontal cortex. Nature 1999, 398(6729):704-708.
92. Vandenbergh D.J., Persico A.M., et al. Human dopamine transporter gene (DAT1) maps to chromosome 5p15.3 and displays a VNTR. Genomics 1992, 14(4):1104-1106.
93. Van Dyck C.H., Malison R.T., et al. Increased dopamine transporter availability associated with the 9-repeat allele of the SLC6A3 gene. J. Nucl. Med. 2005, 46:745-751.
94. VanNess S.H., Owens M.J., et al. The variable number of tandem repeats element in DAT1 regulates in vitro dopamine transporter density. BMC Genet. 2005, 6:55.
95. Vanni-Mercier G., Mauguière F., et al. The hippocampus codes the uncertainty of cue-outcome associations: an intracranial electrophysiological study in humans. J. Neurosci. 2009, 29(16):5287-5294.
96. Walton M.E., Kennerley S.W., et al. Weighing up the benefits of work: behavioral and neural analyses of effort-related decision making. Neural Netw. 2006, 19(8):1302-1314.
97. Walton M.E., Groves J., et al. Comparing the role of the anterior cingulate cortex and 6-hydroxydopamine nucleus accumbens lesions on operant effort-based decision making. Eur. J. Neurosci. 2009, 29(8):1678-1691.
98. Watanabe M., Hikosaka K., et al. Coding and monitoring of motivational context in the primate prefrontal cortex. J. Neurosci. 2002, 22(6):2391-2400.
99. Winston J.S., O'Doherty J., et al. Brain systems for assessing facial attractiveness. Neuropsychologia 2007, 45(1):195-206.
100. Yacubian J., Glascher J., et al. Dissociable systems for gain- and loss-related value predictions and errors of prediction in the human brain. J. Neurosci. 2006, 26(37):9530-9537.
101. Yacubian J., Sommer T., et al. Gene-gene interaction associated with neural reward sensitivity. Proc. Natl. Acad. Sci. U.S.A. 2007, 104(19):8125-8130.
102. Yavich L., Forsberg M.M., et al. Site-specific role of catechol-O-methyltransferase in dopamine overflow within prefrontal cortex and dorsal striatum. J. Neurosci. 2007, 27(38):10196-10209.