Splitting the difference: How does the brain code reward episodes?

Annals of the New York Academy of Sciences

Volumn 1104, Issue , 2007, Pages 54-69

  Knutson, Brian ^a   Wimmer, G Elliott ^a  

^a STANFORD UNIVERSITY   (United States)

Author keywords

Accumbens; Anticipation; Computation; FMRI; Human; Prefrontal; Reward

Indexed keywords

BRAIN FUNCTION; BRAIN REGION; CONFERENCE PAPER; ERROR; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; LEARNING; MENTAL DISEASE; NEUROANATOMY; NEUROIMAGING; NONHUMAN; NUCLEUS ACCUMBENS; OUTCOME ASSESSMENT; PREDICTION; PREFRONTAL CORTEX; REWARD; ANIMAL; BIOLOGICAL MODEL; BRAIN; BRAIN MAPPING; DECISION MAKING; HISTOLOGY; METHODOLOGY; NUCLEAR MAGNETIC RESONANCE IMAGING; PHYSIOLOGY; REGRESSION ANALYSIS; REPRODUCIBILITY; REVIEW; THEORETICAL MODEL; TIME;

ANIMALIA;

ANIMALS; BRAIN; BRAIN MAPPING; DECISION MAKING; HUMANS; LEARNING; MAGNETIC RESONANCE IMAGING; MODELS, NEUROLOGICAL; MODELS, THEORETICAL; NUCLEUS ACCUMBENS; PREFRONTAL CORTEX; REGRESSION ANALYSIS; REPRODUCIBILITY OF RESULTS; REWARD; TIME FACTORS;

EID: 34447643424     PISSN: 00778923     EISSN: 17496632     Source Type: Book Series    
DOI: 10.1196/annals.1390.020     Document Type: Conference Paper

References (38)

1. TAYLOR, J. 1958. Selected Writings of John Hughlings Jackson. Basic Books. New York.
2. MACLEAN, P.D. 1990. The Triune Brain in Evolution. Plenum Press. New York.
3. LEDOUX, J.E. 1993. Emotional memory: in search of systems and synapses. In Brain Mechanisms: Papers in Memory of Robert Thompson, Vol. 702. F.M. Crinella & J. Yu, Eds.: 149-157. Ann. N.Y. Acad. Sci. New York.
4. PANKSEPP, J. 1998. Affective Neuroscience: The Foundations of Human and Animal Emotions. Oxford University Press. New York.
5. KNUTSON, B. & J.C. COOPER. 2005. Functional magnetic resonance imaging of reward prediction. Curr. Opin. Neurol. 18: 411-417.
6. CRAIG, W. 1918. Appetites and aversions as constituents of instincts. Biol. Bull. 34: 91-107.
7. KNUTSON, B. et al. 2003. A region of mesial prefrontal cortex tracks monetarily rewarding outcomes: characterization with rapid event-related FMRI. Neuroimage 18: 263-272.
8. SUTTON, R.S. & A.G. BARTO. 1990. Time-derivative models of Pavlovian reinforcement. In Learning and Computational Neuroscience: Foundations of Adaptive Networks. M. Gabriel & J. Moore, Eds.: 497-537. MIT Press. Boston.
9. MONTAGUE, P.R., P. DAYAN & T.J. SEJNOWSKI. 1996. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 16: 1936-1947.
10. SCHULTZ, W., P. DAYAN & P.R. MONTAGUE. 1997. A neural substrate of prediction and reward. Science 275: 1593-1599.
11. MIRENOWICZ, J. & W. SCHULTZ. 1996. Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379: 449-451.
12. FIORILLO, C.D., P.N. TOBLER & W. SCHULTZ. 2003. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299: 1898-1902.
13. KAHNEMAN, D. & A. TVERSKY. 1979. Prospect theory: an analysis of decision under risk. Econometrica 47: 263-291.
14. MCCLURE, S.M., G.S. BERNS & P.R. MONTAGUE. 2003. Temporal prediction errors in a passive learning task activate human striatum. Neuron 38: 338-346.
15. O'DOHERTY, J.P. et al. 2003. Temporal difference models and reward-related learning in the human brain. Neuron 38: 329-337.
16. KIM, H., S. SHIMOJO & J.P. O'DOHERTY. 2006. Is avoiding an aversive outcome rewarding? Neural substrates of avoidance learning in the human brain. Public Library of Science Biology 4: e233.
17. DAW, N.D. & K. DOYA. 2006. The computational neurobiology of learning and reward. Curr. Opin. Neurobiol. 16: 199-204.
18. KNUTSON, B. et al. 2001. Dissociation of reward anticipation and outcome with event-related FMRI. Neuroreport 12: 3683-3687.
19. HARUNO, M. et al. 2004. A neural correlate of reward-based behavioral learning in caudate nucleus: a functional magnetic resonance imaging study of a stochastic decision task. J. Neurosci. 24: 1660-1665.
20. JUCKEL, G. et al. 2006. Dysfunction of ventral striatal reward prediction in schizophrenia. Neuroimage 29: 409-416.
21. SCHERES, A. et al. 2007. Ventral striatal hyporesponsiveness during reward anticipation in Attention Deficit/Hyperactivity Disorder. Biol. Psychiatry 61: 720-724.
22. PAVLOV, I.P. 1927. Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex. Oxford University Press. London.
23. KNUTSON, B. et al. 2000. FMRI visualization of brain activity during a monetary incentive delay task. Neuroimage 12: 20-27.
24. KNUTSON, B. et al. 2001. Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J. Neurosci. 21: RC159.
25. COHEN, M.S. 1997. Parametric analysis of fMRI data using linear systems methods. Neuroimage 6: 93-103.
26. COX, R.W. 1996. AFNI: software for analysis and visualization of functional magnetic resonance images. Comp. Biomed. Res. 29: 162-173.
27. GOTTFRIED, J.A., J. O'DOHERTY & R.J. DOLAN. 2003. Appetitive and aversive olfactory learning in humans studied using event-related functional magnetic resonance imaging. J. Neurosci. 22: 10829-10837.
28. O'DOHERTY, J. et al. 2004. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304: 452-454.
29. LOGOTHETIS, N.K. & B.A. WANDELL. 2004. Interpreting the BOLD signal. Ann. Rev. Physiol. 66: 735-769.
30. WIGHTMAN, R.M. & D.L. ROBINSON. 2002. Transient changes in mesolimbic dopamine and their association with 'reward.' J. Neurochem. 82: 721-735.
31. JENKINS, B.G. et al. 2004. Mapping dopamine function in primates using pharmacological magnetic resonance imaging. J. Neurosci. 24: 9553-9560.
32. KNUTSON, B. & S.E.B. GIBBS. 2007. Linking nucleus accumbens dopamine and blood oxygenation. Psychopharmacology 191: 813-822.
33. SHIZGAL, P. 1997. Neural basis of utility estimation. Curr. Opin. Neurobiol. 7: 198-208.
34. KNUTSON, B. et al. 2005. Distributed neural representation of expected value. J. Neurosci 25: 4806-4812.
35. ADCOCK, R.A. et al. 2006. Reward-motivated learning: mesolimbic activation precedes memory formation. Neuron 50: 507-517.
36. DREVETS, W.C. et al. 2001. Amphetamine-induced dopamine release in human ventral striatum correlates with euphoria. Biol. Psychiatry 49: 81-96.
37. JUCKEL, G. et al. 2006. Dysfunction of ventral striatal reward prediction in schizophrenic patients treated with typical but not atypical neuroleptics. Psychopharmacology 187: 222-228.
38. YACUBIAN, J. et al. 2006. Dissociable systems for gain-and loss-related value predictions and errors of prediction in the human brain. J. Neurosci. 26: 9530-9537.