The neural coding of expected and unexpected monetary performance outcomes: Dissociations between active and observational learning

Behavioural Brain Research

Volumn 227, Issue 1, 2012, Pages 241-251

  Bellebaum, C ^a   Jokisch, D ^a,b   Gizewski, E R ^b,c   Forsting, M ^b   Daum, I ^a  

^a RUHR UNIVERSITY BOCHUM   (Germany)

^b UNIVERSITY HOSPITAL ESSEN   (Germany)

^c JUSTUS LIEBIG UNIVERSITY   (Germany)

Author keywords

Learning; Observation; Orbitofrontal cortex; Prediction error; Reward; Striatum

Indexed keywords

ADULT; ARTICLE; BEHAVIOR; CAUDATE NUCLEUS; CONTROLLED STUDY; CORPUS STRIATUM; ERROR; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HIPPOCAMPUS; HUMAN; HUMAN EXPERIMENT; LEARNING; MALE; MONEY; NORMAL HUMAN; OBSERVATIONAL LEARNING; ORBITAL CORTEX; OUTCOME ASSESSMENT; PERFORMANCE; PREDICTION; PRIORITY JOURNAL; PUTAMEN; REWARD; RIGHT HEMISPHERE;

ADOLESCENT; ADULT; ANALYSIS OF VARIANCE; BRAIN; BRAIN MAPPING; CHOICE BEHAVIOR; FEEDBACK, PSYCHOLOGICAL; FEMALE; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; LEARNING; MAGNETIC RESONANCE IMAGING; MALE; OBSERVATION; OXYGEN; PATTERN RECOGNITION, VISUAL; PHOTIC STIMULATION; REACTION TIME; REWARD; TIME FACTORS; YOUNG ADULT;

EID: 83655164141     PISSN: 01664328     EISSN: 18727549     Source Type: Journal    
DOI: 10.1016/j.bbr.2011.10.042     Document Type: Article

References (57)

1. Sutton R.S., Barto AG Reinforcement learning: an introduction 1998, MIT Press, Cambridge, MA.
2. Schultz W., Dayan P., Montague P.R. A neural substrate of prediction and reward. Science 1997, 275:1593-1599.
3. Schultz W. Predictive reward signal of dopamine neurons. J Neurophysiol 1998, 80:1-27.
4. Schultz W. The phasic reward signal of primate dopamine neurons. Adv Pharmacol 1998, 42:686-690.
5. Schultz W. Dopamine neurons and their role in reward mechanisms. Curr Opin Neurobiol 1997, 7:191-197.
6. Zaghloul K.A., Blanco J.A., Weidemann C.T., McGill K., Jaggi J.L., Baltuch G.H., et al. Human substantia nigra neurons encode unexpected financial rewards. Science 2009, 323:1496-1499.
7. Haber S.N., Fudge J.L. The primate substantia nigra and VTA: integrative circuitry and function. Crit Rev Neurobiol 1997, 11:323-342.
8. O'Doherty J., Kringelbach M.L., Rolls E.T., Hornak J., Andrews C. Abstract reward and punishment representations in the human orbitofrontal cortex. Nat Neurosci 2001, 4:95-102.
9. Rolls E.T., McCabe C., Redoute J. Expected value, reward outcome, and temporal difference error representations in a probabilistic decision task. Cereb Cortex 2008, 18:652-663.
10. O'Doherty J., Dayan P., Schultz J., Deichmann R., Friston K., Dolan R.J. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 2004, 304:452-454.
11. Tobler P.N., O'Doherty J.P., Dolan R.J., Schultz W. Reward value coding distinct from risk attitude-related uncertainty coding in human reward systems. J Neurophysiol 2007, 97:1621-1632.
12. Knutson B., Cooper J.C. Functional magnetic resonance imaging of reward prediction. Curr Opin Neurol 2005, 18:411-417.
13. Knutson B., Taylor J., Kaufman M., Peterson R., Glover G. Distributed neural representation of expected value. J Neurosci 2005, 25:4806-4812.
14. Knutson B., Fong G.W., Adams C.M., Varner J.L., Hommer D. Dissociation of reward anticipation and outcome with event-related fMRI. Neuroreport 2001, 12:3683-3687.
15. Delgado M.R. Reward-related responses in the human striatum. Ann N Y Acad Sci 2007, 1104:70-88.
16. Delgado M.R., Miller M.M., Inati S., Phelps E.A. An fMRI study of reward-related probability learning. Neuroimage 2005, 24:862-873.
17. Delgado M.R., Locke H.M., Stenger V.A., Fiez J.A. Dorsal striatum responses to reward and punishment: effects of valence and magnitude manipulations. Cogn Affect Behav Neurosci 2003, 3:27-38.
18. Pagnoni G., Zink C.F., Montague P.R., Berns G.S. Activity in human ventral striatum locked to errors of reward prediction. Nat Neurosci 2002, 5:97-98.
19. Holroyd C.B., Nieuwenhuis S., Yeung N., Nystrom L., Mars R.B., Coles M.G., et al. Dorsal anterior cingulate cortex shows fMRI response to internal and external error signals. Nat Neurosci 2004, 7:497-498.
20. Delgado M.R., Nystrom L.E., Fissell C., Noll D.C., Fiez J.A. Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol 2000, 84:3072-3077.
21. Clark L., Lawrence A.J., Astley-Jones F., Gray N. Gambling near-misses enhance motivation to gamble and recruit win-related brain circuitry. Neuron 2009, 61:481-490.
22. Izuma K., Saito D.N., Sadato N. Processing of social and monetary rewards in the human striatum. Neuron 2008, 58:284-294.
23. Yacubian J., Glascher J., Schroeder K., Sommer T., Braus D.F., Buchel C. Dissociable systems for gain- and loss-related value predictions and errors of prediction in the human brain. J Neurosci 2006, 26:9530-9537.
24. Dickerson K.C., Li J., Delgado M.R. Parallel contributions of distinct human memory systems during probabilistic learning. Neuroimage 2011, 55:266-276.
25. Seymour B., Daw N., Dayan P., Singer T., Dolan R. Differential encoding of losses and gains in the human striatum. J Neurosci 2007, 27:4826-4831.
26. Zink C.F., Pagnoni G., Martin-Skurski M.E., Chappelow J.C., Berns G.S. Human striatal responses to monetary reward depend on saliency. Neuron 2004, 42:509-517.
27. Frank M.J., Seeberger L.C., O'Reilly R.C. By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 2004, 306:1940-1943.
28. Bellebaum C., Koch B., Schwarz M., Daum I. Focal basal ganglia lesions are associated with impairments in reward-based reversal learning. Brain 2008, 131:829-841.
29. Cools R., Altamirano L., D'Esposito M. Reversal learning in Parkinson's disease depends on medication status and outcome valence. Neuropsychologia 2006, 44:1663-1673.
30. Cools R., Lewis S.J., Clark L., Barker R.A., Robbins T.W. l-DOPA disrupts activity in the nucleus accumbens during reversal learning in Parkinson's disease. Neuropsychopharmacology 2007, 32:180-189.
31. Gehring W.J., Willoughby A.R. The medial frontal cortex and the rapid processing of monetary gains and losses. Science 2002, 295:2279-2282.
32. Miltner W.H., Braun C.H., Coles M.G.H. Event-related brain potentials following incorrect feedback in a time-estimation task: evidence for a generic neural system for error detection. J Cogn Neurosci 1997, 9:788-798.
33. Bellebaum C., Daum I. Learning-related changes in reward expectancy are reflected in the feedback-related negativity. Eur J Neurosci 2008, 27:1823-1835.
34. Bellebaum C., Kobza S., Thiele S., Daum I. It was not MY fault: event-related brain potentials in active and observational learning from feedback. Cereb Cortex 2010, 20:2874-2883.
35. Burke C.J., Tobler P.N., Baddeley M., Schultz W. Neural mechanisms of observational learning. Proc Natl Acad Sci USA 2010, 107:14431-14436.
36. Andersson J.L., Hutton C., Ashburner J., Turner R., Friston K. Modeling geometric deformations in EPI time series. Neuroimage 2001, 13:903-919.
37. Friston K.J., Glaser D.E., Henson R.N., Kiebel S., Phillips C., Ashburner J. Classical and Bayesian inference in neuroimaging: applications. Neuroimage 2002, 16:484-512.
38. Penny W.D., Holmes A.D. Random effects analysis. Human Brain Function 2003, Academic Press, San Diego, pp. 843-850. R.S.J. Frackowiak, K. Friston, C.D. Frith, R. Dolan, C.J. Price, S.A.J. Zeki (Eds.).
39. Talairach J., Tournoux P. Co-planar stereotaxic atlas of the human brain 1988, Thieme Medical Publishers, New York.
40. Worsley K.J., Marrett S., Neelin P., Vandal A.C., Friston K.J., Evans A.C. A unified statistical approach for determining significant signals in images of cerebral activation. Hum Brain Mapp 1996, 4:58-73.
41. Itagaki S., Katayama J. Self-relevant criteria determine the evaluation of outcomes induced by others. Neuroreport 2008, 19:383-387.
42. O'Doherty J., Rolls E.T., Francis S., Bowtell R., McGlone F. Representation of pleasant and aversive taste in the human brain. J Neurophysiol 2001, 85:1315-1321.
43. Kringelbach M.L., O'Doherty J., Rolls E.T., Andrews C. Activation of the human orbitofrontal cortex to a liquid food stimulus is correlated with its subjective pleasantness. Cereb Cortex 2003, 13:1064-1071.
44. Rolls E.T., O'Doherty J., Kringelbach M.L., Francis S., Bowtell R., McGlone F. Representations of pleasant and painful touch in the human orbitofrontal and cingulate cortices. Cereb Cortex 2003, 13:308-317.
45. Rolls E.T., Kringelbach M.L., de A.I. Different representations of pleasant and unpleasant odours in the human brain. Eur J Neurosci 2003, 18:695-703.
46. O'Doherty J., Rolls E.T., Francis S., Bowtell R., McGlone F., Kobal G., et al. Sensory-specific satiety-related olfactory activation of the human orbitofrontal cortex. Neuroreport 2000, 11:893-897.
47. O'Doherty J.P. Lights, camembert, action! The role of human orbitofrontal cortex in encoding stimuli, rewards, and choices. Ann N Y Acad Sci 2007, 1121:254-272.
48. Kringelbach M.L. The human orbitofrontal cortex: linking reward to hedonic experience. Nat Rev Neurosci 2005, 6:691-702.
49. Breiter H.C., Aharon I., Kahneman D., Dale A., Shizgal P. Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 2001, 30:619-639.
50. O'Doherty J.P., Dayan P., Friston K., Critchley H., Dolan R.J. Temporal difference models and reward-related learning in the human brain. Neuron 2003, 38:329-337.
51. Oya H., Adolphs R., Kawasaki H., Bechara A., Damasio A., Howard M.A. Electrophysiological correlates of reward prediction error recorded in the human prefrontal cortex. Proc Natl Acad Sci USA 2005, 102:8351-8356.
52. Valentin V.V., Dickinson A., O'Doherty J.P. Determining the neural substrates of goal-directed learning in the human brain. J Neurosci 2007, 27:4019-4026.
53. Gottfried J.A., O'Doherty J., Dolan R.J. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 2003, 301:1104-1107.
54. Lisman J.E., Grace A.A. The hippocampal-VTA loop: controlling the entry of information into long-term memory. Neuron 2005, 46:703-713.
55. Poldrack R.A., Clark J., Pare-Blagoev E.J., Shohamy D., Creso M.J., Myers C., et al. Interactive memory systems in the human brain. Nature 2001, 414:546-550.
56. Shohamy D., Myers C.E., Grossman S., Sage J., Gluck M.A., Poldrack R.A. Cortico-striatal contributions to feedback-based learning: converging data from neuroimaging and neuropsychology. Brain 2004, 127:851-859.
57. Elliott R., Agnew Z., Deakin J.F. Hedonic and informational functions of the human orbitofrontal cortex. Cereb Cortex 2010, 20:198-204.