Real and fictive outcomes are processed differently but converge on a common adaptive mechanism

Neuron

Volumn 79, Issue 6, 2013, Pages 1243-1255

  Fischer, AdrianG ^a,c   Ullsperger, Markus ^a,b,c,d  

^a OTTO VON GUERICKE UNIVERSITY   (Germany)

^b RADBOUD UNIVERSITY   (Netherlands)

^c MAX PLANCK INSTITUTE FOR NEUROLOGICAL RESEARCH   (Germany)

^d CENTER FOR BEHAVIORAL BRAIN SCIENCES   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTATION; ADULT; ARTICLE; BEHAVIOR; BRAIN MAPPING; DECISION MAKING; DOUBLE DISSOCIATION; ELECTROENCEPHALOGRAPHY; EVENT RELATED POTENTIAL; FEEDBACK SYSTEM; FEMALE; HUMAN; HUMAN EXPERIMENT; LEARNING; MALE; MATHEMATICAL MODEL; NORMAL HUMAN; PARIETAL CORTEX; PREDICTION; PRIORITY JOURNAL; PUNISHMENT; REINFORCEMENT; REWARD; TASK PERFORMANCE;

ADULT; ALGORITHMS; ANALYSIS OF VARIANCE; BRAIN; BRAIN MAPPING; COMPUTER SIMULATION; CONDITIONING, OPERANT; DECISION MAKING; ELECTROENCEPHALOGRAPHY; EVENT-RELATED POTENTIALS, P300; FEEDBACK, PSYCHOLOGICAL; FEMALE; HUMANS; LOGISTIC MODELS; MALE; MODELS, BIOLOGICAL; REACTION TIME; REALITY TESTING; YOUNG ADULT;

EID: 84884276335     PISSN: 08966273     EISSN: 10974199     Source Type: Journal    
DOI: 10.1016/j.neuron.2013.07.006     Document Type: Article

References (55)

1. Amiez C., Joseph J.P., Procyk E. Reward encoding in the monkey anterior cingulate cortex. Cereb. Cortex 2006, 16:1040-1055.
2. Behrens T.E.J., Woolrich M.W., Walton M.E., Rushworth M.F.S. Learning the value of information in an uncertain world. Nat. Neurosci. 2007, 10:1214-1221.
3. Benjamini Y., Yekutieli D. The control of the false discovery rate in multiple testing under dependency. Ann. Stat. 2001, 29:1165-1188.
4. Boorman E.D., Behrens T.E., Rushworth M.F. Counterfactual choice and learning in a neural network centered on human lateral frontopolar cortex. PLoS Biol. 2011, 9:e1001093.
5. Burke C.J., Tobler P.N., Baddeley M., Schultz W. Neural mechanisms of observational learning. Proc. Natl. Acad. Sci. USA 2010, 107:14431-14436.
6. Caplin A., Dean M. Axiomatic methods, dopamine and reward prediction error. Curr. Opin. Neurobiol. 2008, 18:197-202.
7. Chase H.W., Swainson R., Durham L., Benham L., Cools R. Feedback-related negativity codes prediction error but not behavioral adjustment during probabilistic reversal learning. J.Cogn. Neurosci. 2011, 23:936-946.
8. Chiu P.H., Lohrenz T.M., Montague P.R. Smokers' brains compute, but ignore, a fictive error signal in a sequential investment task. Nat. Neurosci. 2008, 11:514-520.
9. Cohen M.X., Cavanagh J.F. Single-trial regression elucidates the role of prefrontal theta oscillations in response conflict. Front. Psychol. 2011, 2:30.
10. de Bruijn E.R.A., de Lange F.P., von Cramon D.Y., Ullsperger M. When errors are rewarding. J.Neurosci. 2009, 29:12183-12186.
11. Delorme A., Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J.Neurosci. Methods 2004, 134:9-21.
12. Fabiani M., Karis D., Donchin E. Effects of mnemonic strategy manipulation in a Von Restorff paradigm. Electroencephalogr. Clin. Neurophysiol. 1990, 75:22-35.
13. Ferdinand N.K., Mecklinger A., Kray J., Gehring W.J. The processing of unexpected positive response outcomes in the mediofrontal cortex. J.Neurosci. 2012, 32:12087-12092.
14. Gehring W.J., Willoughby A.R. The medial frontal cortex and the rapid processing of monetary gains and losses. Science 2002, 295:2279-2282.
15. Gold J.I., Shadlen M.N. The neural basis of decision making. Annu. Rev. Neurosci. 2007, 30:535-574.
16. Groppe D.M., Urbach T.P., Kutas M. Mass univariate analysis of event-related brain potentials/fields I: a critical tutorial review. Psychophysiology 2011, 48:1711-1725.
17. Gruendler T.O.J., Ullsperger M., Huster R.J. Event-related potential correlates of performance-monitoring in a lateralized time-estimation task. PLoS ONE 2011, 6:e25591.
18. Hayden B.Y., Nair A.C., McCoy A.N., Platt M.L. Posterior cingulate cortex mediates outcome-contingent allocation of behavior. Neuron 2008, 60:19-25.
19. Hayden B.Y., Pearson J.M., Platt M.L. Fictive reward signals in the anterior cingulate cortex. Science 2009, 324:948-950.
20. Holroyd C.B., Coles M.G.H. The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol. Rev. 2002, 109:679-709.
21. Hunt L.T., Kolling N., Soltani A., Woolrich M.W., Rushworth M.F.S., Behrens T.E.J. Mechanisms underlying cortical activity during value-guided choice. Nat. Neurosci. 2012, 15:470-476. S1-S3.
22. Jocham G., Hunt L.T., Near J., Behrens T.E.J. A mechanism for value-guided choice based on the excitation-inhibition balance in prefrontal cortex. Nat. Neurosci. 2012, 15:960-961.
23. Jocham G., Neumann J., Klein T.A., Danielmeier C., Ullsperger M. Adaptive coding of action values in the human rostral cingulate zone. J.Neurosci. 2009, 29:7489-7496.
24. Kennerley S.W., Walton M.E., Behrens T.E.J., Buckley M.J., Rushworth M.F.S. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 2006, 9:940-947.
25. Klein T.A., Neumann J., Reuter M., Hennig J., von Cramon D.Y., Ullsperger M. Genetically determined Differences in Learning from errors. Science 2007, 318:1642-1645.
26. Knutson B., Taylor J., Kaufman M., Peterson R., Glover G. Distributed neural representation of expected value. J.Neurosci. 2005, 25:4806-4812.
27. Leichnetz G.R. Connections of the medial posterior parietal cortex (area 7m) in the monkey. Anat. Rec. 2001, 263:215-236.
28. Lohrenz T., McCabe K., Camerer C.F., Montague P.R. Neural signature of fictive learning signals in a sequential investment task. Proc. Natl. Acad. Sci. USA 2007, 104:9493-9498.
29. Mars R.B., Jbabdi S., Sallet J., O'Reilly J.X., Croxson P.L., Olivier E., Noonan M.P., Bergmann C., Mitchell A.S., Baxter M.G., et al. Diffusion-weighted imaging tractography-based parcellation of the human parietal cortex and comparison with human and macaque resting-state functional connectivity. J.Neurosci. 2011, 31:4087-4100.
30. Mars R.B., Debener S., Gladwin T.E., Harrison L.M., Haggard P., Rothwell J.C., Bestmann S. Trial-by-trial fluctuations in the event-related electroencephalogram reflect dynamic changes in the degree of surprise. J.Neurosci. 2008, 28:12539-12545.
31. Miltner W.H.R., Braun C.H., Coles M.G.H. Event-related brain potentials following incorrect feedback in a time-estimation task: Evidence for a "generic" neural system for error detection. J.Cogn. Neurosci. 1997, 9:788-798.
32. Nichols T., Brett M., Andersson J., Wager T., Poline J.-B. Valid conjunction inference with the minimum statistic. Neuroimage 2005, 25:653-660.
33. Nieuwenhuis S., Holroyd C.B., Mol N., Coles M.G.H. Reinforcement-related brain potentials from medial frontal cortex: origins and functional significance. Neurosci. Biobehav. Rev. 2004, 28:441-448.
34. O'Leary D.P. Robust regression computation using iteratively reweighted least squares. SIAM J. Matrix Anal. Appl. 1990, 11:466-480.
35. Paller K.A., Kutas M., Mayes A.R. Neural correlates of encoding in an incidental learning paradigm. Electroencephalogr. Clin. Neurophysiol. 1987, 67:360-371.
36. Parvizi J., Van Hoesen G.W., Buckwalter J., Damasio A. Neural connections of the posteromedial cortex in the macaque. Proc. Natl. Acad. Sci. USA 2006, 103:1563-1568.
37. Philiastides M.G., Biele G., Vavatzanidis N., Kazzer P., Heekeren H.R. Temporal dynamics of prediction error processing during reward-based decision making. Neuroimage 2010, 53:221-232.
38. Plassmann H., O'Doherty J.P., Rangel A. Appetitive and aversive goal values are encoded in the medial orbitofrontal cortex at the time of decision making. J.Neurosci. 2010, 30:10799-10808.
39. Polich J. Updating P300: an integrative theory of P3a and P3b. Clin. Neurophysiol. 2007, 118:2128-2148.
40. Ridderinkhof K.R., Ullsperger M., Crone E.A., Nieuwenhuis S. The role of the medial frontal cortex in cognitive control. Science 2004, 306:443-447.
41. Rizzolatti G., Fogassi L., Gallese V. Neurophysiological mechanisms underlying the understanding and imitation of action. Nat. Rev. Neurosci. 2001, 2:661-670.
42. Rousselet G.A., Pernet C.R., Bennett P.J., Sekuler A.B. Parametric study of EEG sensitivity to phase noise during face processing. BMC Neurosci. 2008, 9:98.
43. Schultz W., Dayan P., Montague P.R. A neural substrate of prediction and reward. Science 1997, 275:1593-1599.
44. Subiaul F., Cantlon J.F., Holloway R.L., Terrace H.S. Cognitive imitation in rhesus macaques. Science 2004, 305:407-410.
45. Summerfield C., Trittschuh E.H., Monti J.M., Mesulam M.M., Egner T. Neural repetition suppression reflects fulfilled perceptual expectations. Nat. Neurosci. 2008, 11:1004-1006.
46. Sutton R.S., Barto A.G. Reinforcement Learning: an Introduction 1998, MIT Press, Cambridge.
47. Talmi D., Fuentemilla L., Litvak V., Duzel E., Dolan R.J. An MEG signature corresponding to an axiomatic model of reward prediction error. Neuroimage 2012, 59:635-645.
48. Ullsperger M., von Cramon D.Y. Error monitoring using external feedback: specific roles of the habenular complex, the reward system, and the cingulate motor area revealed by functional magnetic resonance imaging. J.Neurosci. 2003, 23:4308-4314.
49. Ullsperger P., Gille H.G. The late positive component of the ERP and adaptation-level theory. Biol. Psychol. 1988, 26:299-306.
50. van Schie H.T., Mars R.B., Coles M.G.H., Bekkering H. Modulation of activity in medial frontal and motor cortices during error observation. Nat. Neurosci. 2004, 7:549-554.
51. Walton M.E., Devlin J.T., Rushworth M.F.S. Interactions between decision making and performance monitoring within prefrontal cortex. Nat. Neurosci. 2004, 7:1259-1265.
52. Watkins C.J.C.H., Dayan P. Q-Learning. Mach. Learn. 1992, 8:279-292.
53. Wessel J.R., Danielmeier C., Ullsperger M. Error awareness revisited: accumulation of multimodal evidence from central and autonomic nervous systems. J.Cogn. Neurosci. 2011, 23:3021-3036.
54. Wunderlich K., Rangel A., O'Doherty J.P. Economic choices can be made using only stimulus values. Proc. Natl. Acad. Sci. USA 2010, 107:15005-15010.
55. Yu R., Zhou X. Brain responses to outcomes of one's own and other's performance in a gambling task. Neuroreport 2006, 17:1747-1751.