Feedback valuation processing within the prefrontal cortex

Handbook of Reward and Decision Making

Volumn , Issue , 2009, Pages 213-228

  Amiez, Céline ^a   Petrides, Michael ^a  

^a MCGILL UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84857035294     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1016/B978-0-12-374620-7.00009-1     Document Type: Chapter

References (96)

1. Mackintosh N.J. The Psychology of Animal Learning 1974, Academic Press, London.
2. Sutton R.S., Barto A.G. Reinforcement Learning: An Introduction 1998, MIT Press, Cambridge, MA.
3. Fiorillo C.D., Tobler P.N., Schultz W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 2003, 299:1898-1902.
4. Holroyd C.B., Coles M.G. The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol. Rev. 2002, 109:679-709.
5. Schultz W., Dickinson A. Neuronal coding of prediction errors. Annu. Rev. Neurosci. 2000, 23:473-500.
6. Williams S.M., Goldman-Rakic P.S. Widespread origin of the primate mesofrontal dopamine system. Cereb. Cortex 1998, 8:321-345.
7. Robbins T.W., Roberts A.C. Differential regulation of fronto-executive function by the monoamines and acetylcholine. Cereb. Cortex 2007, 17(Suppl 1):i151-i160.
8. Walker S.C., Robbins T.W., Roberts A.C. Differential contributions of dopamine and serotonin to orbitofrontal cortex function in the marmoset. Cereb. Cortex 2008.
9. Krawczyk D.C. Contributions of the prefrontal cortex to the neural basis of human decision making. Neurosci. Biobehav. Rev. 2002, 26:631-664.
10. Rushworth M.F., Behrens T.E. Choice, uncertainty and value in prefrontal and cingulate cortex. Nat. Neurosci. 2008, 11:389-397.
11. Rushworth M.F., Behrens T.E., Rudebeck P.H., Walton M.E. Contrasting roles for cingulate and orbitofrontal cortex in decisions and social behaviour. Trends Cogn. Sci. 2007, 11:168-176.
12. Rushworth M.F., Buckley M.J., Behrens T.E., Walton M.E., Bannerman D.M. Functional organization of the medial frontal cortex. Curr. Opin. Neurobiol. 2007, 17:220-227.
13. Rushworth M.F., Walton M.E., Kennerley S.W., Bannerman D.M. Action sets and decisions in the medial frontal cortex. Trends Cogn. Sci. 2004, 8:410-417.
14. Paus T. Primate anterior cingulate cortex: where motor control, drive and cognition interface. Nat. Rev. Neurosci. 2001, 2:417-424.
15. Procyk E., Ford Dominey P., Amiez C., Joseph J.P. The effects of sequence structure and reward schedule on serial reaction time learning in the monkey. Brain Res. Cogn. Brain Res. 2000, 9:239-248.
16. Walton M.E., Devlin J.T., Rushworth M.F. Interactions between decision making and performance monitoring within prefrontal cortex. Nat. Neurosci. 2004, 7:1259-1265.
17. Petrides M., Pandya D.N. Comparative architectonic analysis of the human and the macaque frontal cortex. Handbook Neuropsychol. 1994, 9:17-58.
18. Matsumoto K., Suzuki W., Tanaka K. Neuronal correlates of goal-based motor selection in the prefrontal cortex. Science 2003, 301:229-232.
19. Sallet J., Quilodran R., Rothe M., Vezoli J., Joseph J.P., Procyk E. Expectations, gains, and losses in the anterior cingulate cortex. Cogn. Affect. Behav. Neurosci. 2007, 7:327-336.
20. Amiez C., Joseph J.P., Procyk E. Reward encoding in the monkey anterior cingulate cortex. Cereb. Cortex 2006, 16:1040-1055.
21. Critchley H.D., Mathias C.J., Dolan R.J. Neural activity in the human brain relating to uncertainty and arousal during anticipation. Neuron 2001, 29:537-545.
22. Rogers R.D., Ramnani N., Mackay C., Wilson J.L., Jezzard P., Carter C.S., Smith S.M. Distinct portions of anterior cingulate cortex and medial prefrontal cortex are activated by reward processing in separable phases of decision-making cognition. Biol. Psychiatry 2004, 55:594-602.
23. Kable J.W., Glimcher P.W. The neural correlates of subjective value during intertemporal choice. Nat. Neurosci. 2007, 10:1625-1633.
24. Dreher J.C., Kohn P., Berman K.F. Neural coding of distinct statistical properties of reward information in humans. Cereb. Cortex 2006, 16:561-573.
25. Falkenstein M., Hohnsbein J., Hoormann J., Blanke L. Effects of crossmodal divided attention on late ERP components. II. Error processing in choice reaction tasks. Electroencephalogr. Clin. Neurophysiol. 1991, 78:447-455.
26. Fiehler K., Ullsperger M., von Cramon D.Y. Neural correlates of error detection and error correction: is there a common neuroanatomical substrate?. Eur. J. Neurosci. 2004, 19:3081-3087.
27. Garavan H., Ross T.J., Kaufman J., Stein E.A. A midline dissociation between error-processing and response-conflict monitoring. Neuroimage 2003, 20:1132-1139.
28. Gehring W.J., Goss B., Coles M.G.H., Meyer D.E., Donchin E. A neural system for error-detection and compensation. Psychol. Sci. 1993, 4:385-390.
29. Hester R., Foxe J.J., Molholm S., Shpaner M., Garavan H. Neural mechanisms involved in error processing: a comparison of errors made with and without awareness. Neuroimage 2005, 27:602-608.
30. Holroyd C.B., Nieuwenhuis S., Yeung N., Nystrom L., Mars R.B., Coles M.G., Cohen J.D. Dorsal anterior cingulate cortex shows fMRI response to internal and external error signals. Nat. Neurosci. 2004, 7:497-498.
31. Klein T.A., Endrass T., Kathmann N., Neumann J., von Cramon D.Y., Ullsperger M. Neural correlates of error awareness. Neuroimage 2007, 34:1774-1781.
32. Magno E., Foxe J.J., Molholm S., Robertson I.H., Garavan H. The anterior cingulate and error avoidance. J. Neurosci. 2006, 26:4769-4773.
33. Monchi O., Petrides M., Petre V., Worsley K., Dagher A. Wisconsin Card Sorting revisited: distinct neural circuits participating in different stages of the task identified by event-related functional magnetic resonance imaging. J. Neurosci. 2001, 21:7733-7741.
34. Taylor S.F., Martis B., Fitzgerald K.D., Welsh R.C., Abelson J.L., Liberzon I., Himle J.A., Gehring W.J. Medial frontal cortex activity and loss-related responses to errors. J. Neurosci. 2006, 26:4063-4070.
35. Ullsperger M., Nittono H., von Cramon D.Y. When goals are missed: dealing with self-generated and externally induced failure. Neuroimage 2007, 35:1356-1364.
36. Ullsperger M., von Cramon D.Y. Error monitoring using external feedback: specific roles of the habenular complex, the reward system, and the cingulate motor area revealed by functional magnetic resonance imaging. J. Neurosci. 2003, 23:4308-4314.
37. Ullsperger M., von Cramon D.Y. Subprocesses of performance monitoring: a dissociation of error processing and response competition revealed by event-related fMRI and ERPs. Neuroimage 2001, 14:1387-1401.
38. Behrens T.E., Woolrich M.W., Walton M.E., Rushworth M.F. Learning the value of information in an uncertain world. Nat. Neurosci. 2007, 10:1214-1221.
39. Cohen M.X. Individual differences and the neural representations of reward expectation and reward prediction error. Soc. Cogn. Affect. Neurosci. 2007, 2:20-30.
40. Ito S., Stuphorn V., Brown J.W., Schall J.D. Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 2003, 302:120-122.
41. Quilodran R., Rothe M., Procyk E. Behavioral shifts and action valuation in the anterior cingulate cortex. Neuron 2008, 57:314-325.
42. Amiez C., Joseph J.P., Procyk E. Anterior cingulate error-related activity is modulated by predicted reward. Eur. J. Neurosci. 2005, 21:3447-3452.
43. Matsumoto M., Matsumoto K., Abe H., Tanaka K. Medial prefrontal cell activity signaling prediction errors of action values. Nat. Neurosci. 2007, 10:647-656.
44. Bechara A., Damasio A.R., Damasio H., Anderson S.W. Insensitivity to future consequences following damage to human prefrontal cortex. Cognition 1994, 50:7-15.
45. Bechara A., Damasio H., Tranel D., Damasio A.R. Deciding advantageously before knowing the advantageous strategy. Science 1997, 275:1293-1295.
46. Bechara A., Tranel D., Damasio H. Characterization of the decision-making deficit of patients with ventromedial prefrontal cortex lesions. Brain 2000, 123(Pt 11):2189-2202.
47. Hornak J., O'Doherty J., Bramham J., Rolls E.T., Morris R.G., Bullock P.R., Polkey C.E. Reward-related reversal learning after surgical excisions in orbito-frontal or dorsolateral prefrontal cortex in humans. J. Cogn. Neurosci. 2004, 16:463-478.
48. Rolls E.T., Hornak J., Wade D., McGrath J. Emotion-related learning in patients with social and emotional changes associated with frontal lobe damage. J. Neurol. Neurosurg. Psychiatry 1994, 57:1518-1524.
49. Izquierdo A., Suda R.K., Murray E.A. Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. J. Neurosci. 2004, 24:7540-7548.
50. Petrides M., Alivisatos B., Frey S. Differential activation of the human orbital, mid-ventrolateral, and mid-dorsolateral prefrontal cortex during the processing of visual stimuli. Proc. Natl. Acad. Sci. USA 2002, 99:5649-5654.
51. Nobre A.C., Coull J.T., Frith C.D., Mesulam M.M. Orbitofrontal cortex is activated during breaches of expectation in tasks of visual attention. Nat. Neurosci. 1999, 2:11-12.
52. Petrides M. The orbitofrontal cortex: novelty, deviation from expectation, and memory. Ann. NY Acad. Sci. 2007, 1121:33-53.
53. Breiter H.C., Aharon I., Kahneman D., Dale A., Shizgal P. Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 2001, 30:619-639.
54. O'Doherty J., Critchley H., Deichmann R., Dolan R.J. Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices. J. Neurosci. 2003, 23:7931-7939.
55. O'Doherty J., Kringelbach M.L., Rolls E.T., Hornak J., Andrews C. Abstract reward and punishment representations in the human orbitofrontal cortex. Nat. Neurosci. 2001, 4:95-102.
56. Thut G., Schultz W., Roelcke U., Nienhusmeier M., Missimer J., Maguire R.P., Leenders K.L. Activation of the human brain by monetary reward. Neuroreport 1997, 8:1225-1228.
57. Tobler P.N., O'Doherty J.P., Dolan R.J., Schultz W. Reward value coding distinct from risk attitude-related uncertainty coding in human reward systems. J. Neurophysiol. 2007, 97:1621-1632.
58. O'Doherty J.P., Deichmann R., Critchley H.D., Dolan R.J. Neural responses during anticipation of a primary taste reward. Neuron 2002, 33:815-826.
59. De Araujo I.E., Rolls E.T. Representation in the human brain of food texture and oral fat. J. Neurosci. 2004, 24:3086-3093.
60. Small D.M., Zatorre R.J., Dagher A., Evans A.C., Jones-Gotman M. Changes in brain activity related to eating chocolate: from pleasure to aversion. Brain 2001, 124:1720-1733.
61. Rolls E.T., O'Doherty J., Kringelbach M.L., Francis S., Bowtell R., McGlone F. Representations of pleasant and painful touch in the human orbitofrontal and cingulate cortices. Cereb. Cortex 2003, 13:308-317.
62. Ursu S., Carter C.S. Outcome representations, counterfactual comparisons and the human orbitofrontal cortex: implications for neuroimaging studies of decision-making. Brain Res. Cogn. Brain Res. 2005, 23:51-60.
63. Hikosaka K., Watanabe M. Delay activity of orbital and lateral prefrontal neurons of the monkey varying with different rewards. Cereb. Cortex 2000, 10:263-271.
64. Padoa-Schioppa C., Assad J.A. Neurons in the orbitofrontal cortex encode economic value. Nature 2006, 441:223-226.
65. Procyk E., Amiez C., Quilodran R., Joseph J.P. Modulations of prefrontal activity related to cognitive control and performance monitoring. Attention and Performance XXII: Sensorimotor Foundations of Higher Cognition 2007, 235-250. Oxford University Press, Oxford. Y. Rossetti, P. Haggard, M. Kawato (Eds.).
66. Roesch M.R., Olson C.R. Neuronal activity related to reward value and motivation in primate frontal cortex. Science 2004, 304:307-310.
67. Rolls E.T. The orbitofrontal cortex. Philos. Trans. R. Soc. Lond. B Biol. Sci. 1996, 351:1433-1443. discussion 1443-1434.
68. Rolls E.T., Yaxley S., Sienkiewicz Z.J. Gustatory responses of single neurons in the caudolateral orbitofrontal cortex of the macaque monkey. J. Neurophysiol. 1990, 64:1055-1066.
69. Schultz W., Tremblay L., Hollerman J.R. Reward processing in primate orbitofrontal cortex and basal ganglia. Cereb. Cortex 2000, 10:272-284.
70. Thorpe S.J., Rolls E.T., Maddison S. The orbitofrontal cortex: neuronal activity in the behaving monkey. Exp. Brain Res. 1983, 49:93-115.
71. Tremblay L., Schultz W. Modifications of reward expectation-related neuronal activity during learning in primate orbitofrontal cortex. J. Neurophysiol. 2000, 83:1877-1885.
72. Tremblay L., Schultz W. Relative reward preference in primate orbitofrontal cortex. Nature 1999, 398:704-708.
73. Tremblay L., Schultz W. Reward-related neuronal activity during go-nogo task performance in primate orbitofrontal cortex. J. Neurophysiol. 2000, 83:1864-1876.
74. Niki H., Sakai M., Kubota K. Delayed alternation performance and unit activity of the caudate head and medial orbitofrontal gyrus in the monkey. Brain Res. 1972, 38:343-353.
75. Turken A.U., Swick D. The effect of orbitofrontal lesions on the error-related negativity. Neurosci. Lett. 2008, 441:7-10.
76. Watanabe M. Reward expectancy in primate prefrontal neurons. Nature 1996, 382:629-632.
77. Amemori K., Sawaguchi T. Contrasting effects of reward expectation on sensory and motor memories in primate prefrontal neurons. Cereb. Cortex 2006, 16:1002-1015.
78. Kobayashi S., Lauwereyns J., Koizumi M., Sakagami M., Hikosaka O. Influence of reward expectation on visuospatial processing in macaque lateral prefrontal cortex. J. Neurophysiol. 2002, 87:1488-1498.
79. Leon M.I., Shadlen M.N. Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron 1999, 24:415-425.
80. Watanabe M., Sakagami M. Integration of cognitive and motivational context information in the primate prefrontal cortex. Cereb. Cortex 2007, 17(Suppl 1):i101-i109.
81. Lee D., Rushworth M.F., Walton M.E., Watanabe M., Sakagami M. Functional specialization of the primate frontal cortex during decision making. J. Neurosci. 2007, 27:8170-8173.
82. Procyk E., Goldman-Rakic P.S. Modulation of dorsolateral prefrontal delay activity during self-organized behavior. J. Neurosci. 2006, 26:11313-11323.
83. Niki H., Watanabe M. Prefrontal unit activity and delayed response: relation to cue location versus direction of response. Brain Res. 1976, 105:79-88.
84. Watanabe M. Prefrontal unit activity during associative learning in the monkey. Exp. Brain Res. 1990, 80:296-309.
85. Tsujimoto S., Sawaguchi T. Neuronal representation of response-outcome in the primate prefrontal cortex. Cereb. Cortex 2004, 14:47-55.
86. Barraclough D.J., Conroy M.L., Lee D. Prefrontal cortex and decision making in a mixed-strategy game. Nat. Neurosci. 2004, 7:404-410.
87. Seo H., Barraclough D.J., Lee D. Dynamic signals related to choices and outcomes in the dorsolateral prefrontal cortex. Cereb. Cortex 2007, 17(Suppl 1):i110-i117.
88. Pochon J.B., Levy R., Fossati P., Lehericy S., Poline J.B., Pillon B., Le Bihan, D., Dubois B. The neural system that bridges reward and cognition in humans: an fMRI study. Proc. Natl. Acad. Sci. USA, 2002, 99(8):5669-5674.
89. Ramnani N., Miall R.C. Instructed delay activity in the human prefrontal cortex is modulated by monetary reward expectation. Cereb. Cortex 2003, 13(3):318-327.
90. Sailer U., Robinson S., Fischmeister F.P., Moser E., Kryspin-Exner, I., Bauer H. Imaging the changing role of feedback during learning in decision-making. Neuroimage 2007, 37(4):1474-1486.
91. Landmann C., Dehaene S., Pappata S., Jobert A., Bottlaender M., Roumenov D., Le Bihan D. Dynamics of prefrontal and cingulate activity during a reward-based logical deduction task. Cereb. Cortex 2007, 17:749-759.
92. Amiez C., Procyk E., Honore J., Sequeira H., Joseph J.P. Reward anticipation, cognition, and electrodermal activity in the conditioned monkey. Exp. Brain Res. 2003, 149:267-275.
93. Petrides M. Frontal lobes and working memory: evidence from investigations of the effects of cortical excisions in non-human primates. Handbook of Neuropsychology 1994, 59-82. Elsevier, Amsterdam. F. Boller, J. Grafman (Eds.).
94. Petrides M. Lateral prefrontal cortex: architectonic and functional organization. Philos. Trans. R. Soc. Lond. B Biol. Sci. 2005, 360:781-795.
95. Tobler P.N., Fiorillo C.D., Schultz W. Adaptive coding of reward value by dopamine neurons. Science 2005, 307:1642-1645.
96. Padoa-Schioppa C. Orbitofrontal cortex and the computation of economic value. Ann. NY Acad. Sci. 2007, 1121:232-253.