Dorsal anterior cingulate cortex integrates reinforcement history to guide voluntary behavior

Cortex

Volumn 44, Issue 5, 2008, Pages 548-559

  Holroyd, Clay B ^a   Coles, Michael G H ^b  

^a UNIVERSITY OF VICTORIA   (Canada)

^b RADBOUD UNIVERSITY NIJMEGEN   (Netherlands)

Author keywords

Anterior cingulate cortex; Decision making; Error related negativity; Prediction error; Reinforcement

Indexed keywords

DOPAMINE;

ADULT; ANALYTICAL ERROR; ARTICLE; ASSOCIATION; BEHAVIOR; BRAIN FUNCTION; CINGULATE GYRUS; CONFLICT; CONTROLLED STUDY; DECISION MAKING; DOPAMINERGIC SYSTEM; ELECTROENCEPHALOGRAPHY; EVENT RELATED POTENTIAL; FEEDBACK SYSTEM; HUMAN; HUMAN EXPERIMENT; MATHEMATICAL MODEL; MESENCEPHALON; MOTOR CONTROL; MOTOR SYSTEM; NORMAL HUMAN; PREDICTION; PRINCIPAL COMPONENT ANALYSIS; PROCESS MODEL; REINFORCEMENT; REWARD; SIGNAL PROCESSING; STIMULUS RESPONSE; TASK PERFORMANCE; THEORY; VOLUNTARY MOVEMENT; VOLUNTEER;

EID: 44249114297     PISSN: 00109452     EISSN: 00109452     Source Type: Journal    
DOI: 10.1016/j.cortex.2007.08.013     Document Type: Article

References (67)

1. C. Amiez J.-P. Joseph E. Procyk Anterior cingulate error related activity is modulated by predicted reward European Journal of Neuroscience 21 2005 3447 3452
2. T.E. Baker C.B. Holroyd Which way do I go? Neural activation in response to feedback processing and decision making in a virtual T-maze task Psychophysiology 43 2006 S22
3. M.M. Botvinick T.S. Braver D.M. Barch C.S. Carter J.D. Cohen Conflict monitoring and cognitive control Psychological Review 108 2001 624 652
4. M.M. Botvinick J.D. Cohen C.S. Carter Conflict monitoring and anterior cingulate cortex: an update Trends in Cognitive Sciences 8 2004 539 546
5. A.C. Catania Reinforcement schedules: the role of responses preceding the one that produces the reinforcer Journal of the Experimental Analysis of Behavior 15 1971 271 287
6. M.X. Cohen C. Ranganath Reinforcement learning signals predict future decisions Journal of Neuroscience 27 2007 371 378
7. M.G.H. Coles M.K. Scheffers C. Holroyd Berger's dream? The error-related negativity and modern cognitive psychophysiology H. Witte U. Zwiener B. Schack A. D`ring Quantitative and Topological EEG and MEG Analysis 1998 Druckhaus Mayer Verlag Jena-Erlangen 96 102
8. S. Debener M. Ullsperger M. Siegel K. Fiehler D.Y. von Cramon A.K. Engel Trial-by-trial coupling of concurrent electroencephalogram and functional magnetic resonance imaging identifies the dynamics of performance monitoring The Journal of Neuroscience 25 2005 11730 11737
9. S. Dehaene M.I. Posner D.M. Tucker Localization of a neural system for error detection and compensation Psychological Science 5 1994 303 305
10. E. Donchin E.F. Heffley III Multivariate analysis of event-related potential data: A tutorial review D. Otto Multidisciplinary Perspectives in Event-related Potential Research 1978 US Government Printing Office Washington, DC 555 572 EPA-600/9-77-043
11. F.C.L. Donkers G.J.M. van Boxtel Mediofrontal negativities to averted gains and losses in the slot-machine task: a further investigation Journal of Psychophysiology 19 2005 256 262
12. D.M. Egelman C. Person P.R. Montague A computational role for dopamine delivery in human decision-making Journal of Cognitive Neuroscience 10 1998 623 630
13. M. Falkenstein J. Hohnsbein J. Hoormann L. Blanke Effects of errors in choice reaction tasks on the ERP under focused and divided attention C. Brunia A. Gaillard A. Kok Psychophysiological Brain Research 1990 Tilburg University Press Tilburg, The Netherlands 192 195
14. M.J. Frank B.S. Woroch T. Curran Error-related negativity predicts reinforcement learning and conflict biases Neuron 47 2005 495 501
15. W.J. Gehring B. Goss M.G.H. Coles D.E. Meyer E. Donchin A neural system for error detection and compensation Psychological Science 4 1993 385 390
16. W.J. Gehring A.R. Willoughby The medial frontal cortex and the rapid processing of monetary gains and losses Science 295 2002 2279 2282
17. W.J. Gehring A.R. Willoughby Response Science 296 2002 1611
18. J. Gibson O.E. Krigolson C.B. Holroyd Sensitivity of the feedback error-related negativity to reward probability Psychophysiology 43 2006 S41 S42
19. G. Gratton M.G.H. Coles E. Donchin A new method for off-line removal of ocular artifact Electroencephalography and Clinical Neurophysiology 55 1983 468 484
20. G. Hajcak J. Moser C.B. Holroyd R.F. Simons The feedback-related negativity reflects the binary evaluation of good versus bad outcomes Biological Psychology 71 2006 148 154
21. R.J. Herrnstein Experiments on stable suboptimality in individual behavior R.J. Herrnstein The Matching Law 1997 Russell Sage Foundation New York 266 273
22. R.J. Herrnstein Melioration: a theory of distributed choice R.J. Herrnstein The Matching Law 1997 Russell Sage Foundation New York 274 292
23. J. Hewig R. Trippe H. Hecht M.G.H. Coles C.B. Holroyd W.H.R. Miltner An electrophysiological analysis of decision-making in blackjack Cerebral Cortex 17 2007 865 877
24. C.B. Holroyd A note on the Oddball N200 and the feedback ERN M. Ullsperger M. Falkenstein Errors, Conflicts, and the Brain: Current Opinions on Performance Monitoring 2004 MPI of Cognitive Neuroscience Leipzig 211 218
25. C.B. Holroyd M.G.H. Coles The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity Psychological Review 109 2002 679 709
26. C.B. Holroyd O.E. Krigolson Reward prediction error signals associated with a modified time estimation task Psychophysiology 44 2007 913 917
27. C.B. Holroyd N. Yeung Alcohol and error processing Trends in Neurosciences 26 2003 402 404
28. C.B. Holroyd J. Dien M.G.H. Coles Error-related scalp potentials elicited by hand and foot movements: evidence for an output-independent error-processing system in humans Neuroscience Letters 242 1998 65 68
29. C.B. Holroyd M.G.H. Coles S. Nieuwenhuis Medial prefrontal cortex and error potentials Science 296 2002 1610 1611
30. C.B. Holroyd S. Nieuwenhuis N. Yeung J.D. Cohen Errors in reward prediction are reflected in the event-related brain potential Neuroreport 14 2003 2481 2484
31. C.B. Holroyd J.T. Larsen J.D. Cohen Context dependence of the event-related brain potential associated with reward and punishment Psychophysiology 41 2004 245 253
32. C.B. Holroyd S. Nieuwenhuis R. Mars M.G.H. Coles Anterior cingulate cortex, selection for action, and error processing M. Posner Cognitive Neuroscience of Attention 2004 Guilford Publishing, Inc. New York 219 231
33. C.B. Holroyd S. Nieuwenhuis N. Yeung L. Nystrom R. Mars M.G.H. Coles J.D. Cohen Dorsal anterior cingulate cortex shows fMRI response to internal and external error signals Nature Neuroscience 7 2004 497 498
34. C.B. Holroyd N. Yeung M.G.H. Coles J.D. Cohen A mechanism for error detection in speeded response time tasks Journal of Experimental Psychology: General 134 2005 163 191
35. C.B. Holroyd G. Hajcak J.T. Larsen The good, the bad and the neutral: electrophysiological responses to feedback stimuli Brain Research 1105 2006 93 101
36. S.W. Kennerley M.E. Walton T.E.J. Behrens M.J. Buckley M.F.S. Rushworth Optimal decision making and the anterior cingulate cortex Nature Neuroscience 9 2006 940 947
37. O.E. Krigolson C.B. Holroyd Evidence for hierarchical error processing in the human brain Neuroscience 137 2006 13 17
38. O.E. Krigolson C.B. Holroyd Hierarchical error processing: different errors, different systems Brain Research 1155 2007 70 80
39. O.E. Krigolson C.B. Holroyd Predictive information and error processing: the role of medial–frontal cortex during motor control Psychophysiology 44 2007 586 595
40. B. Lau P.W. Glimcher Dynamic response-by-response models of matching behavior in rhesus monkeys Journal of the Experimental Analysis of Behavior 84 2005 555 579
41. R.B. Mars M.G.H. Coles M.J. Grol C.B. Holroyd S. Nieuwenhuis W. Hulstijn I. Toni Neural dynamics of error processing in medial frontal cortex Neuroimage 28 2005 1007 1013
42. M. Matsumoto K. Matsumoto H. Abe K. Tanaka Medial prefrontal cell activity signaling prediction errors of action values Nature Neuroscience 10 2007 647 656
43. W.H.R. Miltner C.H. Braun M.G.H. Coles Event-related brain potentials following incorrect feedback in a time-estimation task: evidence for a “generic” neural system for error detection Journal of Cognitive Neuroscience 9 1997 788 798
44. P.R. Montague G.S. Berns Neural economics and the biological substrates of valuation Neuron 36 2002 265 284
45. P.R. Montague S.E. Hyman J.D. Cohen Computational roles for dopamine in behavioural control Nature 431 2004 760 767
46. S. Nieuwenhuis K.R. Ridderinkhof D. Talsma M.G.H. Coles C.B. Holroyd A. Kok M.W. Van der Molen A computational account of altered error processing in older age: dopamine and the error-related negativity Cognitive, Affective, and Behavioral Neuroscience 2 2002 19 36
47. S. Nieuwenhuis C.B. Holroyd N. Mol M.G.H. Coles Reinforcement-related brain potentials from medial frontal cortex: origins and functional significance Neuroscience and Biobehavioral Reviews 28 2004 441 448
48. S. Nieuwenhuis N. Yeung C.B. Holroyd A. Schurger J.D. Cohen Sensitivity of electrophysiological activity from medial frontal cortex to utilitarian and performance feedback Cerebral Cortex 14 2004 741 747
49. S. Nieuwenhuis H.A. Slagter N.J.A. von Geusau D. Heslenfeld C.B. Holroyd Knowing good from bad: differential activation of human cortical areas by positive and negative outcomes European Journal of Neuroscience 21 2005 3161 3168
50. K. Pakzad-Vaezi O.E. Krigolson C.B. Holroyd Underlying unity: the relationship between the N200 and the feedback ERN Psychophysiology 43 2006 S75
51. T. Paus Primate anterior cingulate cortex: where motor control, drive, and cognition interface Nature Reviews Neuroscience 2 2001 417 424
52. G.F. Potts L.E. Martin P. Burton P.R. Montague When things are better or worse than expected: the medial frontal cortex and the allocation of processing resources Journal of Cognitive Neuroscience 18 2006 1112 1119
53. K.R. Ridderinkhof M. Ullsperger E.A. Crone S. Nieuwenhuis The role of the medial frontal cortex in cognitive control Science 306 2004 443 447
54. M.F.S. Rushworth M.E. Walton S.W. Kennerley D.M. Bannerman Action sets and decision in the medial frontal cortex Trends in Cognitive Sciences 8 2004 410 417
55. A.G. Sanfey G. Lowenstein S.M. McClure J.D. Cohen Neuroeconomics: cross-currents in research on decision making Trends in Cognitive Sciences 10 2006 108 116
56. W. Schultz Getting formal with dopamine and reward Neuron 36 2002 241 263
57. K.M. Spencer J. Dien E. Donchin Spatiotemporal analysis of the late ERP responses to deviant stimuli Psychophysiology 38 2001 343 358
58. R.S. Sutton Learning to predict by the methods of temporal differences Machine Learning 3 1988 9 44
59. R.S. Sutton A.G. Barto Reinforcement learning: an introduction 1998 MIT Press Cambridge, MA
60. A. Toyomaki H. Murohashi Discrepancy between feedback negativity and subjective evaluation in gambling Neuroreport 16 2005 1865 1868
61. D.M. Tucker Spatial sampling of head electrical fields: the geodesic sensor net Electroencephalography and Clinical Neurophysiology 87 1993 154 163
62. D.M. Tucker A. Hartry-Speiser L. McDougal P. Luu D. deGrandpre Mood and spatial memory: emotion and right hemisphere contributions to spatial cognition Biological Psychology 50 1999 103 125
63. M. Ullsperger D.Y. von Cramon Error monitoring using external feedback: specific roles of the habenular complex, the reward system, and the cingulate motor area revealed by functional resonance imaging The Journal of Neuroscience 23 2003 4308 4314
64. K.G. Volz R.I. Schubotz D.Y. von Cramon Frontomedian activation depends on both feedback validity and valence: fMRI evidence for contextual feedback evaluation Neuroimage 27 2005 564 571
65. M.E. Walton J.T. Devlin M.F.S. Rushworth Interactions between decision making and performance monitoring within prefrontal cortex Nature Neuroscience 7 2004 1259 1265
66. N. Yeung M. Botvinick J.D. Cohen The neural basis of error detection: conflict monitoring and the error-related negativity Psychological Review 111 2004 931 959
67. N. Yeung A.G. Sanfey Independent coding of reward magnitude and valence in the human brain Journal of Neuroscience 24 2004 6258 6264