Motivation of extended behaviors by anterior cingulate cortex

Trends in Cognitive Sciences

Volumn 16, Issue 2, 2012, Pages 122-128

  Holroyd, Clay B ^a   Yeung, Nick ^b  

^a UNIVERSITY OF VICTORIA   (Canada)

^b UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ACTION SEQUENCES; ANTERIOR CINGULATE CORTICES; COGNITIVE CONTROL; HIERARCHICAL REINFORCEMENT LEARNING; NEUROPSYCHOLOGICAL;

MOTIVATION; REINFORCEMENT;

REINFORCEMENT LEARNING;

DOPAMINE;

ANTERIOR CINGULATE; BEHAVIOR; COGNITION; EVENT RELATED POTENTIAL; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; LEARNING; MENTAL PERFORMANCE; MOTIVATION; PSYCHOLOGICAL THEORY; REINFORCEMENT; REVIEW; REWARD; WORKING MEMORY;

AKINETIC MUTISM; BEHAVIOR; DOPAMINE; GOALS; GYRUS CINGULI; HUMANS; MOTIVATION; PSYCHOMOTOR PERFORMANCE; REINFORCEMENT (PSYCHOLOGY);

EID: 84856318423     PISSN: 13646613     EISSN: 1879307X     Source Type: Journal    
DOI: 10.1016/j.tics.2011.12.008     Document Type: Review

References (70)

1. Damasio A.R., Van Hoesen G.W. Emotional disturbances associated with focal lesions of the limbic frontal lobe. Neuropsychology of Human Emotion 1983, 85-110. Guilford Press. K.M. Heilman, P. Satz (Eds.).
2. Devinsky O., et al. Contributions of anterior cingulate cortex to behaviour. Brain 1995, 118:279-306.
3. Németh G., et al. Akinetic mutism associated with bicingular lesions: clinicopathological and functional anatomical correlates. Eur. Arch. Psychiatr. Neurol. Sci. 1988, 237:218-222.
4. Vogt B.A. Regions and subregions of the cingulate cortex. Cingulate Neurobiology and Disease 2009, 3-30. Oxford University Press. B.A. Vogt (Ed.).
5. Stuss D.T., et al. Multiple frontal systems controlling response speed. Neuropsychologia 2005, 43:396-417.
6. Kouneiher F., et al. Motivation and cognitive control in the human prefrontal cortex. Nat. Neurosci. 2009, 12:939-945.
7. Mulert C., et al. Evidence for a close relationship between conscious effort and anterior cingulate cortex activity. Int. J. Psychophysiol. 2005, 56:65-80.
8. Naccache L., et al. Effortless control: Executive attention and conscious feeling of mental effort are dissociable. Neuropsychologia 2005, 43:1318-1328.
9. Croxson P.L., et al. Effort-based cost-benefit valuation and the human brain. J. Neurosci. 2009, 29:4531-4541.
10. Walton M.E., et al. Weighing up the benefits of work: behavioral and neural analyses of effort-related decision making. Neural Netw. 2006, 19:1302-1314.
11. Critchley H.D., et al. Anterior cingulate activity during error and autonomic response. NeuroImage 2005, 27:885-895.
12. Mueller V.A., et al. The role of the preSMA and the rostral cingulate zone in internally selected actions. NeuroImage 2007, 37:1354-1361.
13. Kerns J.G., et al. Anterior cingulate conflict monitoring and adjustments in control. Science 2004, 303:1023-1026.
14. Yeung, N. Conflict monitoring and cognitive control. In: Oxford Handbook of Cognitive Neuroscience (Ochsner, K. and Kosslyn, S., eds), Oxford University Press (in press).
15. Holroyd C.B., Coles M.G.H. The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol. Rev. 2002, 109:679-709.
16. Rushworth M.F.S., et al. Contrasting roles for cingulate and orbitofrontal cortex in decisions and social behaviour. Trends Cogn. Sci. 2007, 11:168-176.
17. Botvinick M.M. Conflict monitoring and decision making: reconciling two perspectives on anterior cingulate function. Cogn. Affect. Behav. Neurosci. 2007, 7:356-366.
18. Nachev P. The blind executive. NeuroImage 2011, 57:312-313.
19. Amiez C., et al. Reward encoding in the monkey anterior cingulate cortex. Cereb. Cortex 2006, 16:1040-1055.
20. Cohen R.A., et al. Impairments of attention after cingulotomy. Neurology 1999, 53:819-824.
21. Kennerley S.W., et al. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 2006, 9:940-947.
22. Pears A., et al. Lesions of the orbitofrontal but not medial prefrontal cortex disrupt conditioned reinforcement in primates. J. Neurosci. 2003, 23:11189-11201.
23. Rushworth M.F.S., et al. The effect of cingulate cortex lesions on task switching and working memory. J. Cogn. Neurosci. 2003, 15:338-353.
24. Holroyd C.B., Yeung N. An integrative theory of anterior cingulate cortex function: option selection in hierarchical reinforcement learning. Neural Basis of Motivational and Cognitive Control 2011, 333-349. MIT Press. R.B. Mars (Ed.).
25. Botvinick M.M., et al. Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective. Cognition 2009, 113:262-280.
26. Sakai K. Task set and prefrontal cortex. Annu. Rev. Neurosci. 2008, 31:219-245.
27. Niv Y. Reinforcement learning in the brain. J. Math. Psychol. 2009, 53:139-154.
28. Cohen M.X., Frank M.J. Neurocomputational models of basal ganglia function in learning, memory and choice. Behav. Brain Res. 2009, 199:141-156.
29. O'Doherty J., et al. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 2004, 304:452-454.
30. Gläscher J., et al. Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. Cereb. Cortex 2009, 19:483-495.
31. Tremblay L., Schultz W. Relative reward preference in primate orbitofrontal cortex. Nature 1999, 398:704-708.
32. Schultz W. Multiple dopamine functions at different time courses. Annu. Rev. Neurosci. 2007, 30:259-288.
33. McClure S.M., et al. A computational substrate for incentive salience. Trends Neurosci. 2003, 26:423-428.
34. Niv Y., et al. A normative perspective on motivation. Trends Cogn. Sci. 2006, 10:375-381.
35. Denk F., et al. Differential involvement of serotonin and dopamine systems in cost-benefit decisions about delay or effort. Psychopharmacology 2005, 179:587-596.
36. Schweimer J., et al. Involvement of catecholamine neurotransmission in the rat anterior cingulate in effort-related decision making. Behav. Neurosci. 2005, 119:1687-1692.
37. Schweimer J., Hauber W. Dopamine D1 receptors in the anterior cingulate cortex regulate effort-based decision making. Learn. Mem. 2006, 13:777-782.
38. Assadi S.M., et al. Dopamine modulates neural networks involved in effort-based decision-making. Neurosci. Biobehav. Rev. 2009, 33:383-393.
39. Doya K. Modulators of decision making. Nat. Neurosci. 2008, 11:410-416.
40. Niv Y. Cost, benefit, tonic, phasic: what do response rates tell us about dopamine and motivation?. Ann. N. Y. Acad. Sci. 2007, 1104:357-376.
41. Durstewitz D., Seamans J.K. The dual-state theory of prefrontal cortex dopamine function with relevance to catechol-O-methyltransferase genotypes and schizophrenia. Biol. Psychiatry 2008, 64:739-749.
42. O'Reilly R.C. Biologically based computational models of high-level cognition. Science 2006, 314:91-94.
43. Alexander W.H., Brown J.W. Medial prefrontal cortex as an action-outcome predictor. Nat. Neurosci. 2011, 14:1338-1344.
44. Yeung N., et al. ERP correlates of feedback and reward processing in the presence and absence of response choice. Cereb. Cortex 2005, 15:535-544.
45. Coulthard E.J., et al. Control over conflict during movement preparation: role of posterior parietal cortex. Neuron 2008, 58:144-157.
46. Forstmann B.U., et al. Voluntary selection of task sets revealed by functional magnetic resonance imaging. J. Cogn. Neurosci. 2006, 18:388-398.
47. Johnston K., et al. Top-down control-signal dynamics in anterior cingulate and prefrontal cortex neurons following task switching. Neuron 2007, 53:453-462.
48. Rushworth M.F.S., et al. Role of the human medial frontal cortex in task switching: a combined fMRI and TMS study. J. Neurophysiol. 2002, 87:2577-2592.
49. Dosenbach N.U.F., et al. A dual-networks architecture of top-down control. Trends Cogn. Sci. 2008, 12:99-105.
50. Balaguer-Ballester E., et al. Attracting dynamics of frontal cortex ensembles during memory-guided decision-making. PLoS Comput. Biol. 2011, 7:e1002057.
51. Hoshi E., et al. Neurons in the rostral cingulate motor area monitor multiple phases of visuomotor behavior with modest parametric selectivity. J. Neurophysiol. 2005, 94:640-656.
52. Shidara M., Richmond B.J. Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science 2002, 296:1709-1711.
53. Duncan J. The multiple-demand (MD) system of the primate brain: mental programs for intelligent behaviour. Trends Cogn. Sci. 2010, 14:172-179.
54. Ribas-Fernandes J.J.F. A neural signature of hierarchical reinforcement learning. Neuron 2011, 71:370-379.
55. Botvinick M.M. Hierarchical models of behavior and prefrontal function. Trends Cogn. Sci. 2008, 12:201-208.
56. Durstewitz D., et al. Abrupt transitions between prefrontal neural ensemble states accompany behavioral transitions during rule learning. Neuron 2010, 66:438-448.
57. Haynes J.-D., et al. Reading hidden intentions in the human brain. Curr. Biol. 2007, 17:323-328.
58. Shen W., et al. Dichotomous dopaminergic control of striatal synaptic plasticity. Science 2008, 321:848-851.
59. Yin H.H., et al. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 2004, 19:181-189.
60. Seo H., Lee D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 2007, 27:8366-8377.
61. Jocham G., Ullsperger M. Neuropharmacology of performance monitoring. Neurosci. Biobehav. Rev. 2009, 33:48-60.
62. Cohen J.D., et al. Computational perspectives on dopamine function in prefrontal cortex. Curr. Opin. Neurobiol. 2002, 12:223-229.
63. Gorelova, N. et al. (in press) The glutamatergic component of the mesocortical pathway emanating from different subregions of the ventral midbrain. Cereb. Cortex doi:10.1093/cercor/bhr107.
64. Holroyd C.B., et al. The feedback correct-related positivity: sensitivity of the event-related brain potential to unexpected positive feedback. Psychophysiology 2008, 45:688-697.
65. Carlson J.M., et al. Ventral striatal and medial prefrontal BOLD activation is correlated with reward-related electrocortical activity: a combined ERP and fMRI study. NeuroImage 2011, 57:1608-1616.
66. Marco-Pallarés J., et al. Learning by doing: an fMRI study of feedback-related brain activations. Neuroreport 2007, 18:1423-1426.
67. Zaghloul K.A., et al. Human substantia nigra neurons encode unexpected financial rewards. Science 2009, 323:1496-1499.
68. Overbeek T.J.M., et al. Dissociable components of error processing: on the functional significance of the Pe vis-à-vis the ERN/Ne. J. Psychophysiol. 2005, 19:319-329.
69. Emeric E.E., et al. Performance monitoring local field potentials in the medial frontal cortex of primates: anterior cingulate cortex. J. Neurophysiol. 2008, 99:759-772.
70. Vezoli J., Procyk E. Frontal feedback-related potentials in nonhuman primates: modulation during learning and under haloperidol. J. Neurosci. 2009, 29:15675-15683.