Adaptive coding of action values in the human rostral cingulate zone

Journal of Neuroscience

Volumn 29, Issue 23, 2009, Pages 7489-7496

  Jocham, Gerhard ^a   Neumann, Jane ^b   Klein, Tilmann A ^a   Danielmeier, Claudia ^a   Ullsperger, Markus ^a  

^a MAX PLANCK INSTITUTE FOR NEUROLOGICAL RESEARCH   (Germany)

^b MAX PLANCK INSTITUTE FOR HUMAN COGNITIVE AND BRAIN SCIENCES   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE BEHAVIOR; ADULT; ARTICLE; CINGULATE GYRUS; CONTROLLED STUDY; ELECTROENCEPHALOGRAM; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; IMAGE ANALYSIS; LEARNING; MALE; NEGATIVE FEEDBACK; NORMAL HUMAN; PRIORITY JOURNAL; PROBABILITY; REINFORCEMENT; REWARD; TASK PERFORMANCE;

ADAPTATION, PSYCHOLOGICAL; ADULT; ALGORITHMS; ANALYSIS OF VARIANCE; FEMALE; GYRUS CINGULI; HUMANS; KNOWLEDGE OF RESULTS (PSYCHOLOGY); MAGNETIC RESONANCE IMAGING; MALE; PROBABILITY; REVERSAL LEARNING; REWARD; TIME FACTORS; YOUNG ADULT;

EID: 67049160586     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.0349-09.2009     Document Type: Article

References (45)

1. Abler B, Walter H, Erk S, Kammerer H, Spitzer M (2006) Prediction error as a linear function of reward probability is coded in human nucleus accumbens. Neuroimage 31:790-795. (Pubitemid 43776671)
2. Amiez C, Joseph JP, Procyk E (2006) Reward encoding in the monkey anterior cingulate cortex. Cereb Cortex 16:1040-1055. (Pubitemid 43992453)
3. Barbas H, De Olmos J (1990) Projections from the amygdala to basoventral and mediodorsal prefrontal regions in the rhesus monkey. J Comp Neurol 300:549-571. (Pubitemid 20368494)
4. Bates JF, Goldman-Rakic PS (1993) Prefrontal connections of medial motor areas in the rhesus monkey. J Comp Neurol 336:211-228. (Pubitemid 23289660)
5. Behrens TE, Woolrich MW, Walton ME, Rushworth MF (2007) Learning the value of information in an uncertain world. Nat Neurosci 10:1214-1221. (Pubitemid 47339952)
6. Budhani S, Marsh AA, Pine DS, Blair RJ (2007) Neural correlates of response reversal: considering acquisition. Neuroimage 34:1754-1765. (Pubitemid 46201904)
7. Cohen MX, Krohn-Grimberghe A, Elger CE, Weber B (2007) Dopamine gene predicts the brain's response to dopaminergic drug. Eur J Neurosci 26:3652-3660. (Pubitemid 350243518)
8. Cools R, Clark L, Owen AM, Robbins TW (2002) Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J Neurosci 22:4563-4567.
9. Frank MJ, Moustafa AA, Haughey HM, Curran T, Hutchison KE (2007) Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc Natl Acad Sci U S A 104:16311-16316. (Pubitemid 350099356)
10. Gehring WJ, Willoughby AR (2002) The medial frontal cortex and the rapid processing of monetary gains and losses. Science 295:2279-2282. (Pubitemid 34258845)
11. Hatanaka N, Tokuno H, Hamada I, Inase M, Ito Y, Imanishi M, Hasegawa N, Akazawa T, Nambu A, TakadaM (2003) Thalamocortical and intracortical connections of monkey cingulate motor areas. J Comp Neurol 462:121-138.
12. He SQ, Dum RP, Strick PL (1995) Topographic organization of corticospinal projections from the frontal lobe: motor areas on the medial surface of the hemisphere. J Neurosci 15:3284-3306.
13. Jenkinson M, Smith S (2001) A global optimisation method for robust affine registration of brain images. Med Image Anal 5:143-156. (Pubitemid 35282800)
14. Jenkinson M, Bannister P, Brady M, Smith S (2002) Improved optimization for the robust and accurate linear registration and motion correction of brain images. Neuroimage 17:825-841. (Pubitemid 35317016)
15. Jocham G, Klein TA, Neumann J, von Cramon DY, Reuter M, Ullsperger M (2009) Dopamine DRD2 polymorphism alters reversal learning and associated neural activity. J Neurosci 29:3695-3704.
16. Kennerley SW, Walton ME, Behrens TE, Buckley MJ, Rushworth MF (2006) Optimal decision making and the anterior cingulate cortex. Nat Neurosci 9:940-947. (Pubitemid 43980509)
17. Kerns JG, Cohen JD, MacDonald AW 3rd, Cho RY, Stenger VA, Carter CS (2004) Anterior cingulate conflict monitoring and adjustments in control. Science 303:1023-1026. (Pubitemid 38209708)
18. Klein TA, Neumann J, Reuter M, Hennig J, von Cramon DY, Ullsperger M (2007) Genetically determined differences in learning from errors. Science 318:1642-1645. (Pubitemid 350233666)
19. Kringelbach ML, Rolls ET (2003) Neural correlates of rapid reversal learning in a simple model of human social interaction. Neuroimage 20:1371-1383. (Pubitemid 37268154)
20. Matsumoto K, Suzuki W, Tanaka K (2003) Neuronal correlates of goal-based motor selection in the prefrontal cortex. Science 301:229-232. (Pubitemid 36858478)
21. Matsumoto M, Matsumoto K, Abe H, Tanaka K (2007) Medial prefrontal cell activity signaling prediction errors of action values. Nat Neurosci 10:647-656. (Pubitemid 46652446)
22. McClure SM, Berns GS, Montague PR (2003) Temporal prediction errors in a passive learning task activate human striatum. Neuron 38:339-346. (Pubitemid 36555658)
23. Menon M, Jensen J, Vitcu I, Graff-Guerrero A, Crawley A, Smith MA, Kapur S (2007) Temporal difference modeling of the blood-oxygen level dependent response during aversive conditioning in humans: effects of dopaminergic modulation. Biol Psychiatry 62:765-772. (Pubitemid 47381433)
24. Mitchell DG, Rhodes RA, Pine DS, Blair RJ (2008) The contribution of ventrolateral and dorsolateral prefrontal cortex to response reversal. Behav Brain Res 187:80-87. (Pubitemid 350225613)
25. Mitchell T (1997) Machine learning. New York: McGraw-Hill.
26. Morecraft RJ, Van Hoesen GW (1992) Cingulate input to the primary and supplementary motor cortices in the rhesus monkey: evidence for somatotopy in areas 24c and 23c. J Comp Neurol 322:471-489.
27. O'Doherty JP (2004) Reward representations and reward-related learning in the human brain: insights from neuroimaging. Curr Opin Neurobiol 14:769-776. (Pubitemid 39576211)
28. O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304:452-454. (Pubitemid 38495943)
29. Ongür D, Price JL (2000) The organization of networks within the orbital and medial prefrontal cortex of rats, monkeys and humans. Cereb Cortex 10:206-219. (Pubitemid 30159289)
30. Picard N, Strick PL (1996) Motor areas of the medial wall: a review of their location and functional activation. Cereb Cortex 6:342-353. (Pubitemid 26161547)
31. Ramnani N, Elliott R, Athwal BS, Passingham RE (2004) Prediction error for free monetary reward in the human prefrontal cortex. Neuroimage 23:777-786. (Pubitemid 39446513)
32. Ridderinkhof KR, Ullsperger M, Crone EA, Nieuwenhuis S (2004) The role of the medial frontal cortex in cognitive control. Science 306:443-447. (Pubitemid 39372436)
33. Shima K, Tanji J (1998) Role for cingulate motor area cells in voluntary movement selection based on reward. Science 282:1335-1338. (Pubitemid 28524499)
34. Smith SM, Jenkinson M, Woolrich MW, Beckmann CF, Behrens TEJ, Johansen-Berg H, Bannister PR, De Luca M, Drobnjak I, Flitney DE, Niazy RK, Saunders J, Vickers J, Zhang Y, De Stefano N, Brady JM, Matthews PM (2004) Advances in functional and structuralMRimage analysis and implementation as FSL. Neuroimage 23 [Suppl 1]:S208-S219.
35. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. Cambridge, MA: MIT.
36. Swick D, Turken AU (2002) Dissociation between conflict detection and error monitoring in the human anterior cingulate cortex. Proc Natl Acad Sci U S A 99:16354-16359. (Pubitemid 35462799)
37. Takada M, Tokuno H, Hamada I, Inase M, Ito Y, Imanishi M, Hasegawa N, Akazawa T, Hatanaka N, Nambu A (2001) Organization of inputs from cingulate motor areas to basal ganglia in macaque monkey. Eur J Neurosci 14:1633-1650. (Pubitemid 35478032)
38. Ullsperger M, von Cramon DY (2001) Subprocesses of performance monitoring: a dissociation of error processing and response competition revealed by event-related fMRI and ERPs. Neuroimage 14:1387-1401. (Pubitemid 33124186)
39. Ullsperger M, von Cramon DY (2003) Error monitoring using external feedback: specific roles of the habenular complex, the reward system, and the cingulate motor area revealed by functional magnetic resonance imaging. J Neurosci 23:4308-4314. (Pubitemid 36958451)
40. Ullsperger M, von Cramon DY (2004) Neuroimaging of performance monitoring: error detection and beyond. Cortex 40:593-604. (Pubitemid 39303089)
41. Walton ME, Devlin JT, Rushworth MF (2004) Interactions between decision making and performance monitoring within prefrontal cortex. Nat Neurosci 7:1259-1265. (Pubitemid 39426246)
42. Watkins CJCH, Dayan P (1992) Q-learning. Machine Learning 8:279-292.
43. Williams ZM, Bush G, Rauch SL, Cosgrove GR, Eskandar EN (2004) Human anterior cingulate neurons and the integration of monetary reward with motor responses. Nat Neurosci 7:1370-1375. (Pubitemid 41057760)
44. Woolrich MW, Ripley BD, Brady M, Smith SM (2001) Temporal autocorrelation in univariate linear modeling of FMRI data. Neuroimage 14:1370-1386. (Pubitemid 33124185)
45. Woolrich MW, Behrens TE, Beckmann CF, Jenkinson M, Smith SM (2004) Multilevel linear modelling for FMRI group analysis using Bayesian inference. Neuroimage 21:1732-1747. (Pubitemid 38446594)