Human reversal learning under conditions of certain versus uncertain outcomes

NeuroImage

Volumn 56, Issue 1, 2011, Pages 315-322

  D'Cruz, Anna Maria ^a,b   Ragozzino, Michael E ^a,b   Mosconi, Matthew W ^a   Pavuluri, Mani N ^a,c   Sweeney, John A ^a,b  

^a Center for Cognitive Medicine   (United States)

^b Department of Psychology ^*  (United States)

^c Pediatric Brain Research and Intervention Center   (United States)

Author keywords

Cognitive flexibility; FMRI; Human; Reinforcement learning; Reversal learning; Reward processing

Indexed keywords

ADULT; ANTERIOR CINGULATE; ARTICLE; BEHAVIOR; BRAIN FUNCTION; BRAIN REGION; COGNITION; CONTROLLED STUDY; CORPUS STRIATUM; DECISION MAKING; FACILITATION; FEEDBACK SYSTEM; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; IMAGE ANALYSIS; INSULA; LEARNING; MALE; MESENCEPHALON; MOTIVATION; MULTIPLE CHOICE TEST; NEGATIVE FEEDBACK; NORMAL HUMAN; POSTERIOR PARIETAL CORTEX; PREFRONTAL CORTEX; PREMOTOR CORTEX; PRIORITY JOURNAL; REINFORCEMENT; REWARD; TASK PERFORMANCE;

ADULT; BRAIN; BRAIN MAPPING; CHOICE BEHAVIOR; FEMALE; HUMANS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; MALE; REVERSAL LEARNING; REWARD;

EID: 79953049199     PISSN: 10538119     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2011.01.068     Document Type: Article

References (55)

1. Alexander W.H., Brown J.W. Competition between learned reward and error outcome predictions in anterior cingulate cortex. Neuroimage 2010, 49(4):3210-3218. 10.1016/j.neuroimage.2009.11.065.
2. Aron A.R., Shohamy D., Clark J., Myers C., Gluck M.A., Poldrack R.A. Human midbrain sensitivity to cognitive feedback and uncertainty during classification learning. J. Neurophysiol. 2004, 92(2):1144-1152. 10.1152/jn.01209.2003.
3. Baker T.E., Holroyd C.B. Which way do I go? Neural activation in response to feedback and spatial processing in a virtual T-maze. Cereb. Cortex (New York N. Y.) 2009, 19(8):1708-1722. 10.1093/cercor/bhn223.
4. Balleine B.W., Delgado M.R., Hikosaka O. The role of the dorsal striatum in reward and decision-making. J. Neurosci. 2007, 27(31):8161-8165. 10.1523/JNEUROSCI.1554-07.2007.
5. Boulougouris V., Robbins T.W. Pre-surgical training ameliorates orbitofrontal-mediated impairments in spatial reversal learning. Behav. Brain Res. 2009, 197(2):469-475. 10.1016/j.bbr.2008.10.005.
6. Boulougouris V., Glennon J.C., Robbins T.W. Dissociable effects of selective 5-HT2A and 5-HT2C receptor antagonists on serial spatial reversal learning in rats. Neuropsychopharmacol. Off. Publ. Am. Coll. Neuropsychopharmacol. 2008, 33(8):2007-2019. 10.1038/sj.npp. 1301584.
7. Budhani S., Marsh A.A., Pine D.S., Blair R.J.R. Neural correlates of response reversal: considering acquisition. Neuroimage 2007, 34(4):1754-1765. 10.1016/j.neuroimage.2006.08.060.
8. Cavanna A.E., Trimble M.R. The precuneus: a review of its functional anatomy and behavioural correlates. Brain 2006, 129(3):564-583. 10.1093/brain/awl004.
9. Clarke H.F., Robbins T.W., Roberts A.C. Lesions of the medial striatum in monkeys produce perseverative impairments during reversal learning similar to those produced by lesions of the orbitofrontal cortex. J. Neurosci. 2008, 28(43):10972-10982. 10.1523/JNEUROSCI.1521-08.2008.
10. Cools R., Lewis S.J., Clark L., Barker R.A., Robbins T.W. L-DOPA disrupts activity in the nucleus accumbens during reversal learning in Parkinson's disease. Neuropsychopharmacol. Off. Publ. Am. Coll. Neuropsychopharmacol. 2007, 32(1):180-189. 10.1038/sj.npp. 1301153.
11. Cools R., Frank M.J., Gibbs S.E., Miyakawa A., Jagust W., D'Esposito M. Striatal dopamine predicts outcome-specific reversal learning and its sensitivity to dopaminergic drug administration. J. Neurosci. 2009, 29(5):1538-1543. 10.1523/JNEUROSCI.4467-08.2009.
12. Dias R., Robbins T.W., Roberts A.C. Primate analogue of the Wisconsin card sorting test: effects of excitotoxic lesions of the prefrontal cortex in the marmoset. Behav. Neurosci. 1996, 110(5):872-886.
13. Dong H., Csernansky C.A., Martin M.V., Bertchume A., Vallera D., Csernansky J.G. Acetylcholinesterase inhibitors ameliorate behavioral deficits in the Tg2576 mouse model of Alzheimer's disease. Psychopharmacology 2005, 181(1):145-152. 10.1007/s00213-005-2230-6.
14. Egner T., Monti J.M.P., Trittschuh E.H., Wieneke C.A., Hirsch J., Mesulam M. Neural integration of top-down spatial and feature-based information in visual search. J. Neurosci. 2008, 28(24):6141-6151. 10.1523/JNEUROSCI.1262-08.2008.
15. Evers E.A., Cools R., Clark L., van der Veen F.M., Jolles J., Sahakian B.J., et al. Serotonergic modulation of prefrontal cortex during negative feedback in probabilistic reversal learning. Neuropsychopharmacol. Off. Publ. Am. Coll. Neuropsychopharmacol. 2005, 30(6):1138-1147. 10.1038/sj.npp. 1300663.
16. Ferry A.T., Lu X.C., Price J.L. Effects of excitotoxic lesions in the ventral striatopallidal-thalamocortical pathway on odor reversal learning: inability to extinguish an incorrect response. Exp. Brain Research. Exp. Hirnforschung. Experimentation Cerebrale 2000, 131(3):320-335.
17. Floresco S.B., Zhang Y., Enomoto T. Neural circuits subserving behavioral flexibility and their relevance to schizophrenia. Behav. Brain Res. 2009, 204(2):396-409. 10.1016/j.bbr.2008.12.001.
18. Ghahremani D.G., Monterosso J., Jentsch J.D., Bilder R.M., Poldrack R.A. Neural components underlying behavioral flexibility in human reversal learning. Cereb. Cortex 2010, 20(8):1843-1852. 10.1093/cercor/bhp247.
19. Gläscher J., Hampton A.N., O'Doherty J.P. Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. Cereb. Cortex (New York N. Y.) 2009, 19(2):483-495. 10.1093/cercor/bhn098.
20. Gläscher J., Daw N., Dayan P., O'Doherty J.P. States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 2010, 66(4):585-595. 10.1016/j.neuron.2010.04.016.
21. Goldberg M.E., Bruce C.J. Primate frontal eye fields. III. Maintenance of a spatially accurate saccade signal. J. Neurophysiol. 1990, 64(2):489-508.
22. Gregorios-Pippas L., Tobler P.N., Schultz W. Short-term temporal discounting of reward value in human ventral striatum. J. Neurophysiol. 2009, 101(3):1507-1523. 10.1152/jn.90730.2008.
23. Hampton A.N., O'Doherty J.P. Decoding the neural substrates of reward-related decision making with functional MRI. Proc. Natl Acad. Sci. USA 2007, 104(4):1377-1382. 10.1073/pnas.0606297104.
24. Hauber W., Sommer S. Prefrontostriatal circuitry regulates effort-related decision making. Cereb. Cortex 2009, 19(10):2240-2247. 10.1093/cercor/bhn241.
25. Hayden B.Y., Platt M.L. Neurons in anterior cingulate cortex multiplex information about reward and action. J. Neurosci. 2010, 30(9):3339-3346. 10.1523/JNEUROSCI.4874-09.2010.
26. Hayden B.Y., Nair A.C., McCoy A.N., Platt M.L. Posterior cingulate cortex mediates outcome-contingent allocation of behavior. Neuron 2008, 60(1):19-25. 10.1016/j.neuron.2008.09.012.
27. Hillman K.L., Bilkey D.K. Neurons in the rat anterior cingulate cortex dynamically encode cost-benefit in a spatial decision-making task. J. Neurosci. 2010, 30(22):7705-7713. 10.1523/JNEUROSCI.1273-10.2010.
28. Jenkinson M., Bannister P., Brady M., Smith S. Improved optimization for the robust and accurate linear registration and motion correction of brain images. Neuroimage 2002, 17(2):825-841. 10.1016/S1053-8119(02)91132-8.
29. Kenner N.M., Mumford J.A., Hommer R.E., Skup M., Leibenluft E., Poldrack R.A. Inhibitory motor control in response stopping and response switching. J. Neurosci. 2010, 30(25):8512-8518. 10.1523/JNEUROSCI.1096-10.2010.
30. Kim J., Ragozzino M.E. The involvement of the orbitofrontal cortex in learning under changing task contingencies. Neurobiol. Learn. Mem. 2005, 83(2):125-133. 10.1016/j.nlm.2004.10.003.
31. Lawrence A.D., Sahakian B.J., Rogers R.D., Hodges J.R., Robbins T.W. Discrimination, reversal, and shift learning in Huntington's disease: mechanisms of impaired response selection. Neuropsychologia 1999, 37(12):1359-1374. 10.1016/S0028-3932(99)00035-4.
32. Matsuzaka Y., Tanji J. Changing directions of forthcoming arm movements: neuronal activity in the presupplementary and supplementary motor area of monkey cerebral cortex. J. Neurophysiol. 1996, 76(4):2327-2342.
33. McAlonan K., Brown V.J. Orbital prefrontal cortex mediates reversal learning and not attentional set shifting in the rat. Behav. Brain Res. 2003, 146(1-2):97-103. 10.1016/j.bbr.2003.09.019.
34. Merriam E.P., Colby C.L. Active vision in parietal and extrastriate cortex. Neuroscientist 2005, 11(5):484-493. 10.1177/1073858405276871.
35. Minzenberg M.J., Watrous A.J., Yoon J.H., Ursu S., Carter C.S. Modafinil shifts human locus coeruleus to low-tonic, high-phasic activity during functional MRI. Science (New York, N.Y.) 2008, 322(5908):1700-1702. 10.1126/science.1164908.
36. Mosconi M.W., Kay M., D'Cruz A.M., Seidenfeld A., Guter S., Stanford L.D., et al. Impaired inhibitory control is associated with higher-order repetitive behaviors in autism spectrum disorders. Psychol. Med. 2009, 1-8. 10.1017/S0033291708004984.
37. O'Doherty J., Critchley H., Deichmann R., Dolan R.J. Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices. J. Neurosci. 2003, 23(21):7931-7939.
38. Palencia C.A., Ragozzino M.E. The effect of N-methyl-d-aspartate receptor blockade on acetylcholine efflux in the dorsomedial striatum during response reversal learning. Neuroscience 2006, 143(3):671-678. 10.1016/j.neuroscience.2006.08.024.
39. Paus T., Petrides M., Evans A.C., Meyer E. Role of the human anterior cingulate cortex in the control of oculomotor, manual, and speech responses: a positron emission tomography study. J. Neurophysiol. 1993, 70(2):453-469.
40. Picard N., Strick P.L. Motor areas of the medial wall: a review of their location and functional activation. Cereb. Cortex 1996, 6(3):342-353. 10.1093/cercor/6.3.342.
41. Preuschoff K., Quartz S.R., Bossaerts P. Human insula activation reflects risk prediction errors as well as risk. J. Neurosci. 2008, 28(11):2745-2752. 10.1523/JNEUROSCI.4286-07.2008.
42. Ragozzino M.E., Mohler E.G., Prior M., Palencia C.A., Rozman S. Acetylcholine activity in selective striatal regions supports behavioral flexibility. Neurobiol. Learn. Mem. 2009, 91(1):13-22. 10.1016/j.nlm.2008.09.008.
43. Rolls E.T., McCabe C., Redoute J. Expected value, reward outcome, and temporal difference error representations in a probabilistic decision task. Cereb. Cortex 2008, 18(3):652-663. 10.1093/cercor/bhm097.
44. Rosano C., Sweeney J.A., Melchitzky D.S., Lewis D.A. The human precentral sulcus: chemoarchitecture of a region corresponding to the frontal eye fields. Brain Res. 2003, 972(1-2):16-30. 10.1016/S0006-8993(03)02431-4.
45. Rosenberg D.R., Averbach D.H., O'Hearn K.M., Seymour A.B., Birmaher B., Sweeney J.A. Oculomotor response inhibition abnormalities in pediatric obsessive-compulsive disorder. Arch. Gen. Psychiatry 1997, 54(9):831-838.
46. Schultz W. Dopamine signals for reward value and risk: basic and recent data. Behav. Brain Funct. 2010, 6(1):24. Retrieved from http://www.behavioralandbrainfunctions.com/content/6/1/24.
47. Schultz W., Apicella P., Ljungberg T. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J. Neurosci. 1993, 13(3):900-913.
48. Schultz W., Dayan P., Montague R.R. A neural substrate of prediction and reward. Science 1997, 275:1593-1599.
49. Smith S.M. Fast robust automated brain extraction. Hum. Brain Mapp. 2002, 17(3):143-155. 10.1002/hbm.10062.
50. Smith S.M., Jenkinson M., Woolrich M.W., Beckmann C.F., Behrens T.E.J., Johansen-Berg H., et al. Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage 2004, 23(Supplement 1):S208-S219. 10.1016/j.neuroimage.2004.07.051.
51. Velanova K., Wheeler M.E., Luna B. Maturational changes in anterior cingulate and frontoparietal recruitment support the development of error processing and inhibitory control. Cereb. Cortex 2008, 18(11):2505-2522. 10.1093/cercor/bhn012.
52. Woo S.H., Lee K.M. Effect of the number of response alternatives on brain activity in response selection. Hum. Brain Mapp. 2007, 28(10):950-958. 10.1002/hbm.20317.
53. Xue G., Ghahremani D.G., Poldrack R.A. Neural substrates for reversing stimulus-outcome and stimulus-response associations. J. Neurosci. 2008, 28(44):11196-11204. 10.1523/JNEUROSCI.4001-08.2008.
54. Yeung N., Nieuwenhuis S. Dissociating response conflict and error likelihood in anterior cingulate cortex. J. Neurosci. 2009, 29(46):14506-14510. 10.1523/JNEUROSCI.3615-09.2009.
55. Zenon A., Filali N., Duhamel J.R., Olivier E. Salience representation in the parietal and frontal cortex. J. Cogn. Neurosci. 2010, 22(5):918-930. 10.1162/jocn.2009.21233.