Functionally Dissociable Influences on Learning Rate in a Dynamic Environment

Neuron

Volumn 84, Issue 4, 2014, Pages 870-881

  McGuire, Joseph T ^a   Nassar, Matthew R ^a,b   Gold, Joshua I ^a   Kable, Joseph W ^a  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

^b BROWN UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; ARTICLE; BOLD SIGNAL; BRAIN FUNCTION; CONNECTOME; CONTROLLED STUDY; CORPUS STRIATUM; DECISION MAKING; FEMALE; FRONTAL CORTEX; FUNCTIONAL DISSOCIATION; HUMAN; LEARNING; LEARNING ENVIRONMENT; MALE; PARIETAL CORTEX; PREFRONTAL CORTEX; TASK PERFORMANCE; VISUAL CORTEX; ADOLESCENT; BRAIN; BRAIN MAPPING; ENVIRONMENT; FUNCTIONAL NEUROIMAGING; IMAGE PROCESSING; NUCLEAR MAGNETIC RESONANCE IMAGING; PHYSIOLOGY; REWARD; UNCERTAINTY; YOUNG ADULT;

ADOLESCENT; ADULT; BRAIN; BRAIN MAPPING; ENVIRONMENT; FEMALE; FUNCTIONAL NEUROIMAGING; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; LEARNING; MAGNETIC RESONANCE IMAGING; MALE; REWARD; UNCERTAINTY; YOUNG ADULT;

EID: 84918819779     PISSN: 08966273     EISSN: 10974199     Source Type: Journal    
DOI: 10.1016/j.neuron.2014.10.013     Document Type: Article

References (48)

1. Adams, R.P., and MacKay, D.J.C. (2007). Bayesian online changepoint detection. In arXiv preprint arXiv:0710.3742.
2. Aguirre G.K., Zarahn E., D'esposito M. The variability of human, BOLD hemodynamic responses. Neuroimage 1998, 8:360-369.
3. Aston-Jones G., Cohen J.D. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 2005, 28:403-450.
4. Badre D., Doll B.B., Long N.M., Frank M.J. Rostrolateral prefrontal cortex and individual differences in uncertainty-driven exploration. Neuron 2012, 73:595-607.
5. Bartra O., McGuire J.T., Kable J.W. The valuation system: a coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value. Neuroimage 2013, 76:412-427.
6. Behrens T.E.J., Woolrich M.W., Walton M.E., Rushworth M.F.S. Learning the value of information in an uncertain world. Nat. Neurosci. 2007, 10:1214-1221.
7. Berns G.S., McClure S.M., Pagnoni G., Montague P.R. Predictability modulates human brain response to reward. J.Neurosci. 2001, 21:2793-2798.
8. Boorman E.D., Behrens T.E., Rushworth M.F. Counterfactual choice and learning in a neural network centered on human lateral frontopolar cortex. PLoS Biol. 2011, 9:e1001093.
9. Braver T.S. The variable nature of cognitive control: a dual mechanisms framework. Trends Cogn. Sci. 2012, 16:106-113.
10. Bright M.G., Bulte D.P., Jezzard P., Duyn J.H. Characterization of regional heterogeneity in cerebrovascular reactivity dynamics using novel hypocapnia task and BOLD fMRI. Neuroimage 2009, 48:166-175.
11. Cavanagh J.F., Figueroa C.M., Cohen M.X., Frank M.J. Frontal theta reflects uncertainty and unexpectedness during exploration and exploitation. Cereb. Cortex 2012, 22:2575-2586.
12. Chang C., Thomason M.E., Glover G.H. Mapping and correction of vascular hemodynamic latency in the BOLD signal. Neuroimage 2008, 43:90-102.
13. Cole M.W., Yeung N., Freiwald W.A., Botvinick M. Cingulate cortex: diverging data from humans and monkeys. Trends Neurosci. 2009, 32:566-574.
14. Collins A.G.E., Frank M.J. How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis. Eur. J. Neurosci. 2012, 35:1024-1035.
15. Cox R.W. AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput. Biomed. Res. 1996, 29:162-173.
16. Cox R.W. AFNI: what a long strange trip it's been. Neuroimage 2012, 62:743-747.
17. Critchley H.D., Tang J., Glaser D., Butterworth B., Dolan R.J. Anterior cingulate activity during error and autonomic response. Neuroimage 2005, 27:885-895.
18. Daw N.D., O'Doherty J.P., Dayan P., Seymour B., Dolan R.J. Cortical substrates for exploratory decisions in humans. Nature 2006, 441:876-879.
19. De Martino B., Fleming S.M., Garrett N., Dolan R.J. Confidence in value-based choice. Nat. Neurosci. 2013, 16:105-110.
20. Fearnhead P., Liu Z. On-line inference for multiple changepoint problems. J.R. Stat. Soc. Ser. A Stat. Soc. 2007, 69:589-605.
21. Fedorenko E., Duncan J., Kanwisher N. Broad domain generality in focal regions of frontal and parietal cortex. Proc. Natl. Acad. Sci. USA 2013, 110:16616-16621.
22. Fischer A.G., Ullsperger M. Real and fictive outcomes are processed differently but converge on a common adaptive mechanism. Neuron 2013, 79:1243-1255.
23. Fleming S.M., Huijgen J., Dolan R.J. Prefrontal contributions to metacognition in perceptual decision making. J.Neurosci. 2012, 32:6117-6125.
24. Foerde K., Braun E.K., Shohamy D. A trade-off between feedback-based learning and episodic memory for feedback events: evidence from Parkinson's disease. Neurodegener. Dis. 2013, 11:93-101.
25. Friston K.J., Buechel C., Fink G.R., Morris J., Rolls E., Dolan R.J. Psychophysiological and modulatory interactions in neuroimaging. Neuroimage 1997, 6:218-229.
26. Greve D.N., Fischl B. Accurate and robust brain image alignment using boundary-based registration. Neuroimage 2009, 48:63-72.
27. Handwerker D.A., Ollinger J.M., D'Esposito M. Variation of BOLD hemodynamic responses across subjects and brain regions and their effects on statistical analyses. Neuroimage 2004, 21:1639-1651.
28. Hayden B.Y., Pearson J.M., Platt M.L. Fictive reward signals in the anterior cingulate cortex. Science 2009, 324:948-950.
29. Jenkinson M., Smith S. A global optimisation method for robust affine registration of brain images. Med. Image Anal. 2001, 5:143-156.
30. Jenkinson M., Bannister P., Brady M., Smith S. Improved optimization for the robust and accurate linear registration and motion correction of brain images. Neuroimage 2002, 17:825-841.
31. Jenkinson M., Beckmann C.F., Behrens T.E.J., Woolrich M.W., Smith S.M. FSL. Neuroimage 2012, 62:782-790.
32. Kim H., Cabeza R. Trusting our memories: dissociating the neural correlates of confidence in veridical versus illusory memories. J.Neurosci. 2007, 27:12190-12197.
33. Li J., Schiller D., Schoenbaum G., Phelps E.A., Daw N.D. Differential roles of human striatum and amygdala in associative learning. Nat. Neurosci. 2011, 14:1250-1252.
34. Moberget T., Gullesen E.H., Andersson S., Ivry R.B., Endestad T. Generalized role for the cerebellum in encoding internal models: evidence from semantic processing. J.Neurosci. 2014, 34:2871-2878.
35. Mumford J.A., Turner B.O., Ashby F.G., Poldrack R.A. Deconvolving BOLD activation in event-related designs for multivoxel pattern classification analyses. Neuroimage 2012, 59:2636-2643.
36. Nassar M.R., Wilson R.C., Heasly B., Gold J.I. An approximately Bayesian delta-rule model explains the dynamics of belief updating in a changing environment. J.Neurosci. 2010, 30:12366-12378.
37. Nassar M.R., Rumsey K.M., Wilson R.C., Parikh K., Heasly B., Gold J.I. Rational regulation of learning dynamics by pupil-linked arousal systems. Nat. Neurosci. 2012, 15:1040-1046.
38. Nichols T.E., Holmes A.P. Nonparametric permutation tests for functional neuroimaging: a primer with examples. Hum. Brain Mapp. 2002, 15:1-25.
39. Nieuwenhuis S., Aston-Jones G., Cohen J.D. Decision making, the P3, and the locus coeruleus-norepinephrine system. Psychol. Bull. 2005, 131:510-532.
40. O'Reilly J.X., Schüffelgen U., Cuell S.F., Behrens T.E.J., Mars R.B., Rushworth M.F.S. Dissociable effects of surprise and model update in parietal and anterior cingulate cortex. Proc. Natl. Acad. Sci. USA 2013, 110:E3660-E3669.
41. Payzan-LeNestour E., Dunne S., Bossaerts P., O'Doherty J.P. The neural representation of unexpected uncertainty during value-based decision making. Neuron 2013, 79:191-201.
42. Rushworth M.F.S., Behrens T.E.J. Choice, uncertainty and value in prefrontal and cingulate cortex. Nat. Neurosci. 2008, 11:389-397.
43. Seeley W.W., Menon V., Schatzberg A.F., Keller J., Glover G.H., Kenna H., Reiss A.L., Greicius M.D. Dissociable intrinsic connectivity networks for salience processing and executive control. J.Neurosci. 2007, 27:2349-2356.
44. Smith S.M., Jenkinson M., Woolrich M.W., Beckmann C.F., Behrens T.E.J., Johansen-Berg H., Bannister P.R., De Luca M., Drobnjak I., Flitney D.E., et al. Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage 2004, 23(Suppl 1):S208-S219.
45. Vilares I., Howard J.D., Fernandes H.L., Gottfried J.A., Kording K.P. Differential representations of prior and likelihood uncertainty in the human brain. Curr. Biol. 2012, 22:1641-1648.
46. Wilson R.C., Nassar M.R., Gold J.I. Bayesian online learning of the hazard rate in change-point problems. Neural Comput. 2010, 22:2452-2476.
47. Yu A.J., Dayan P. Uncertainty, neuromodulation, and attention. Neuron 2005, 46:681-692.
48. Zarahn E., Aguirre G.K., D'Esposito M. Temporal isolation of the neural correlates of spatial mnemonic processing with fMRI. Brain Res. Cogn. Brain Res. 1999, 7:255-268.