Rostrolateral prefrontal cortex and individual differences in uncertainty-driven exploration

Neuron

Volumn 73, Issue 3, 2012, Pages 595-607

  Badre, David ^a   Doll, BradleyB ^a   Long, NicoleM ^a   Frank, MichaelJ ^a  

^a BROWN UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; ARTICLE; CONTROLLED STUDY; DECISION MAKING; EXPLORATORY BEHAVIOR; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; LEARNING; MALE; MATHEMATICAL MODEL; NORMAL HUMAN; PREFRONTAL CORTEX; PRIORITY JOURNAL; REINFORCEMENT; REWARD; ROSTROLATERAL PREFRONTAL CORTEX;

ADOLESCENT; ADULT; BRAIN MAPPING; CHOICE BEHAVIOR; COMPUTER SIMULATION; EXPLORATORY BEHAVIOR; FEMALE; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; INDIVIDUALITY; MAGNETIC RESONANCE IMAGING; MALE; MODELS, NEUROLOGICAL; NEUROPSYCHOLOGICAL TESTS; OXYGEN; PREFRONTAL CORTEX; REACTION TIME; REINFORCEMENT (PSYCHOLOGY); UNCERTAINTY; YOUNG ADULT;

EID: 84856734754     PISSN: 08966273     EISSN: 10974199     Source Type: Journal    
DOI: 10.1016/j.neuron.2011.12.025     Document Type: Article

References (40)

1. Aston-Jones G., Cohen J.D. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 2005, 28:403-450.
2. Badre D. Cognitive control, hierarchy, and the rostro-caudal organization of the frontal lobes. Trends Cogn. Sci. (Regul. Ed.) 2008, 12:193-200.
3. Boorman E.D., Behrens T.E., Woolrich M.W., Rushworth M.F. How green is the grass on the other side? Frontopolar cortex and the evidence in favor of alternative courses of action. Neuron 2009, 62:733-743.
4. Braver T.S., Bongiolatti S.R. The role of frontopolar cortex in subgoal processing during working memory. Neuroimage 2002, 15:523-536.
5. Bunge S.A., Wendelken C. Comparing the bird in the hand with the ones in the bush. Neuron 2009, 62:609-611.
6. Bunge S.A., Wendelken C., Badre D., Wagner A.D. Analogical reasoning and prefrontal cortex: evidence for separable retrieval and integration mechanisms. Cereb. Cortex 2005, 15:239-249.
7. Christoff K., Prabhakaran V., Dorfman J., Zhao Z., Kroger J.K., Holyoak K.J., Gabrieli J.D. Rostrolateral prefrontal cortex involvement in relational integration during reasoning. Neuroimage 2001, 14:1136-1149.
8. Christoff K., Ream J.M., Geddes L.P.T., Gabrieli J.D.E. Evaluating self-generated information: anterior prefrontal contributions to human cognition. Behav. Neurosci. 2003, 117:1161-1168.
9. Cohen J.D., McClure S.M., Yu A.J. Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philos. Trans. R. Soc. Lond. B Biol. Sci. 2007, 362:933-942.
10. Dale A.M. Optimal experimental design for event-related fMRI. Hum. Brain Mapp. 1999, 8:109-114.
11. Daw N.D., O'Doherty J.P., Dayan P., Seymour B., Dolan R.J. Cortical substrates for exploratory decisions in humans. Nature 2006, 441:876-879.
12. Dayan P., Sejnowski T.J. Exploration bonuses and dual control. Mach. Learn. 1996, 25:5-22.
13. Doll B.B., Frank M.J. The basal ganglia in reward and decision making: computational models and empirical studies. Handbook of Reward and Decision Making 2009, 399-425. Academic Press, Oxford. J.C. Dreher, L. Tremblay (Eds.).
14. Ellsberg D. Risk, ambiguity, and the savage axioms. Q.J. Econ. 1961, 75:643-669.
15. Frank M.J., Scheres A., Sherman S.J. Understanding decision-making deficits in neurological conditions: insights from models of natural action selection. Philos. Trans. R. Soc. Lond. B Biol. Sci. 2007, 362:1641-1654.
16. Frank M.J., Doll B.B., Oas-Terpstra J., Moreno F. Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nat. Neurosci. 2009, 12:1062-1068.
17. Gilbert S.J., Spengler S., Simons J.S., Steele J.D., Lawrie S.M., Frith C.D., Burgess P.W. Functional specialization within rostral prefrontal cortex (area 10): a meta-analysis. J.Cogn. Neurosci. 2006, 18:932-948.
18. Gittins J.C., Jones D.A. A dynamic allocation index for the sequential design of experiments. Statistics 1974, 241-266. North Holland Publishing Company, Amsterdam. J. Gani, K. Sarkadi, I. Vincze (Eds.).
19. Han S., Huettel S.A., Dobbins I.G. Rule-dependent prefrontal cortex activity across episodic and perceptual decisions: an fMRI investigation of the criterial classification account. J.Cogn. Neurosci. 2009, 21:922-937.
20. Jepma M., Te Beek E.T., Wagenmakers E.J., van Gerven J.M., Nieuwenhuis S. The role of the noradrenergic system in the exploration-exploitation trade-off: a psychopharmacological study. Front Hum Neurosci 2010, 4:170.
21. Koechlin E., Summerfield C. An information theoretical approach to prefrontal executive function. Trends Cogn. Sci. (Regul. Ed.) 2007, 11:229-235.
22. Koechlin E., Basso G., Pietrini P., Panzer S., Grafman J. The role of the anterior prefrontal cortex in human cognition. Nature 1999, 399:148-151.
23. Koechlin E., Ody C., Kouneiher F. The architecture of cognitive control in the human prefrontal cortex. Science 2003, 302:1181-1185.
24. Krawczyk D.C., Michelle McClelland M., Donovan C.M. A hierarchy for relational reasoning in the prefrontal cortex. Cortex 2011, 47:588-597.
25. Kroger J.K., Sabb F.W., Fales C.L., Bookheimer S.Y., Cohen M.S., Holyoak K.J. Recruitment of anterior dorsolateral prefrontal cortex in human reasoning: a parametric study of relational complexity. Cereb. Cortex 2002, 12:477-485.
26. Lau B., Glimcher P.W. Dynamic response-by-response models of matching behavior in rhesus monkeys. J.Exp. Anal. Behav. 2005, 84:555-579.
27. Maia T.V. Reinforcement learning, conditioning, and the brain: Successes and challenges. Cogn. Affect. Behav. Neurosci. 2009, 9:343-364.
28. Montague P.R., Dayan P., Sejnowski T.J. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J.Neurosci. 1996, 16:1936-1947.
29. Moustafa A.A., Cohen M.X., Sherman S.J., Frank M.J. A role for dopamine in temporal decision making and reward maximization in parkinsonism. J.Neurosci. 2008, 28:12294-12304.
30. Payzan-LeNestour E., Bossaerts P. Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings. PLoS Comput. Biol. 2011, 7:e1001048.
31. Ramnani N., Owen A.M. Anterior prefrontal cortex: insights into function from anatomy and neuroimaging. Nat. Rev. Neurosci. 2004, 5:184-194.
32. Schönberg T., Daw N.D., Joel D., O'Doherty J.P. Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. J.Neurosci. 2007, 27:12860-12867.
33. Speed A. Abstract relational categories, graded persistence, and prefrontal cortical representation. Cognitive Neuroscience 2010, 1:126-152.
34. Strauss G.P., Frank M.J., Waltz J.A., Kasanova Z., Herbener E.S., Gold J.M. Deficits in positive reinforcement learning and uncertainty-driven exploration are associated with distinct aspects of negative symptoms in schizophrenia. Biol. Psychiatry 2011, 69:424-431.
35. Sutton R.S., Barto A.G. Reinforcement Learning: An Introduction 1998, MIT Press, Cambridge, MA.
36. Tsujimoto S., Genovesio A., Wise S.P. Frontal pole cortex: encoding ends at the end of the endbrain. Trends Cogn. Sci. (Regul. Ed.) 2011, 15:169-176.
37. Wallis J.D. Polar exploration. Nat. Neurosci. 2010, 13:7-8.
38. Yantis S., Serences J.T. Cortical mechanisms of space-based and object-based attentional control. Curr. Opin. Neurobiol. 2003, 13:187-193.
39. Yantis S., Schwarzbach J., Serences J.T., Carlson R.L., Steinmetz M.A., Pekar J.J., Courtney S.M. Transient neural activity in human parietal cortex during spatial attention shifts. Nat. Neurosci. 2002, 5:995-1002.
40. Yoshida W., Ishii S. Resolution of uncertainty in prefrontal cortex. Neuron 2006, 50:781-789.