An evolutionary computational theory of prefrontal executive function in decision-making

Philosophical Transactions of the Royal Society B: Biological Sciences

Volumn 369, Issue 1655, 2014, Pages

  Koechlin, Etienne ^a  

^a INSERM   (France)

Author keywords

Bayesian inference; Decision making; Executive control; Prefrontal cortex; Reasoning; Reinforcement learning

Indexed keywords

BAYESIAN ANALYSIS; BEHAVIORAL RESPONSE; COMPUTER SIMULATION; DECISION MAKING; LEARNING; NUMERICAL MODEL; RODENT;

ANIMAL; BAYES THEOREM; BIOLOGICAL MODEL; DECISION MAKING; EXECUTIVE FUNCTION; HUMAN; LEARNING; PHYSIOLOGY; PREFRONTAL CORTEX; PRIMATE; REINFORCEMENT; RODENT;

ANIMALS; BAYES THEOREM; DECISION MAKING; EXECUTIVE FUNCTION; HUMANS; LEARNING; MODELS, NEUROLOGICAL; PREFRONTAL CORTEX; PRIMATES; REINFORCEMENT (PSYCHOLOGY); RODENTIA;

EID: 84907493818     PISSN: 09628436     EISSN: 14712970     Source Type: Journal    
DOI: 10.1098/rstb.2013.0474     Document Type: Article

References (57)

1. Uylings HB, Groenewegen HJ, Kolb B. 2003 Do rats have a prefrontal cortex? Behav. Brain Res. 146, 3-17. (doi:10.1016/j.bbr.2003.09.028)
2. Fuster JM. 1989 The prefrontal cortex, 2nd edn. New York, NY: Raven Press.
3. Uylings HB, Jacobsen AM, Zilles K, Amunts K. 2006 Left-right asymmetry in volume and number of neurons in adult Broca’s area. Cortex 42, 652-658. doi:10.1016/S0010-9452(08) 70401-5
4. Schenker NM, Hopkins WD, Spocter MA, Garrison AR, Stimpson CD, Erwin JM, Hof PR, Sherwood CC. 2010 Broca’s area homologue in chimpanzees. Cereb. Cortex 20, 730–742. (doi:10.1093/cercor/bhp138)
5. Broca P. 1861 Remarques sur le siège de la faculté du langage articulé suivie d’une observation d’aphémie. Bull. Soc. Anat. (Paris) 6, 330.
6. Semendeferi K, Armstrong E, Schleicher A, Zilles K, Van Hoesen GW. 2001 Prefrontal cortex in humans and apes: a comparative study of area 10. Am. J. Phys. Anthropol. 114, 224–241. (doi:10.1002/1096-8644(200103)114:3,224::AID-AJPA1022.3.0.CO;2-I)
7. Teffer K, Semendeferi K. 2012 Human prefrontal cortex: evolution, development, and pathology. Prog. Brain Res. 195, 191-218. (doi:10.1016/B978-0-444-53860-4.00009-X)
8. Koechlin E. 2011 Frontal pole function: what is specifically human? Trends Cogn. Sci. 15, 241. (doi:10.1016/j.tics.2011.04.005)
9. Neubert FX, Mars RB, Thomas AG, Sallet J, Rushworth M. 2014 Comparison of human ventral cortex areas for cognitive control and language with areas in monkey frontal cortex. Neuron 82, 700-713. (doi:10.1016/j.neuron.2013.11.012)
10. Koechlin E, Hyafil A. 2007 Anterior prefrontal function and the limits of human decision-making. Science 318, 594–598. (doi:10.1126/science.1142995)
11. Stephenson-Jones M, Samuelsson E, Ericsson J, Robertson B, Grillner S. 2011 Evolutionary conservation of the basal ganglia as a common vertebrate mechanism for action selection. Curr. Biol. 21, 1081-1091. (doi:10.1016/j.cub.2011.05.001)
12. Doya K. 2007 Reinforcement learning: computational theory and biological mechanisms. HFSP J. 1, 30–40. (doi:10.2976/1.2732246/10.2976/1)
13. Samejima K, Ueda Y, Doya K, Kimura M. 2005 Representation of action-specific reward values in the striatum. Science 310, 1337-1340. (doi:10.1126/science.1115270)
14. Schultz W, Dayan P, Montague PR. 1997 A neural substrate of prediction and reward. Science 275, 1593-1599. (doi:10.1126/science.275.5306.1593)
15. Sutton RS, Barto AG. 1998 Reinforcement learning. Cambridge, MA: MIT Press.
16. Kahnt T, Park SQ, Cohen MX, Beck A, Heinz A, Wrase J. 2009 Dorsal striatal–midbrain connectivity in humans predicts how reinforcements are used to guide decisions. J. Cogn. Neurosci. 21, 1332–1345. (doi:10.1162/jocn.2009.21092)
17. Atallah HE, Lopez-Paniagua D, Rudy JW, O’Reilly RC. 2007 Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nat. Neurosci. 10, 126-131. (doi:10.1038/nn1817)
18. O’Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ. 2004 Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452–454. (doi:10.1126/science.1094285)
19. Teh YW, Jordan MI, Beal MJ, Blei DM. 2006 Hierarchical dirichlet processes. J. Am. Stat. Assoc. 101, 1566-1581. (doi:10.1198/01621450600 0000302)
20. Daw ND, Niv Y, Dayan P. 2005 Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704-1711. (doi:10.1038/nn1560)
21. Asaad WF, Rainer G, Miller EK. 1998 Neural activity in the primate prefrontal cortex during associative learning. Neuron 21, 1399–1407. (doi:10.1016/S0896-6273(00)80658-3)
22. Pasupathy A, Miller EK. 2005 Different time courses of learning-related activity in the prefrontal cortex and striatum. Nature 433, 873-876. (doi:10.1038/nature03287)
23. Nassar MR, Wilson RC, Heasly B, Gold JI. 2010 An approximately Bayesian delta-rule model explains the dynamics of belief updating in a changing environment. J. Neurosci. 30, 12366-12378. (doi:10.1523/JNEUROSCI.0822-10.2010)
24. Karlsson MP, Tervo DG, Karpova AY. 2012 Network resets in medial prefrontal cortex mark the onset of behavioral uncertainty. Science 338, 135–139. (doi:10.1126/science.1226518)
25. Collins A, Koechlin E. 2012 Reasoning, learning, and creativity: frontal lobe function and human decision-making. PLoS Biol. 10, e1001293. (doi:10.1371/journal.pbio.1001293).
26. Histed MH, Pasupathy A, Miller EK. 2009 Learning substrates in the primate prefrontal cortex and striatum: sustained activity related to successful actions. Neuron 63, 244–253. (doi:10.1016/j.neuron.2009.06.019)
27. Hadj-Bouziane F, Meunier M, Boussaoud D. 2003 Conditional visuo-motor learning in primates: a key role for the basal ganglia. J. Physiol. Paris 97, 567–579. (doi:10.1016/j.jphysparis.2004.01.014)
28. Packard MG, Knowlton BJ. 2002 Learning and memory functions of the basal ganglia. Annu. Rev. Neurosci. 25, 563–593. (doi:10.1146/annurev.neuro.25.112701.142937)
29. Jones JL, Esber GR, McDannald MA, Gruber AJ, Hernandez A, Mirenzi A, Schoenbaum G. 2012 Orbitofrontal cortex supports behavior and learning using inferred but not cached values. Science 338, 953–956. (doi:10.1126/science.1227489)
30. McDannald MA, Lucantonio F, Burke KA, Niv Y, Schoenbaum G. 2011 Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. J. Neurosci. 31, 2700–2705. (doi:10.1523/JNEUROSCI.5499-10.2011)
31. Rudebeck PH, Murray EA. 2011 Dissociable effects of subtotal lesions within the macaque orbital prefrontal cortex on reward-guided behavior. J. Neurosci. 31, 10569–10578. (doi:10.1523/JNEUROSCI.0091-11.2011)
32. Noonan MP, Walton ME, Behrens TEJ, Sallet J, Buckley MJ, Rushworth MFS. 2010 Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex. Proc. Natl Acad. Sci. USA 107, 20547–20552. (doi:10.1073/pnas.1012246107)
33. Hampton AN, Bossaerts P, O’Doherty JP. 2006 The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J. Neurosci. 26, 8360-8367. (doi:10.1523/JNEUROSCI.1010-06.2006)
34. Boorman ED, Behrens TE, Woolrich MW, Rushworth MF. 2009 How green is the grass on the other side? Frontopolar cortex and the evidence in favor of alternative courses of action. Neuron 62, 733–743. (doi:10.1016/j.neuron.2009.05.014)
35. Quilodran R, Rothe M, Procyk E. 2008 Behavioral shifts and action valuation in the anterior cingulate cortex. Neuron 57, 314–325. (doi:10.1016/j.neuron.2007.11.031)
36. Kolling N, Behrens TE, Mars RB, Rushworth MF. 2012 Neural mechanisms of foraging. Science 336, 95-98. (doi:10.1126/science.1216930)
37. Isoda M, Hikosaka O. 2007 Switching from automatic to controlled action by monkey medial frontal cortex. Nat. Neurosci. 10, 240-248. (doi:10.1038/nn1830)
38. Donahue CH, Seo H, Lee D. 2013 Cortical signals for rewarded actions and strategic exploration. Neuron 80, 223–234. (doi:10.1016/j.neuron.2013.07.040)
39. Alexander WH, Brown JW. 2011 Medial prefrontal cortex as an action-outcome predictor. Nat. Neurosci. 14, 1338-1344. (doi:10.1038/nn.2921)
40. Dosenbach NU et al. 2006 A core system for the implementation of task sets. Neuron 50, 799–812. (doi:10.1016/j.neuron.2006.04.031)
41. Liljeholm M, O’Doherty JP. 2012 Contributions of the striatum to learning, motivation, and performance: an associative account. Trends Cogn. Sci. 16, 467–475. (doi:10.1016/j.tics.2012.07.007)
42. Alexander GE, DeLong MR, Strick PL. 1986 Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 9, 357-381. (doi:10.1146/annurev.ne.09.030186.002041)
43. Miller EK, Cohen JD. 2001 An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167-202. (doi:10.1146/annurev.neuro.24.1.167)
44. Koechlin E, Ody C, Kouneiher F. 2003 The architecture of cognitive control in the human prefrontal cortex. Science 302, 1181–1185. (doi:10.1126/science.1088545)
45. Badre D, Kayser AS, D’Esposito M. 2010 Frontal cortex and the discovery of abstract action rules. Neuron 66, 315–326. (doi:10.1016/j.neuron.2010.03.025)
46. Barbas H, Pandya DN. 1989 Architecture and intrinsic connections of the prefrontal cortex in the rhesus monkey. J. Comp. Neurol. 286, 353–375. (doi:10.1002/cne.902860306)
47. Doya K, Samejima K, Katagiri K, Kawato M. 2002 Multiple model-based reinforcement learning. Neural Comput. 14, 1347-1369. (doi:10.1162/089976602753712972)
48. Cowan N. 2005 Working-memory capacity limits in a theoretical context. In Human learning and memory (eds C Izawa, N Ohta), pp. 155–175. Hillsdale, NJ: Erlbaum.
49. Koechlin E, Basso G, Pietrini P, Panzer S, Grafman J. 1999 The role of the anterior prefrontal cortex in human cognition. Nature 399, 148–151. (doi:10.1038/20178)
50. Sakai K. 2008 Task set and prefrontal cortex. Annu. Rev. Neurosci. 31, 219–245. (doi:10.1146/annurev.neuro.31.060407.125642)
51. Yu A, Dayan P. 2005 Uncertainty, neuromodulation, and attention. Neuron 46, 681-692. (doi:10.1016/j.neuron.2005.04.026)
52. De Martino B, Fleming SM, Garrett N, Dolan RJ. 2013 Confidence in value-based choice. Nat. Neurosci. 16, 105–110. (doi:10.1038/nn.3279)
53. Oberauer K. 2010 Declarative and procedural working memory: common principles, common capacity limits? Psychol. Belgica 50, 277–308. (doi:10.5334/pb-50-3-4-277)
54. Risse S, Oberauer K. 2010 Selection of objects and tasks in working memory. Q. J. Exp. Psychol. 63, 784-804. (doi:10.1080/17470210903147486)
55. Oberauer K. 2002 Access to information in working memory: exploring the focus of attention. J. Exp. Psychol. Learn. Mem. Cogn. 28, 411–421. (doi:10.1037/0278-7393.28.3.411)
56. Donoso M, Collins A, Koechlin E. 2014 Foundations of human reasoning in the prefrontal cortex Science 344, 1481–1486. (doi:10.1126/science.1252254)
57. Genovesio A, Wise S, Passingham R. 2014 Prefrontal-parietal function: from foraging to foresight. Trends Cogn. Sci. 18, 72-81. (doi:10.1016/j.tics.2013.11.007)