Behavioral regulation and the modulation of information coding in the lateral prefrontal and cingulate cortex

Cerebral Cortex

Volumn 25, Issue 9, 2015, Pages 3197-3218

  Khamassi, Mehdi ^a,b,c,d   Quilodran, René ^a,b,e   Enel, Pierre ^a,b   Dominey, Peter F ^a,b   Procyk, Emmanuel ^a,b  

^a STEM CELL AND BRAIN RESEARCH INSTITUTE   (France)

^b UNIVERSITÉ DE LYON   (France)

^c UNIVERSITÉ PIERRE ET MARIE CURIE   (France)

^d CNRS   (France)

^e UNIVERSIDAD DE VALPARAÍSO   (Chile)

Author keywords

Adaptation; Cingulate; Decision; Feedback; Reinforcementlearning; Reward

Indexed keywords

ADAPTATION; ANIMAL EXPERIMENT; ARTICLE; BEHAVIOR CHANGE; BRAIN NERVE CELL; CINGULATE GYRUS; DECISION MAKING; DORSAL ANTERIOR CINGULATE CORTEX; ELECTROPHYSIOLOGICAL PROCEDURES; EXECUTIVE FUNCTION TEST; EXPERIMENTAL MONKEY; EXPLORATORY BEHAVIOR; FEEDBACK SYSTEM; LATERAL PREFRONTAL CORTEX; LEARNING; MALE; MONITORING; NONHUMAN; PRIORITY JOURNAL; REINFORCEMENT; TASK PERFORMANCE; THEORETICAL MODEL; UNCERTAINTY; WORKING MEMORY; ACTION POTENTIAL; ANIMAL; CYTOLOGY; ELECTROPHYSIOLOGY; NERVE CELL; PHYSIOLOGY; PREFRONTAL CORTEX; PROBLEM SOLVING; REGRESSION ANALYSIS; RHESUS MONKEY;

ACTION POTENTIALS; ANIMALS; DECISION MAKING; ELECTROPHYSIOLOGY; FEEDBACK, PHYSIOLOGICAL; GYRUS CINGULI; MACACA MULATTA; MALE; MODELS, THEORETICAL; NEURONS; PREFRONTAL CORTEX; PROBLEM SOLVING; REGRESSION ANALYSIS; REINFORCEMENT (PSYCHOLOGY);

EID: 84973432203     PISSN: 10473211     EISSN: 14602199     Source Type: Journal    
DOI: 10.1093/cercor/bhu114     Document Type: Article

References (69)

1. Alexander WH, Brown JW. 2010. Computational models of performance monitoring and cognitive control. Top Cogn Sci. 2:658-677.
2. Alexander WH, Brown JW. 2011. Medial prefrontal cortex as an action-outcome predictor. Nat Neurosci. 14:1338-1344.
3. Amiez C, Joseph JP, Procyk E. 2005. Anterior cingulate error-related activity is modulated by predicted reward. Eur J Neurosci. 21:3447-3452.
4. Amiez C, Neveu R, Warrot D, Petrides M, Knoblauch K, Procyk E. 2013. The location of feedback-related activity in the midcingulate cortex is predicted by local morphology. J Neurosci. 33:2217-2228.
5. Aston-Jones G, Cohen JD. 2005. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu Rev Neurosci. 28:403-450.
6. Barkley RA 2001. Linkages between attention and executive functions. In: Reid Lyon G, Krasnegor NA, editors. Attention, memory and executive function. P.H. Brooks. p. 307-326.
7. Barraclough DJ, Conroy ML, Lee D. 2004. Prefrontal cortex and decision making in a mixed-strategy game. Nat Neurosci. 7:404-410.
8. Bartumeus F, da Luz MG, Viswanathan GM, Catalan J. 2005. Animal search strategies: a quantitative random-walk analysis. Ecology. 86:3078-2087.
9. Behrens TE, Hunt LT, Rushworth MF. 2009. The computation of social behavior. Science. 324:1160-1164.
10. Behrens TE, Woolrich MW, Walton ME, Rushworth MF. 2007. Learning the value of information in an uncertain world. Nat Neurosci. 10:1214-1221.
11. Botvinick MM, Braver TS, Barch DM, Carter CS, Cohen JD. 2001. Conflict monitoring and cognitive control. Psychol Rev. 108:624-652.
12. Brown JW, Braver TS. 2005. Learned predictions of error likelihood in the anterior cingulate cortex. Science. 307:1118-1121.
13. Cohen JD, Aston-Jones G, Gilzenrat MS. 2004. A systems-level perspective on attention and cognitive control. In: Posner MI, editor. Cognitive Neuroscience of attention. New York: Guilford. p. 71-90.
14. Cohen JD, McClure SM, Yu AJ. 2007. Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philos Trans R Soc Lond B Biol Sci. 362: 933-942.
15. Collins AG, Frank MJ. 2012. How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis. Eur J Neurosci. 35:1024-1035.
16. Daw ND 2011. Trial-by-trial data analysis using computational models. In: Affect, Learning and Decision Making. New York: Oxford University Press.
17. Daw ND, O'Doherty JP, Dayan P, Seymour B, Dolan RJ. 2006. Cortical substrates for exploratory decisions in humans. Nature. 441:876-879.
18. Dehaene S, Kerszberg M, Changeux JP. 1998. A neuronal model of a global workspace in effortful cognitive tasks. Proc Natl Acad Sci USA. 95:14529-14534.
19. De Lillo C, Visalerberghi E, Aversano M. 1997. The organization of exhaustive searches in a patchy space by capuchin monkeys (Cebus apella). J Comp Psychol. 111:82-90.
20. Desrochers TM, Jin DZ, Goodman ND, Graybiel AM. 2010. Optimal habits can develop spontaneously through sensitivity to local cost. Proc Natl Acad Sci USA. 107:20512-20517.
21. Domenech P, Dreher JC. 2010. Decision threshold modulation in the human brain. J Neurosci. 30:14305-14317.
22. Dosenbach NU, Visscher KM, Palmer ED, Miezin FM, Wenger KK, Kang HC, Burgund ED, Grimes AL, Schlaggar BL, Petersen SE. 2006. A core system for the implementation of task sets. Neuron. 50:799-812.
23. Doya K. 2002. Metalearning and neuromodulation. Neural Netw. 15: 495-506.
24. Durstewitz D, Seamans JK. 2008. The dual-state theory of prefrontal cortex dopamine function with relevance to catechol-o-methyltransferase genotypes and schizophrenia. Biol Psychiatry. 64:739-749.
25. Durstewitz D, Vittoz NM, Floresco SB, Seamans JK. 2010. Abrupt transitions between prefrontal neural ensemble states accompany behavioral transitions during rule learning. Neuron. 66:438-448.
26. Enomoto K, Matsumoto N, Nakai S, Satoh T, Sato TK, Ueda Y, Inokawa H, Haruno M, Kimura M. 2011. Dopamine neurons learn to encode the long-term value of multiple future rewards. Proc Natl Acad Sci USA. 108:15462-15467.
27. Fonio E, Benjamini Y, Golani I. 2009. Freedom of movement and the stability of its unfolding in free exploration of mice. Proc Natl Acad Sci USA. 106:21335-21340.
28. Frank MJ, Doll BB, Oas-Terpstra J, Moreno F. 2009. Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nat Neurosci. 12:1062-1068.
29. Holroyd CB, Coles MG. 2002. The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol Rev. 109:679-709.
30. Humphries MD, Khamassi M, Gurney K. 2012. Dopaminergic control of the exploration-exploitation trade-off via the basal ganglia. Front Neurosci. 6:9.
31. Ishii S, Yoshida W, Yoshimoto J. 2002. Control of exploitation-exploration meta-parameter in reinforcement learning. Neural Netw. 15:665-687.
32. Ito M, Doya K. 2009. Validation of decision-making models and analysis of decision variables in the rat basal ganglia. J Neurosci. 29:9861-9874.
33. Kaping D, Vinck M, Hutchison RM, Everling S, Womelsdorf T. 2011. Specific contributions of ventromedial, anterior cingulate, and lateral prefrontal cortex for attentional selection and stimulus valuation. PLoS Biol. 9:e1001224.
34. Kennerley SW, Wallis JD. 2009. Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables. Eur J Neurosci. 29:2061-2073.
35. Kennerley SW, Walton ME. 2011. Decision making and reward in frontal cortex: complementary evidence from neurophysiological and neuropsychological studies. Behav Neurosci. 125:297-317.
36. Kennerley SW, Walton ME, Behrens TE, Buckley MJ, Rushworth MF. 2006. Optimal decision making and the anterior cingulate cortex. Nat Neurosci. 9:940-947.
37. Kerns JG, Cohen JD, MacDonald AW 3rd, Cho RY, Stenger VA, Carter CS. 2004. Anterior cingulate conflict monitoring and adjustments in control. Science. 303:1023-1026.
38. Khamassi M, Enel P, Dominey PF, Procyk E. 2013. Medial prefrontal cortex and the adaptive regulation of reinforcement learning parameters. Prog Brain Res. 202:441-464.
39. Khamassi M, Lallee S, Enel P, Procyk E, Dominey PF. 2011. Robot cognitive control with a neurophysiologically inspired reinforcement learning model. Front Neurorobot. 5:1.
40. Kolling N, Behrens TE, Mars RB, Rushworth MF. 2012. Neural mechanisms of foraging. Science. 336:95-98.
41. Kouneiher F, Charron S, Koechlin E. 2009. Motivation and cognitive control in the human prefrontal cortex. Nat Neurosci. 12:939-945.
42. Krichmar JL 2008. The neuromodulatory system-a framework for survival and adaptive behavior in a challenging world. Adapt Behav. 16:385-399.
43. Landmann C, Dehaene S, Pappata S, Jobert A, Bottlaender M, Roumenov D, Le Bihan D. 2007. Dynamics of prefrontal and cingulate activity during a reward-based logical deduction task. Cereb Cortex. 17:749-759.
44. Lau B. 2014. Matlab code for diagnosing collinearity in a regression design matrix. figshare. http://dx.doi.org/10.6084/m9.figshare.1008225.
45. Leung HC, Gore JC, Goldman-Rakic PS. 2002. Sustained mnemonic response in the human middle frontal gyrus during on-line storage of spatial memoranda. J Cogn Neurosci. 14:659-671.
46. Luk CH, Wallis JD. 2009. Dynamic encoding of responses and outcomes by neurons in medial prefrontal cortex. J Neurosci. 29:7526-7539.
47. MacDonald AW 3rd, Cohen JD, Stenger VA, Carter CS. 2000. Dissociating the role of the dorsolateral prefrontal and anterior cingulate cortex in cognitive control. Science. 288:1835-1838.
48. Matsumoto M, Matsumoto K, Abe H, Tanaka K. 2007. Medial prefrontal cell activity signaling prediction errors of action values. Nat Neurosci. 10:647-656.
49. McClure SM, Gilzenrat MS, Cohen JD. 2006. An exploration-exploitation model based on norepinephrine and dopamine activity. In: Weiss Y, Sholkopf B, Platt J, edirors. Advances in neural information processing systems. Cambridge (MA): MIT Press. p. 867-874.
50. Miller EK, Cohen JD. 2001. An integrative theory of prefrontal cortex function. Annu Rev Neurosci. 24:167-202.
51. Panzeri S, Senatore R, Montemurro MA, Petersen RS. 2007. Correcting for the sampling bias problem in spike train information measures. J Neurophysiol. 98:1064-1072.
52. Panzeri S, Treves A. 1996. Analytical estimates of limited sampling biases in different information measures. Network: Computation in Neural Systems. 7:87-107.
53. Procyk E, Goldman-Rakic PS. 2006. Modulation of dorsolateral prefrontal delay activity during self-organized behavior. J Neurosci. 26:11313-11323.
54. Procyk E, Tanaka YL, Joseph JP. 2000. Anterior cingulate activity during routine and non-routine sequential behaviors in macaques. Nat Neurosci. 3:502-508.
55. Quian Quiroga R, Panzeri S. 2009. Extracting information from neuronal populations: information theory and decoding approaches. Nat Rev Neurosci. 10:173-185.
56. Quilodran R, Rothé M, Procyk E. 2008. Behavioral shifts and action valuation in the anterior cingulate cortex. Neuron. 57(2):314-325.
57. Rothe M, Quilodran R, Sallet J, Procyk E. 2011. Coordination of high gamma activity in anterior cingulate and lateral prefrontal cortical areas during adaptation. J Neurosci. 31:11110-11117.
58. Rushworth MF, Behrens TE. 2008. Choice, uncertainty and value in prefrontal and cingulate cortex. Nat Neurosci. 11:389-397.
59. Satoh T, Nakai S, Sato T, Kimura M. 2003. Correlated coding of motivation and outcome of decision by dopamine neurons. J Neurosci. 23:9913-9923.
60. Schultz W, Dayan P, Montague PR. 1997. A neural substrate of prediction and reward. Science. 275:1593-1599.
61. Schweighofer N, Doya K. 2003. Meta-learning in reinforcement learning. Neural Netw. 16:5-9.
62. Seo H, Lee D. 2009. Behavioral and neural changes after gains and losses of conditioned reinforcers. J Neurosci. 29:3627-3641.
63. Seo H, Lee D. 2008. Cortical mechanisms for reinforcement learning in competitive games. Philos Trans R Soc Lond B Biol Sci. 363: 3845-3857.
64. Seo H, Lee D. 2007. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J Neurosci. 27:8366-8377.
65. Sutton RS, Barto AG. 1998. Reinforcement learning: an introduction. Cambridge (MA) London, England: MIT Press.
66. Treves A, Panzeri S. 1995. The upward bias in measures of information derived from limited data samples. Neural Comput. 7:399-407.
67. Vogt BA, Vogt L, Farber NB, Bush G. 2005. Architecture and neurocytology of monkey cingulate gyrus. J Comp Neurol. 485:218-239.
68. Wang XJ 2010. Neurophysiological and computational principles of cortical rhythms in cognition. Physiol Rev. 90:1195-1268.
69. Wilson CR, Gaffan D, Browning PG, Baxter MG. 2010. Functional localization within the prefrontal cortex: missing the forest for the trees? Trends Neurosci. 33:533-540.