Action-value comparisons in the dorsolateral prefrontal cortex control choice between goal-directed actions

Nature Communications

Volumn 5, Issue , 2014, Pages

  Morris, Richard W ^a   Dezfouli, Amir ^a   Griffiths, Kristi R ^a   Balleine, Bernard W ^a  

^a UNIVERSITY OF SYDNEY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

BRAIN; DECISION MAKING; IMAGING METHOD; NEUROLOGY; PHYSIOLOGY; PROBABILITY; SIGNALING;

ARTICLE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; FUNCTIONAL NEUROIMAGING; MOTOR CORTEX; PREFRONTAL CORTEX; PROBABILITY; TASK PERFORMANCE; ADOLESCENT; ADULT; ANATOMY AND HISTOLOGY; BAYES THEOREM; BRAIN MAPPING; COMPARATIVE STUDY; DECISION MAKING; FEMALE; HUMAN; LEARNING; MALE; MOTIVATION; NUCLEAR MAGNETIC RESONANCE IMAGING; PHYSIOLOGY; STATISTICAL MODEL; YOUNG ADULT;

ADOLESCENT; ADULT; BAYES THEOREM; BRAIN MAPPING; CHOICE BEHAVIOR; DECISION MAKING; FEMALE; GOALS; HUMANS; LEARNING; MAGNETIC RESONANCE IMAGING; MALE; MODELS, STATISTICAL; PREFRONTAL CORTEX; TASK PERFORMANCE AND ANALYSIS; YOUNG ADULT;

EID: 84904803301     PISSN: None     EISSN: 20411723     Source Type: Journal    
DOI: 10.1038/ncomms5390     Document Type: Article

References (67)

1. Gittins J. C. Bandit processes and dynamic allocation indices. J. R. Statist. Soc. B 148-177 (1979
2. Monica B. & Tze Leung L. Incomplete learning from endogenous data in dynamic allocation. Econometrica 68 1511-1516 (2000
3. Scott S. L. A modern Bayesian look at the multi-armed bandit. Appl. Stochastic Models Bus. Ind. 26 639-658 (2010
4. Dickinson A. & Balleine B. Motivational control of instrumental action. Curr. Dir. Psychol. Sci. 4 162-167 (1995
5. Platt M. & Glimcher P. Neural correlates of decision variables in parietal cortex. Nature 400 233-238 (1999 (Pubitemid 29334132)
6. Rangel A. Camerer C. & Montague P. A framework for studying the neurobiology of value-based decision making. Nat. Rev. Neurosci. 9 545-556 (2008 (Pubitemid 351873461)
7. Tversky A. & Kahneman D. The framing of decisions and the psychology of choice. Science 211 453-458 (1981 (Pubitemid 11145313)
8. Von Neumann J. & Morgenstern O. The Theory of Games and Economic Behavior (Princeton University Press 1947
9. Dayan P. & Abbott L. F. Theoretical Neuroscience (MIT Press 2001
10. Sutton R. S. & Barto A. G. Reinforcement Learning (MIT 1998
11. Watkins C. J. C. H. & Dayan P. Q-Learning. Machine Learning 8 279-292 (1992
12. Rangel A. & Hare T. Neural computations associated with goal-directed choice. Curr. Opin. Neurobiol. 20 262-270 (2010
13. Fitzgerald T. H. Friston K. J. & Dolan R. J. Action-specific value signals in reward-related regions of the human brain. J. Neurosci. 32 16417-16423 (2012
14. Hunt L. T. Woolrich M. W. Rushworth M. F. S. Behrens T. E. J. & Diedrichsen J. Trial-type dependent frames of reference for value comparison. PLoS Comput. Biol. 9 e1003225 (2013
15. Thompson W. R. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika 25 285-294 (1933
16. Thompson W. R. On the theory of apportionment. Am. J. Math. 57 450-456 (1935
17. Granmo O.-C. Solving two-armed bernoulli bandit problems using a bayesian learning automaton. Int. J. Intell. Comput. Cybernet. 3 207-234 (2010
18. Lau B. & Glimcher P. W. Value representations in the primate striatum during matching behavior. Neuron 58 451-463 (2008 (Pubitemid 351615183)
19. Palminteri S. Boraud T. Lafargue G. Dubois B. & Pessiglione M. Brain hemispheres selectively track the expected value of contralateral options. J. Neurosci. 29 13465-13472 (2009
20. Samejima K. Ueda Y. Doya K. & Kimura M. Representation of actionspecific reward values in the striatum. Science 310 1337-1340 (2005 (Pubitemid 41698867)
21. Wunderlich K. Rangel A. & O'Doherty J. P. Neural computations underlying action-based decision making in the human brain. Proc. Natl Acad. Sci. USA 106 17199-17204 (2009
22. Mai J. K. Paxinos G. & Voss T. Atlas of the Human Brain 3rd edn (Elsevier 2008
23. Maldjian J. A. Laurienti P. J. Kraft R. A. & Burdette J. H. An automated method for neuroanatomic and cytoarchitectonic atlas-based interrogation of fMRI data sets. Neuroimage 19 1233-1239 (2003 (Pubitemid 36918259)
24. Glascher J. Visualization of group inference data in functional neuroimaging. Neuroinformatics 7 73-82 (2009
25. Penny W. D. Trujillo-Barreto N. J. & Friston K. J. Bayesian fMRI time series analysis with spatial priors. Neuroimage 24 350-362 (2005 (Pubitemid 41359032)
26. Rosa M.J. Bestmann S. Harrison L. & Penny W. Bayesian model selection maps for group studies. Neuroimage 49 217-224 (2010
27. Boorman E. Behrens T. Woolrich M. & Rushworth M. How green is the grass on the other side? Frontopolar cortex and the evidence in favor of alternative courses of action. Neuron 62 733-743 (2009
28. Daw N. D. O'Doherty J. P. Dayan P. Seymour B. & Dolan R. J. Cortical substrates for exploratory decisions in humans. Nature 441 876-879 (2006
29. Glascher J. Hampton A. N. & O'Doherty J. P. Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. Cereb. Cortex 19 483-495 (2009
30. Hampton A. N. Bossaerts P. & O'Doherty J. P. The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J. Neurosci. 26 8360-8367 (2006 (Pubitemid 44309850)
31. Padoa-Schioppa C. & Assad J.A. Neurons in the orbitofrontal cortex encode economic value. Nature 441 223-226 (2006
32. Tanaka S. C. Balleine B.W. & O'Doherty J. P. Calculating consequences: brain systems that encode the causal effects of actions. J. Neurosci. 28 6750-6755 (2008
33. Valentin V. V. Dickinson A. & O'Doherty J. P. Determining the neural substrates of goal-directed learning in the human brain. J. Neurosci. 27 4019-4026 (2007 (Pubitemid 46597159)
34. Marreiros A. C. Kiebel S. J. & Friston K. J. Dynamic causal modelling for fMRI: A two-state model. Neuroimage 39 269-278 (2008 (Pubitemid 350102199)
35. Liljeholm M. Tricomi E. O'Doherty J. P. & Balleine B. W. Neural correlates of instrumental contingency learning: differential effects of action-reward conjunction and disjunction. J. Neurosci. 31 2474-2480 (2011
36. Gottfried J. A. O'Doherty J. & Dolan R. J. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 301 1104-1107 (2003 (Pubitemid 37022314)
37. Wallis J. & Miller E. Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task. Eur. J. Neurosci. 18 2069-2081 (2003 (Pubitemid 37372389)
38. Friston K. J. Functional and effective connectivity in neuroimaging: A synthesis. Hum. Brain Mapp. 2 56-78 (1994 (Pubitemid 124000442)
39. Haber S. N. Kim K. S. Mailly P. & Calzavara R. Reward-related cortical inputs define a large striatal region in primates that interface with associative cortical connections providing a substrate for incentive-based learning. J. Neurosci. 26 8368-8376 (2006 (Pubitemid 44309851)
40. Selemon L. D. & Goldman-Rakic P. S. Longitudinal topography and interdigitation of corticostriatal projections in the rhesus monkey. J. Neurosci. 5 776-794 (1985 (Pubitemid 15100467)
41. Yin H. H. Ostlund S. B. Knowlton B. J. & Balleine B. W. The role of the dorsomedial striatum in instrumental conditioning. Eur. J. Neurosci. 22 513-523 (2005 (Pubitemid 41097624)
42. Seo M. Lee E. & Averbeck B. B. Action selection and action value in frontalstriatal circuits. Neuron 74 947-960 (2012
43. Kim J. & Shadlen M. Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nat. Neurosci. 2 176-185 (1999 (Pubitemid 30491319)
44. Tsujimoto S. Genovesio A. & Wise S. Comparison of strategy signals in the dorsolateral and orbital prefrontal cortex. J. Neurosci. 31 4583-4592 (2011
45. Opris I. Barborica A. & Ferrera V. Microstimulation of the dorsolateral prefrontal cortex biases saccade target selection. J. Cognit. Neurosci. 17 893-904 (2005 (Pubitemid 40800023)
46. Camus M. et al. Repetitive transcranial magnetic stimulation over the right dorsolateral prefrontal cortex decreases valuations during food choices. Eur. J. Neurosci. 30 1980-1988 (2009
47. Kennerley S. & Wallis J. Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables. Eur. J. Neurosci. 29 2061-2073 (2009
48. Wallis J. Orbitofrontal cortex and its contribution to decision-making. Annu. Rev. Neurosci. 30 31-56 (2007 (Pubitemid 47218750)
49. Genovesio A. Brasted P. & Wise S. Representation of future and previous spatial goals by separate neural populations in prefrontal cortex. J. Neurosci. 26 7305-7316 (2006
50. Tanji J. & Hoshi E. Role of the lateral prefrontal cortex in executive behavioral control. Physiol. Rev. 88 37-57 (2008
51. Lu M. Preston J. & Strick P. Interconnections between the prefrontal cortex and the premotor areas in the frontal lobe. J. Comparat. Neurol. 341 375-392 (1994 (Pubitemid 24068528)
52. Preuss T. & Goldman-Rakic P. Connections of the ventral granular frontal cortex of macaques with perisylvian premotor and somatosensory areas: anatomical evidence for somatic representation in primate frontal association cortex. J. Comparat. Neurol. 282 293-316 (1989 (Pubitemid 19109366)
53. Takada M. et al. Organization of prefrontal outflow toward frontal motorrelated areas in macaque monkeys. Eur. J. Neurosci. 19 3328-3342 (2004 (Pubitemid 38859549)
54. Paus T. Primate anterior cingulate cortex: where motor control drive and cognition interface. Nat. Rev. Neurosci. 2 417-424 (2001 (Pubitemid 33674798)
55. Keramati M. Dezfouli A. & Piray P. Speed/accuracy trade-off between the habitual and the goal-directed processes. PLoS Comput. Biol. 7 e1002055 (2011
56. Simon D. A. & Daw N. Environmental statistics and the trade-off between model-based and TD learning in humans. Neural Inf. Process. Syst. 24 1-9 (2011
57. Dayan P. Kakade S. & Montague P. Learning and selective attention. Nat. Neurosci. 3 (Suppl): 1218-1223 (2000
58. Bach D. & Dolan R. Knowing how much you don't know: A neural organization of uncertainty estimates. Nat. Rev. Neurosci. 13 572-586 (2012
59. Behrens T. Woolrich M. Walton M. & Rushworth M. Learning the value of information in an uncertain world. Nat. Neurosci. 10 1214-1221 (2007 (Pubitemid 47339952)
60. Ito M. & Doya K. Validation of decision-making models and analysis of decision variables in the rat basal ganglia. J. Neurosci. 29 9861-9874 (2009
61. Lau B. & Glimcher P. W. Dynamic response-by-response models of matching behaviour in rhesus monkeys. J. Exp. Anal. Behav. 84 555-579 (2005 (Pubitemid 43339125)
62. Wachter A. & Biegler L. T. On the implementation of a primal-dual interior point filter line search algorithm for large scale nonlinear programming. Math. Program. 106 25-57 (2006
63. Daw N. D. in Attention and Performance XXIII: Decision Making Affect and Learning (eds Delgado M. R. Phelps E. A. & Robbins T. W.) 3-38 (Oxford University. Press 2011
64. Glascher J. Daw N. Dayan P. & O'Doherty J. P. States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66 585-595 (2010
65. Hare T. A. O'Doherty J. Camerer C. F. Schultz W. & Rangel A. Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors. J. Neurosci. 28 5623-5630 (2008
66. Stephan K. E. et al. Ten simple rules for dynamic causal modeling. Neuroimage 49 3099-3109 (2010
67. Stephan K. E. Penny W. D. Daunizeau J. Moran R. J. & Friston K. J. Bayesian model selection for group studies. Neuroimage 46 1004-1017 (2009).