The basal ganglia in reward and decision making. Computational models and empirical studies

Handbook of Reward and Decision Making

Volumn , Issue , 2009, Pages 399-425

  Doll, Bradley B ^a   Frank, Michael J ^a  

^a BROWN UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 78649683732     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1016/B978-0-12-374620-7.00019-4     Document Type: Chapter

References (153)

1. Wilson C.J., Callaway J.C. Coupled oscillator model of the dopaminergic neuron of the substantia nigra. J. Neurophysiol. 2000, 83:3084.
2. Daw N.D., Niv Y., Dayan P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 2005, 8:1704-1711.
3. Gurney K., Prescott T.J., Redgrave P. A computational model of action selection in the basal ganglia. I. A new functional anatomy. Biol. Cybern. 2001, 84:401-410.
4. Frank M.J., Loughry B., O'Reilly R.C. Interactions between the frontal cortex and basal ganglia in working memory: a computational model. Cogn. Affect. Behav. Neurosci. 2001, 1:137-160.
5. Houk J.C. Agents of the mind. Biol. Cybern. 2005, 92:427-437.
6. Mink J.W. The basal ganglia: focused selection and inhibition of competing motor program. Prog. Neurobiol. 1996, 50:381-425.
7. Alexander G.E., DeLong M.R., Strick P.L. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 1986, 9:357-381.
8. Alexander G.E., Crutcher M.D. Functional architecture of basal ganglia cicuits: neural substrates of parallel processing. Trends Neurosci. 1990, 13:266-271.
9. Albin R.L., Young A.B., Penney J.B. The functional anatomy of basal ganglia disorders. Trends Neurosci. 1989, 12:366-375.
10. Sato F., Lavallée P., Lévesque M., Parent A. Single-axon tracing study of neurons of the external segment of the globus pallidus in primate. J. Comp. Neurol. 2000, 417:17-31.
11. Cools R., Barker R.A., Sahakian B.J., Robbins T.W. Mechanisms of cognitive set flexibility in Parkinson's disease. Brain 2001, 124:2503-2512.
12. Montague P.R., Dayan P., Sejnowski T.J. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 1996, 16:1936-1947.
13. Schultz W. Multiple dopamine functions at different time courses. Annu. Rev. Neurosci. 2007, 30:259-288.
14. O'Reilly R.C., Frank M.J., Hazy T.E., Watz B. PVLV: the primary value and learned value Pavlovian learning algorithm. Behav. Neurosci. 2007, 121:31-49.
15. Sutton R.S., Barto A.G. Reinforcement Learning: An Introduction 1998, MIT Press, Cambridge, MA.
16. O'Doherty J., Dayan P., Schultz J., Deichmann R., Friston K., Dolan R.J. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 2004, 304:452-454.
17. Ince E., Ciliax B.J., Levey A.I. Differential expression of D and D dopamine and m4 muscarinic acteylcholine receptor proteins in identified striatonigral neurons. Synapse 1997, 27:257-366.
18. Aizman O., Brismar H., Uhlen P., Zettergren E., Levet A.I., Forssberg H., Greengard P., Aperia A. Anatomical and physiological evidence for D1 and D2 dopamine receptor colocalization in neostriatal neurons. Nat. Neurosci. 2000, 3:226-230.
19. Surmeier D.J., Ding J., Day M., Wang Z., Shen W. D1 and D2 dopamine-receptor modulation of striatal glutamatergic signaling in striatal medium spiny neurons. Trends Neurosci. 2007, 30:228-235.
20. Brown J.W., Bullock D., Grossberg S. How laminar frontal cortex and basal ganglia circuits interact to control planned and reactive saccades. Neural Netw. 2004, 17:471-510.
21. Frank M.J. Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and non-medicated Parkinsonism. J. Cogn. Neurosci. 2005, 17:51-72.
22. Kerr J.N., Wickens J.R. Dopamine D-1/D-5 receptor activation is required for long-term potentiation in the rat neostriatum in vitro. J. Neurophysiol. 2001, 85:117-124.
23. Calabresi P., Pisani A., Centonze D., Bernardi G. Synaptic plasticity and physiological interactions between dopamine and glutamate in the striatum. Neurosci. Biobehav. Rev. 1997, 21:519-523.
24. Satoh T., Nakai S., Sato T., Kimura M. Correlated coding of motivation and outcome of decision by dopamine neurons. J. Neurosci. 2003, 23:9913-9923.
25. Bayer H.M., Lau B., Glimcher P.W. Statistics of midbrain dopamine neuron spike trains in the awake primate. J. Neurophysiol. 2007, 98:1428-1439.
26. Shen W., Flajolet M., Greengard P., Surmeier D.J. Dichotomous dopaminergic control of striatal synaptic plasticity. Science 2008, 321:848-851.
27. Braver T.S., Cohen J.D. On the control of control: the role of dopamine in regulating prefrontal function and working memory. Control of Cognitive Processes: Attention and Performance XVIII 2000, 713-737. MIT Press, Cambridge, MA. S. Monsell, J. Driver (Eds.).
28. O'Reilly R.C., Frank M.J. Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Comput. 2006, 18:283-328.
29. Gruber A.J., Dayan P., Gutkin B.S., Solla S.A. Dopamine modulation in the basal ganglia locks the gate to working memory. J. Comput. Neurosci. 2006, 20:153-166.
30. O'Reilly R.C., Munakata Y. Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain 2000, MIT Press, Cambridge, MA.
31. Frank M.J. Hold your horses: a dynamic computational role for the subthalamic nucleus in decision making. Neural Netw. 2006, 19:1120-1136.
32. M.X. Cohen, M.J. Frank, Neurocomputational models of basal ganglia function in learning, memory and choice, Behav. Brain Res., 199 (2009) 141-156.
33. Frank M.J., Claus E.D. Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal. Psychol. Rev. 2006, 113:300-326.
34. Crutcher M.D., Alexander G.E. Movement-related neuronal activity selectively coding either direction or muscle pattern in three motor areas of the monkey. J. Neurophysiol. 1990, 64:151-163.
35. Alexander G.E., Crutcher M.D. Preparation for movement: neural representations of intended direction in three motor areas of the monkey. J. Neurophysiol. 1990, 64:133-150.
36. Nishi A., Snyder G.L., Greengard P. Bidirectional regulation of DARPP-32 phosphorylation by dopamine. J. Neurosci. 1997, 17:8147-8155.
37. Reynolds J.N.J., Hyland B.I., Wickens J.R. A cellular mechanism of reward-related learning. Nature 2001, 412:67-69.
38. Reynolds J.N., Wickens J.R. Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw. 2002, 15:507-521.
39. Beiser D.G., Hua S.E., Houk J.C. Network models of the basal ganglia. Curr. Opin. Neurobiol. 1997, 7:185.
40. Ashby F.G., Ennis J.M., Spiering b.J. A neurobiological theory of automaticity in perceptual categorization. Psychol. Rev. 2007, 114:632-656.
41. Frank M.J., Seeberger L.C., O'Reilly R.C. By carrot or by stick: cognitive reinforcement learning in Parkinsonism. Science 2004, 306:1940-1943.
42. Frank M.J., O'Reilly R.C. A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol. Behav. Neurosci. 2006, 120:497-517.
43. Cools R., Altamirano L., D'Esposito M. Reversal learning in Parkinson's disease depends on medication status and outcome valence. Neuropsychologia 2006, 44:1663-1673.
44. Frank M.J., Santamaria A., O'Reilly R.C., Willcutt E. Testing computational models of dopamine and noradrenaline dysfunction in attention deficit/hyperactivity disorder. Neuropsychopharmacology 2007, 32:1583-1599.
45. Frank M.J., Samanta J., Moustafa A.A., Sherman S.J. Hold your horses: impulsivity, deep brain stimulation and medication in Parkinsonism. Science 2007, 318:1309-1312.
46. Costa R.M., Gutierrez R., Araujo I.E., Coelho M.R.P., Kloth A.D., Gainetdinov R.R., Caron M.G., Nicolelis M.A.L., Simon S.A. Dopamine levels modulate the updating of tastant values. Genes Brain Behav. 2007, 6:314-320.
47. Kish S.J., Shannak K., Hornykiewicz O. Uneven pattern of dopamine loss in the striatum of patients with idiopathic Parkinson's disease. N. Engl. J. Med. 1988, 318:876-880.
48. Harden D.G., Grace A.A. Activation of dopamine cell firing by repeated l-DOPA administration to dopamine-depleted rats: its potential role in mediating the therapeutic response to l-DOPA treatment. J. Neurosci. 1995, 15:6157-6166.
49. Wightman R.M., Amatore C., Engstrom R.C., Hale P.D., Kristensen E.W., Kuhr W.G., May L.J. Real-time characterization of dopamine overflow and uptake in the rat striatum. Neuroscience 1988, 25:513-523.
50. Keller R.W., Kuhr W.G., Wightman R.M., Zigmond M.J. The effect of l-dopa on in vivo dopamine release from nigrostriatal bundle neurons. Brain Res. 1988, 447:191-194.
51. Cools R. Dopaminergic modulation of cognitive function-implications for l-DOPA treatment in Parkinson's disease. Neurosci. Biobehav. Rev. 2006, 30:1-23.
52. Molina J.A., Sáinz-Artiga M.J., Fraile A., Jimenez-Jimenez F.J., Villanueva C., Orti-Pareja M., Bermejo F. Pathologic gambling in Parkinson's disease: a behavioral manifestation of pharmacologic treatment?. Mov. Disord. 2000, 15:869-872.
53. Dodd M.L., Klos K.J., Bower J.H., Geda Y.E., Josephs K.A., Ahlskog J.E. Pathological gambling caused by drugs used to treat Parkinson disease. Arch. Neurol. 2005, 62:1377-1381.
54. Shen W., Tian X., Day M., Ulrich S., Tkatch T., Nathanson N.M., Surmeier D.J. Cholinergic modulation of Kir2 channels selectively elevates dendritic excitability in striatopallidal neurons. Nat. Neurosci. 2007, 10:1458-1466.
55. Knapska E., Kaczmarek L. A gene for neuronal plasticity in the mammalian brain: Zif268/Egr-1/NGFI-A/Krox-24/TIS8/ZENK?. Prog. Neurobiol. 2004, 74:183-211.
56. Carta A.R., Tronci E., Pinna A., Morelli M. Different responsiveness of striatonigral and striatopallidal neurons to l-DOPA after a subchronic intermittent l-DOPA treatment. Eur. J. Neurosci. 2005, 21:1196-1204.
57. Volkow N.D., Gur R.C., Wang G.J., Fowler J.S., Moberg P.J., Ding Y., Hitzemann R., Smith G., Logan J. Association between decline in brain dopamine activity with age and cognitive and motor impairment in healthy individuals. Am. J. Psychiatry 1998, 155:344-349.
58. Martin W.R., Palmer M.R., Patlak C.S., Calne D.B. Nigrostriatal function in humans studied with positron emission tomography. Ann. Neurol. 1989, 26:535-542.
59. Dyck C.H.v., Seibyl J.P., Malison R.T., Laruelle M., Zoghbi S.S., Baldwin R.M., Innis R.B. Age-related decline in dopamine transporters: analysis of striatal subregions, nonlinear effects, and hemispheric asymmetries. Am. J. Geriatr. Psychiatry 2002, 10:36-43.
60. Kraytsberg Y., Kudryavtseva E., McKee A.C., Geula C., Kowall N.W., Khrapko K. Mitochondrial DNA deletions are abundant and cause functional impairment in aged human substantia nigra neurons. Nat. Genet. 2006, 38:518-520.
61. Kaasinen V., Rinne J.O. Functional imaging studies of dopamine system and cognition in normal aging and Parkinson's disease. Neurosci. Biobehav. Rev. 2002, 26:785-793.
62. Bäckman L., Ginovart N., Dixon R.A., Wahlin T.R., Wahlin A., Halldin C., Farde L. Age-related cognitive deficits mediated by changes in the striatal dopamine system. Am. J. Psychiatry 2000, 157:635-637.
63. Frank M.J., Kong L. Learning to avoid in older age. Psychol. Aging 2008, 23:392-398.
64. A.A. Moustafa, M.X. Cohen, S.J. Sherman, M.J. Frank, A role for dopamine in temporal decision making and reward maximization in Parkinsonism, J. Neurosci. 28 (2008) 12294-12304.
65. Schoemaker H., Claustre Y., Fage D., et al. Neurochemical characteristics of amisulpiride, an atypical dopamine D2/D3 receptor antagonist with both presynaptic and limbic selectivity. J. Pharmacol. Exp. Ther. 1997, 280:83-97.
66. Grace A.A. Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: a hypothesis for the etiology of schizophrenia. Neuroscience 1991, 41:1-24.
67. Chen Y.-C.I., Choi J.-K., Andersen S.L., Rosen B.R., Jenkins B.G. Mapping dopamine D2/D3 receptor function using pharmacological magnetic resonance imaging. Psychopharmacology (Berl) 2005, 180:705-715.
68. Garris P.A., Budygin E.A., Phillips P.E.M., Venton B.J., Robinson D.L., Bergstrom B.P., Rebec G.V., Wightman R.M. A role for presynaptic mechanisms in the actions of nomifensine and haloperidol. Neuroscience 2003, 118:819-829.
69. Moghaddam B., Bunney B.S. Acute effects of typical and atypical antipsychotic drugs on the release of dopamine from prefrontal cortex, nucleus accumbens, and stiratum in the rat: an in vivo microdialysis study. J. Neurochem. 1990, 54:1755-1760.
70. Pehek E.A. Comparison of effects of haloperidol administration on amphetamine-stimulated dopamine release in the rat medial prefrontal cortex and dorsal striatum. J. Pharmacol. Exp. Ther. 1999, 289:14-23.
71. Westerink B.H.C. Can antipsychotic drugs be classified by their effects on a particular group of dopamine neurons in the brain?. Eur. J. Pharmacol. 2002, 455:1-18.
72. Wu Q., Reith M.E.A., Walker Q.D., Kuhn C.M., Caroll F.I., Garris P.A. Concurrent autoreceptor-mediated control of dopamine release and uptake during neurotransmission: an in vivo voltammetric study. J. Neurosci. 2002, 22:6272-6281.
73. Pizzagalli D.A., Evins A.E., Schetter E.C., Frank M.J., Pajtas P.E., Santesso D.L., Culhane M. Single dose of a dopamine agonist impairs reinforcement learning in humans: behavioral evidence from a laboratory-based measure of reward responsiveness. Psychopharmacology (Berl) 2008, 196:221-232.
74. D.L. Santesso, A.E. Evins, M.J. Frank, E.C. Schetter, R. Bogdan, D.A. Pizzagalli, Single dose of a dopamine agonist impairs reinforcement learning in humans: Evidence from event-related potentials and computational modeling of striatal-cortical function. Hum. Brain Mapp. (2008).
75. Kimberg D.Y., D'Esposito M., Farah M.J. Effects of bromocriptine on human subjects depend on working memory capacity. Neuroreport 1997, 8:3581-3585.
76. Gibbs S.E., D'Esposito M. A functional MRI study of the effects of bromocriptine, a dopamine receptor agonist, on component processes of working memory. Psychopharmacology (Berl) 2005, 1-10.
77. Mattay V.S., Callicott J.H., Bertolino A., Heaton I., Frank J.A., Coppola R., Berman K.F., Goldberg T.E., Weinberger D. Effects of dextroamphetamine on cognitive performance and cortical activation. Neuroimage 2000, 12:268-275.
78. Cools R., Gibbs S.E., Miyakawa A., Jagust W., D'Esposito M. Working memory capacity predicts dopamine synthesis capacity in the human striatum. J. Neurosci. 2008, 28:1208-1212.
79. R. Cools, M.J. Frank, S.E. Gibbs, A. Miyakawa, W. Jagust, M. D'Esposito, Striatal dopamine predicts outcome-specific reversal learning and its sensitivity to dopaminergic drug administration, J. Neurosci. 29 (2009) 1538-43.
80. Pessiglione M., Seymour B., Flandin G., Dolan R.J., Frith C.D. Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 2006, 442:1042-1045.
81. Dayan P., Huys Q.J.M. Serotonin, inhibition, and negative mood. PLoS Comput. Biol. 2008, 4:e4.
82. Kim H., Shimojo S., O'Doherty J.P. Is avoiding an aversive outcome rewarding? Neural substrates of avoidance learning in the human brain. PLoS Biol. 2006, 4:e233.
83. Frank M.J., Woroch B.S., Curran T. Error-related negativity predicts reinforcement learning and conflict biases. Neuron 2005, 47:495-501.
84. Frank M.J., D'Lauro C., Curran T. Cross-task individual differences in error processing: neural, electrophysiological and genetic components. Cogn. Affect. Behav. Neurosci. 2007, 7:297-308.
85. Klein T.A., Neumann J., Reuter M., Hennig J., Cramon D.Y.v., Ullsperger M. Genetically determined differences in learning from errors. Science 2007, 318:1642-1645.
86. Frank M.J., Moustafa A.A., Haughey H., Curran T., Hutchison K. Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc. Natl. Acad. Sci. USA 2007, 104:16311-16316.
87. Meyer-Lindenberg A., Straub R.E., Lipska B.K., et al. Genetic evidence implicating DARPP-32 in human frontostriatal structure, function, and cognition. J. Clin. Invest. 2007, 117:672-682.
88. Calabresi P., Gubellini P., Centonze D., Picconi B., Bernardi G., Chergui K., Svenningsson P., Fienberg A.A., Greengard P. Dopamine and cAMP-regulated phosphoprotein 32 kDa controls both striatal long-term depression and long-term potentiation, opposing forms of synaptic plasticity. J. Neurosci. 2000, 20:8443-8451.
89. Stipanovich A., Valjent E., Matamales M., et al. A phosphatase cascade by which rewarding stimuli control nucleosomal response. Nature 2008, 453:879-884.
90. Hirvonen M., Laakso K., Rinne J.O., Pohjalainen T., Hietala J. C957T polymorphism of the dopamine D2 receptor (DRD2) gene affects striatal DRD2 availability in vivo (corrigendum). Mol. Psychiatry 2005, 10:889.
91. M.J. Frank, B.B. Doll, J. Oas-Terpstra, F. Moreno, The neurogenetics of exploration and exploitation: prefrontal and striatal depaminergic components. Natr. Neurosci. (In press).
92. Egan M.F., Goldberg T.E., Kolachana B.S., Callicott J.H., Mazzanti C.M., Straub R.E., Goldman D., Weinberger D. Effect of COMT Val met genotype on frontal lobe function and risk for schizophrenia. Proc. Natl. Acad. Sci. USA 2001, 98:6917-6922.
93. Tunbridge E.M., Bannerman D.M., Sharp T., Harrison P.J. Catechol-O-methyltransferase inhibition improves set-shifting performance and elevates stimulated dopamine release in the rat prefrontal cortex. J. Neurosci. 2004, 24:5331-5335.
94. Slifstein M., Kolachana B., Simpson E.H., Tabares P., Cheng B., Duvall M., Frankle W.G., Weinberger D.R., Laruelle M., Abi-Dargham A. COMT genotype predicts cortical-limbic D1 receptor availability measured with [11C]NNC112 and PET. Mol. Psychiatry 2008, 13:821-827.
95. Gogos J.A., Morgan M., Luine V., Santha M., Ogawa S., Pfaff D., Karayiorgou M. Catechol-O-methyltransferase-deficient mice exhibit sexually dimorphic changes in catecholamine levels and behavior. Proc. Natl. Acad. Sci. USA 1998, 95:9991-9996.
96. Durstewitz D., Seamans J.K., Sejnowski T.J. Dopamine-mediated stabilization of delay-period activity in a network model of prefrontal cortex. J. Neurophysiol. 2000, 83:1733-1750.
97. Tunbridge E.M., Harrison P.J., Weinberger D.R. Catechol-O-methyltransferase, cognition, and psychosis: Val158Met and beyond. Biol. Psychiatry 2006, 60:141-151.
98. Seamans J.K., Gorelova N., Durstewitz D., Yang C.R. Bidirectional dopamine modulation of GABAergic inhibition in prefrontal cortical pyramidal neurons. J. Neurosci. 2001, 21:3628-3638.
99. Cragg S.J., Hille C.J., Greenfield S.A. Functional domains in dorsal striatum of the nonhuman primate are defined by the dynamic behavior of dopamine. J. Neurosci. 2002, 22:5705-5712.
100. Arbuthnott G.W., Wickens J. Space, time and dopamine. Trends Neurosci. 2007, 30:62-69.
101. Seamans J.K., Yang C.R. The principal features and mechanisms of dopamine modulation in the prefrontal cortex. Prog. Neurobiol. 2004, 74:1-57.
102. Aston-Jones G., Cohen J.D. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 2005, 28:403-450.
103. Yu A.J., Dayan P. Uncertainty, neuromodulation, and attention. Neuron 2005, 46:681-692.
104. Daw N.D., O'Doherty J.P., Dayan P., Seymour B., Dolan R.J. Cortical substrates for exploratory decisions in humans. Nature 2006, 441:876-879.
105. Cohen J.D., McClure S.M., Yu A.J. Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philos. Trans. R. Soc. Lond. B Biol. Sci. 2007, 362:933-942.
106. Dayan P., Sejnowski T.J. Exploration bonuses and dual control. Mach. Learn. 1996, 25:5-22.
107. L.P. Kaelbling, A.R. Cassandra, J.A. Kurien, Acting under uncertainty: discrete Bayesian models for mobile-robot navigation, in: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, 1996.
108. Li J., McClure S.M., King-Casas B., Montague P.R. Policy adjustment in a dynamic economic game. PLoS ONE 2006, 1:e103.
109. Wittmann B.C., Daw N.D., Seymour B., Dolan R.J. Striatal activity underlies novelty-based choice in humans. Neuron 2008, 58:967-973.
110. Behrens T.E.J., Woolrich M.W., Walton M.E., Rushworth M.F.S. Learning the value of information in an uncertain world. Nat. Neurosci. 2007, 10:1214-1221.
111. O'Doherty J.P., Hampton A., Kim H. Model-based fMRI and its application to reward learning and decision making. Ann. NY Acad. Sci. 2007, 1104:35-53.
112. Bogacz R., Gurney K. The basal ganglia and cortex implement optimal decision making between alternative actions. Neural Comput. 2007, 19:442-477.
113. Nambu A., Tokuno H., Hamada I., Kita H., Imanishi M., Akazawa T., Ikeuchi Y., Hasegawa N. Excitatory cortical inputs to pallidal neurons via the subthalamic nucleus in the monkey. J. Neurophysiol. 2000, 84:289-300.
114. Parent A., Hazrati L. Functional anatomy of the basal ganglia. II. The place of subthalamic nucleus and external pallidum in basal ganglia circuitry. Brain Res. Rev. 1995, 20:128-154.
115. Aron A.R., Behrens T.E., Smith S., Frank M.J., Poldrack R.A. Triangulating a cognitive control network using diffusion-weighted magnetic resonance imaging (MRI) and functional MRI. J. Neurosci. 2007, 27:3743-3752.
116. Isoda M., Hikosaka O. Role for subthalamic nucleus neurons in switching from automatic to controlled eye movement. J. Neurosci. 2008, 28:7209-7218.
117. Baunez C., Humby T., Eagle D.M., Ryan L.J., Dunnett S.B., Robbins T.W. Effects of STN lesions on simple vs choice reaction time tasks in the rat: preserved motor readiness, but impaired response selection. Eur. J. Neurosci. 2001, 13:1609-1616.
118. Baunez C., Robbins T.W. Bilateral lesions of the subthalamic nucleus induce multiple deficits in an attentional task in rats. Eur. J. Neurosci. 1997, 9:2086-2099.
119. Benabid A.L. Deep brain stimulation for Parkinson's disease. Curr. Opin. Neurobiol. 2003, 13:696-706.
120. Frank M.J., Scheres A., Sherman S.J. Understanding decision making deficits in neurological conditions: insights from models of natural action selection. Philos. Trans. R. Soc. Lond. B Biol. Sci. 2007, 362:1641-1654.
121. Diagnostic and Statistical Manual of Mental Disorders 1994, APA, American Psychiatric Press, Washington, DC. fourth ed.
122. Sagvolden T., Johansen E.B., Aase H., Russell V.A. A dynamic developmental theory of attention-deficit/hyperactivity disorder (ADHD) predominantly hyperactive/impulsive and combined subtypes. Behav. Brain Sci. 2005, 28:397-419. discussion 419-468.
123. Biederman J., Spencer T. Attention-deficit/hyperactivity disorder (ADHD) as a noradrenergic disorder. Biol. Psychiatry 1999, 46:1234-1242.
124. Madras B.K., Miller G.M., Fischman A.J. The dopamine transporter and attention-deficit/hyperactivity disorder. Biol. Psychiatry 2005, 57:1397-1409.
125. Aston-Jones G., Cohen J.D. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 2005, 28:403-450.
126. S.M. McClure, M.S. Gilzenrat, J.D. Cohen, An exploration-exploitation model based on norepinepherine and dopamine activity, in: Advances in Neural Information Processing Systems 18, 2005.
127. Usher M., Cohen J.D., Servan-Schreiber D., Rajkowski J., Aston-Jones G. The role of locus coeruleus in the regulation of cognitive performance. Science 1999, 283:549-554.
128. Russell V.A., Wiggins T.M. Increased glutamate-stimulated norepinephrine release from prefrontal cortex slices of spontaneously hypertensive rats. Metab. Brain Dis. 2000, 15:297-304.
129. Russell V., Allie S., Wiggins T. Increased noradrenergic activity in prefrontal cortex slices of an animal model for attention-deficit hyperactivity disorder-the spontaneously hypertensive rat. Behav. Brain Res. 2000, 117:69-74.
130. Rangel A., Camerer C., Montague P.R. A framework for studying the neurobiology of value-based decision making. Nat. Rev. Neurosci. 2008, 9:545-556.
131. Sloman S.A. The empirical case for two systems of reasoning. Psychol. Bull. 1996, 119:3-22.
132. Rule-Governed Behavior: Cognition, Contingencies, and Instructional Control 1989, Plenum Press, New York. S.C. Hayes (Ed.).
133. R.K. Jessup, A.J. Bishara, J.R. Busemeyer, Feedback produces divergence from prospect theory in descriptive choice, Psychol. Sci. 10 (2008) 1015-22.
134. G. Biele, J. Rieskamp, R. Gonzalez, Computational models for the combination of advice and individual learning, Cogn. Sci. in press.
135. B.B. Doll, W.J. Jacobs, A.G. Sanfey, M.J. Frank, Instructional control of reinforcement learning: a behavioral and neurocomputational investigation, Brain Res., Submitted.
136. Hayes S.C. Rule governance: basic behavioral research and applied implications. Curr. Dir. Psychol. Sci. 1993, 2:193-197.
137. Kaufman A., Baron A., Kopp R. Some effects of instructions on human operant behavior. Psychon Monogr. Suppl. 1966, 1:243-250.
138. Galizio M. Contingency-shaped and rule-governed behavior: instructional control of human loss avoidance. J. Exp. Anal. Behav. 1979, 31:53-70.
139. O'Reilly R.C., Rudy J.W. Conjunctive representations in learning and memory: principles of cortical and hippocampal function. Psychol. Rev. 2001, 108:311-345.
140. Bechara A., Damasio H., Tranel D., Anderson S.W. Dissociation of working memory from decision making within the human prefrontal cortex. J. Neurosci. 1998, 18:428-437.
141. Rolls E.T. Convergence of sensory systems in the orbitofrontal cortex in primates and brain design for emotion. Anat. Rec. A Discov. Mol. Cell. Evol. Biol. 2004, 281A:1212-1225.
142. Tremblay L., Schultz W. Reward-related neuronal activity during Go-Nogo task performance in primate orbitofrontal cortex. J. Neurophysiol. 2000, 83:1864-1876.
143. Chudasama Y., Robbins T.W. Dissociable contributions of the orbitofrontal and infralimbic cortex to Pavlovian autoshaping and discrimination reversal learning: further evidence for the functional heterogeneity of the rodent frontal cortex. J. Neurosci. 2003, 23:8771-8780.
144. Jones B., Mishkin M. Limbic lesions and the problem of stimulus-reinforcement associations. Exp. Neurol. 1972, 36:362-377.
145. Rudebeck P.H., Walton M.E., Smyth A.N., Bannerman D.M., Rushworth M.F.S. Separate neural pathways process different decision costs. Nat. Neurosci. 2006, 9:1161-1168.
146. Chase H.W., Clark L., Myers C.E., Gluck M.A., Sahakian B.J., Bullmore E.T., Robbins T.W. The role of the orbitofrontal cortex in human discrimination learning. Neuropsychologia 2008, 46:1326-1337.
147. Roesch M.R., Olson C.R. Neuronal activity related to reward value and motivation in primate frontal cortex. Science 2004, 304:307-310.
148. O'Doherty J.P. Lights, camembert, action! The role of human orbitofrontal cortex in encoding stimuli, rewards, and choices. Ann. NY Acad. Sci. 2007, 1121:254-272.
149. Hazy T.E., Frank M.J., O'Reilly R.C. Banishing the homunculus: making working memory work. Neuroscience 2006, 139:105-118.
150. Hazy T.E., Frank M.J., O'Reilly R.C. Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system. Philos. Trans. R. Soc. Lond. B Biol. Sci. 2007, 139:105-118.
151. McNab F., Klingberg T. Prefrontal cortex and basal ganglia control access to working memory. Nat. Neurosci. 2008, 11:103-107.
152. Moustafa A.A., Sherman S.J., Frank M.J. A dopaminergic basis for working memory, learning, and attentional shifting in Parkinson's disease. Neuropsychologia 2008, 46:3144-3156.
153. Knowlton B.J., Mangels J.A., Squire L.R. A neostriatal habit learning system in humans. Science 1996, 273:1399-1402.