Anatomy of a decision: Striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal

Psychological Review

Volumn 113, Issue 2, 2006, Pages 300-326

  Frank, Michael J ^a,b   Claus, Eric D ^a  

^a UNIVERSITY OF COLORADO   (United States)

^b UNIVERSITY OF ARIZONA   (United States)

Author keywords

Basal ganglia; Decision making; Neural network; Orbitofrontal cortex; Reinforcement learning

Indexed keywords

ANIMAL; ARTICLE; ARTIFICIAL NEURAL NETWORK; CORPUS STRIATUM; DECISION MAKING; FEEDBACK SYSTEM; FRONTAL LOBE; HUMAN; LEARNING; NERVE CELL NETWORK; PHYSIOLOGY; REINFORCEMENT;

ANIMALS; ASSOCIATION LEARNING; CORPUS STRIATUM; DECISION MAKING; FEEDBACK; FRONTAL LOBE; HUMANS; NERVE NET; NEURAL NETWORKS (COMPUTER); PROBABILITY LEARNING; REINFORCEMENT (PSYCHOLOGY); REVERSAL LEARNING;

EID: 33744550336     PISSN: 0033295X     EISSN: None     Source Type: Journal    
DOI: 10.1037/0033-295X.113.2.300     Document Type: Article

References (209)

1. Adell, A., & Artigas, F. (1999). Regulation of the release of 5-hydroxytryptamine in the median raphe nucleus of the rat by catecholaminergic afferents. European Journal of Neuroscience, 11, 2305-2311.
2. Alexander, G. E., & Crutcher, M. D. (1990a). Functional architecture of basal ganglia circuits: Neural substrates of parallel processing. Trends in Neuroscience, 13, 266-271.
3. Alexander, G. E., & Crutcher, M. D. (1990b). Preparation for movement: Neural representations of intended direction in three motor areas of the monkey. Journal of Neuurophysiology, 64, 133-150.
4. Alexander, G., Crutcher, M., & DeLong. M. (1990). Basal ganglia-thalamocortical circuits: Parallel substrates for motor, oculomotor, "prefrontal" and "limbic" functions. In II. Uylings, C. Van Eden, J. De Bruin, M. Corner, & M. Feenstra (Eds.), The prefrontal cortex: Its structure, function, and pathology (pp. 119-146). Amsterdam: Elsevier.
5. Anderson, A. K., Christoff, K., Stappen, I., Panitz, D., Ghahremani, D. G., Glover, G., et al. (2003). Dissociated neural representations of intensity and valence in human olfaction. Nature Neuroscience, 6, 196-202.
6. Annett, L. E., McGregor. A., & Robbins, T. W. (1989). The effects of ibotenic acid lesions of the nucleus accumbens on spatial learning and extinction in the rat. Behavioral Brain Research, 31, 231-242.
7. Arana, F. S., Parkinson, J. A., Hinton, E., Holland, A. J., Owen, A. M., & Roberts, A. C. (2003). Dissociable contributions of the human amygdala and orbitofrontal cortex to incentive motivation and goal selection. Journal of Neuroscience, 23, 9632-9638.
8. Atallah, II. E., Frank, M. J., & O'Reilly, R. C. (2004). Hippocampus, cortex and basal ganglia: Insights from computational models of complementary learning systems. Neurobiology of learning and Memory, 82, 253-267.
9. Aubert, I., Ghorayeb, I., Normand, E., & Bloch, B. (2000). Phenotypical characterization of the neurons expressing the D1 and D2 dopamine receptors in the monkey striatum. Journal of Comparative Neurology, 418, 22-32.
10. Balleine, B. W., & Dickinson, A. (1998). Goal-directed instrumental action: Contingency and incentive learning and their cortical substrates. Neuropharmacology, 37, 407-419.
11. Baxter, M. G., & Murray, E. A. (2002). The amygdala and reward. Nature Reviews Neuroscience, 3, 563-572.
12. Baxter, M. G., Parker, A., Lindner, C. C., Izquierdo, A. D., & Murray, E. A. (2000). Control of response selection by reinforcer value requires interaction of amygdala and orbital prefrontal cortex. Journal of Neuroscience, 20, 4311-4319.
13. Bayer, H. M. (2004). A role for the substantia nigra in learning and motor control. Unpublished doctoral thesis, New York University, New York.
14. Bayer, H. M., & Glimcher, P. W. (2005). Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron, 47, 129-141.
15. Bechara, A., & Damasio, A. R. (2005). The somatic marker hypothesis: A neural theory of economic decision. Games and Economic Behavior, 52, 336-372.
16. Bechara, A., Damasio, H., Damasio, A. R., & Anderson, S. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex. Cognition, 50, 7-15.
17. Bechara. A., Damasio, H., Tranel, D., & Anderson, S. W. (1998). Dissociation of working memory from decision making within the human prefrontal cortex. Journal of Neuroscience, 18, 428-437.
18. Bechara, A., Dolan, S., & Hindes, A. (2002). Decision-making and addiction (Part II): Myopia for the future or hypersensitivity to reward? Neuropsychologia, 40, 1690-1705.
19. Bechara, A., Tranel, D., Damasio, H., & Damasio, A. R. (1996). Failure to respond autonomically to anticipated future outcomes following damage to prefrontal cortex. Cerebral Cortex, 6, 215-225.
20. Beiser, D. G., & Houk, J. C. (1998). Model of cortical-basal ganglionic processing: Encoding the serial order of sensory events. Journal of Neurophysiology, 79, 3168-3188.
21. Bickel, W. K., & Marsch, L. A. (2001). Toward a behavioral economic understanding of drug dependence: Delay discounting processes. Addiction, 96, 73-86.
22. Bokura, H., Yamaguchi, S., & Kobayashi, S. (2001). Electrophysiological correlates for response inhibition in a go/no-go task. Clinical Neurophysiology, 112, 2224-2232.
23. Bolla, K. I., Eldreth, D. A., London, E. D., Kiehl, K. A., Mouratidis, M., Contoreggi, C., et al. (2003). Orbitofrontal cortex dysfunction in abstinent cocaine abusers performing a decision-making task. NeuroImage, 19, 1085-1094.
24. Brown, J., Bullock, D., & Grossberg, S. (1999). How the basal ganglia use parallel excitatory and inhibitory learning pathways to selectively respond to unexpected rewarding cues. Journal of Neuroscience, 19, 10502-10511.
25. Brown, J. W., Bullock, D., & Grossberg, S. (2004). How laminar frontal cortex and basal ganglia circuits interact to control planned and reactive saccades. Neural Networks, 17, 471-510.
26. Busemeyer, J. R., & Stout, J. C. (2002). A contribution of cognitive decision models to clinical assessment: Decomposing performance on the Bechara gambling task. Psychological Assessment, 14, 253-262.
27. Calabresi, P., Saiardi, A., Pisani, A., Baik, J. H., Centonze, D., Mercuri, N. B., et al. (1997). Abnormal synaptic plasticity in the striatum of mice lacking dopamine D2 receptors. Journal of Neuroscience, 17, 4536-4544.
28. Carmichael, S. T., & Price, J. L. (1995). Limbic connections of the orbital and medial prefrontal cortex in macaque monkeys. Journal of Comparative Neurology, 363, 615-641.
29. Cavada, C., Company, T., Tejedor, J., Cruz-Rizzolo, R. J., & Reinoso-Suarez, F. (2000). The anatomical connections of the macaque monkey orbitofrontal cortex: A review. Cerebral Cortex, 10, 220-242.
30. Centonze, D., Picconi, B., Gubellini, P., Bernardi, G., & Calabresi, P. (2001). Dopaminergic control of synaptic plasticity in the dorsal striatum. European Journal of Neuroscience, 13, 1071-1077.
31. Chudasama, Y., & Robbins, T. W. (2003). Dissociable contributions of the orbitofrontal and infralimbic cortex to Pavlovian autoshaping and discrimination reversal learning: Further evidence for the functional heterogeneity of the rodent frontal cortex. Journal of Neuroscience, 23, 8771-8780.
32. Clarke, H. F., Walker, S. C., Crofts, H. S., Dalley, J. W., Robbins, T. W., & Roberts, A. C. (2005). Prefrontal serotonin depletion affects reversal learning but not attentional set shifting. Journal of Neuroscience, 25, 532-538.
33. Claus, E. D. (2005). Rewards and punishments as determinants of behavior: Using personality to understand mechanisms of decision making. Unpublished master's thesis, University of Colorado at Boulder.
34. Cohen, J. D., Dunbar, K., & McClelland, J. L. (1990). On the control of automatic processes: A parallel distributed processing model of the Stroop effect. Psychological Review, 97, 332-361.
35. Cohen, M. X., Heller, A. S., & Ranganath, C. (2005). Functional connectivity with anterior cingulate and orbitofrontal cortices during decision-making. Brain Research, 23, 61-70.
36. Cools, R. (2005). Dopaminergic modulation of cognitive function-Implications for 1-DOPA treatment in Parkinson"s disease. Neuroscience and Biobehavioral Reviews, 30, 1-23.
37. Cools. R., Barker, R. A., Sahakian, B. J., & Robbins, T. W. (2001). Enhanced or impaired cognitive function in Parkinson's disease as a function of dopaminergic medication and task demands. Cerebral Cortex, 11, 1136-1143.
38. Cools, R., Barker, R. A., Sahakian, B. J., & Robbins, T. W. (2003). L-Dopa medication remediates cognitive inflexibility, but increases impulsivity in patients with Parkinson's disease. Neuropsychologia, 41, 1431-1441.
39. Cools. R., Clark, L., Owen, A. M., & Robbins, T. W. (2002). Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. Journal of Neuroscience, 22, 4563-4567.
40. Creese, I., Sibley, D. R., Hamblin, M. W., & Leff, S. E. (1983). The classification of dopamine receptors: Relationship to radioligand binding. Annual Review of Neuroscience, 6, 43-71.
41. Crutcher, M. D., & Alexander. G. E. (1990). Movement-related neuronal activity selectively coding either direction or muscle pattern in three motor areas of the monkey. Journal of Neurophysiology, 64, 151-163.
42. Damasio, A. R. (1994). Descartes' error: Emotion, reason and the human brain. New York: Avon Books.
43. Davidson, R. J. (1992). Anterior cerebral asymmetry and the nature of emotion. Brain and Cognition, 20, 125-151.
44. Davidson, R. J., & Irwin, W. (1999). The functional neuroanatomy of emotion and affective style. Trends in Cognitive Sciences, 3, 11-21.
45. Daw, N. D., Kakade, S., & Dayan, P. (2002). Opponent interactions between serotonin and dopamine. Neural Networks, 15, 603-616.
46. Deakin, J. F. W. (1983). Roles of brain serotonergic neurons in escape, avoidance and other behaviors. Journal of Psychopharmacology, 43, 563-577.
47. Deco, G., & Rolls, E. T. (2005). Synaptic and spiking dynamics underlying reward reversal in the orbitofrontal cortex. Cerebral Cortex, 15, 15-30.
48. Delgado, M. R., Miller, M. M., Inati, S., & Phelps, E. A. (2005). An fMRI study of reward-related probability learning. NeuroImage, 24, 862-873.
49. Di Cara, B., Samuel, D., Salin, P., Kerkerian-Le Goff, L., & Daszuta, A. (2003). Serotonergic regulation of the GABAergic transmission in the rat basal ganglia. Synapse, 50, 144-150.
50. Doder, M., Rabiner, E. A., Turjanski, N., Lees, A. J., & Brooks, D. J. (2003). Tremor in Parkinson's disease and serotonergic dysfunction: An 11C-WAY 100635 PET study. Neurology, 60, 601-605.
51. Dodd, M.L., Klos, K.J., Bower, J.H., Geda, Y.E., Josephs, K.A. & Ahlskog, J.E. (2005). Pathological gambling caused by drugs used to treat Parkinson disease. Archives of Neurology, 62, 1377-1.381
52. Durstewitz, D., Seamans, J. K., & Sejnowski, T. J. (2000). Dopamine-mediated stabilization of delay-period activity in a network model of prefrontal cortex. Journal of Neurophysiology, 83, 1733-1750.
53. Elliott, R., Dolan. R. J., & Frith, C. D. (2000). Dissociable functions in the medial and lateral orbitofrontal cortex: Evidence from human neuroimaging studies. Cerebral Cortex, 10, 308-317.
54. Elman, J. L. (1990). Finding structure in time. Cognitive Science, 14, 179-211.
55. Ernst, M., Bolla, K., Mouratidis, M., Contoreggi, C., Matochik, J. A., Kurian, V., et al. (2002). Decision-making in a risk-taking task: A PET study. Neuropsychopharmacology, 26, 682-691.
56. Estes, W. K. (1961). A descriptive approach to the dynamics of choice behavior. Behavioral Science, 6, 177-184.
57. Evers, E. A., Cools, R., Clark, L., van der Veen, F. M., Jolles, J., Sahakian, B. J., & Robbins, T. W. (2005). Serotonergic modulation of prefrontal cortex during negative feedback in probabilistic reversal learning. Neuropsychopharmacology, 30, 1138-1147.
58. Falgatter, A. J., Herrmann, M. J., Roemmler, J., Ehlis, A. C., Wagener, A., Heidrich, A., et al. (2004). Allelic variation of serotonin transporter function modulates the brain electrical response for error processing. Neuropsychopharmacology, 29, 1506-1511.
59. Fellows, L. K., & Farah, M. J. (2003). Ventromedial frontal cortex mediates affective shifting in humans: Evidence from a reversal learning paradigm. Brain, 126, 1830-1837.
60. Finch, D. M. (1999). Plasticity of responses to synaptic inputs in rat ventral striatal neurons after repeated administration of the dopamine D2 antagonist raclopride. Synapse, 31, 297-301.
61. Frank, M. J. (2005). Dynamic dopamine modulation in the basal ganglia: A neurocomputational account of cognitive deficits in medicated and non-medicated Parkinsonism. Journal of Cognitive Neuroscience, 17, 51-72.
62. Frank, M. J. (in press). Hold your horses: A dynamic computational role for the subthalamic nucleus in decision making. Neural Networks (2006 special issue on the neurobiology of decision making).
63. Frank, M. J., Loughry, B., & O'Reilly, R. C. (2001). Interactions between the frontal cortex and basal ganglia in working memory: A computational model. Cognitive, Affective, and Behavioral Neuroscience, 1, 137-160.
64. Frank, M. J., & O'Reilly, R. C. (in press). A mechanistic account of striatal dopamine function in cognition: Psychopharmacological studies with cabergoline and haloperidol. Behavioral Neuroscience.
65. Frank, M. J., O'Reilly, R. C., & Curran, T. (in press). When memory fails, intuition reigns: Midazolam enhances implicit inference in humans. Psychological Science.
66. Frank, M. J., Rudy, J. W., & O'Reilly, R. C. (2003). Transitivity, flexibility, conjunctive representations and the hippocampus: II. A computational analysis. Hippocampus, 13, 341-354.
67. Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004, December 10). By carrot or by stick: Cognitive reinforcement learning in Parkinsonism. Science, 306, 1940-1943.
68. Frank, M. J., Woroch, B. S., & Curran, T. (2005). Error-related negativity predicts reinforcement learning and conflict biases. Neuron, 47, 495-501.
69. Fuster, J. M. (1989). The prefrontal cortex: Anatomy, physiology and neuropsychology of the frontal lobe. New York: Raven Press.
70. Fuster, J. M. (1997). The prefrontal cortex: Anatomy, physiology and neuropsychology of the frontal lobe (3rd ed.). New York: Lippincott-Raven.
71. Gallagher, M., McMahan. R. W., & Schoenbaum, G. (1999). Orbitofrontal cortex and representation of incentive value in associate learning. Journal of Neuroscience, 19, 6610-6614.
72. Gerfen, C. R. (1992). The neostriatal mosaic: Multiple levels of compartmental organization in the basal ganglia. Annual Review of Neuroscience, 15, 285-320.
73. Gerfen, C. R. (2000). Molecular effects of dopamine on striatal projection pathways. Trends in Neurosciences, 23, S64-S70.
74. Goldman-Rakic, P. S. (1995). Architecture of the prefrontal cortex and the central executive. Annals of the New York Academy of Sciences, 769, 71-83.
75. Gonzalez, C., Danda, J., Koshino, H., & Just, M. (2005). The framing effect and risky decisions: Examining cognitive functions with fMRI. Journal of Economic Psychology, 26, 1-20.
76. Goto, Y., & Grace, A. A. (2005). Dopaminergic modulation of limbic and cortical drive of nucleus accumbens in goal-directed behavior. Nature Neuroscience, 8, 805-812.
77. Gottfried, J. A., O'Doherty, J., & Dolan, R. J. (2002). Appetitive and aversive olfactory learning in humans studied using event-related functional magnetic resonance imaging. Journal of Neuroscience, 22, 10829-10837.
78. Gottfried, J. A., O'Doherty, J., & Dolan, R. J. (2003, August 22). Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science, 301, 1104-1107.
79. Grant, S., Contoreggi, C., & London, E. D. (2000). Drug abusers show impaired performance in a laboratory test of decision making. Neuropsychologia, 38, 1180-1187.
80. Gurney, K., Prescott, T. J., & Redgrave, P. (2001). A computational model of action selection in the basal ganglia: I. A new functional anatomy. Biological Cybernetics, 84, 401-410.
81. Haber, S., Kunishio, K., Mizobuchi, M., & Lynd-Dalta, E. (1995). The orbital and medial prefrontal circuit through the primate basal ganglia. Journal of Neuroscience, 15, 4851-4867.
82. Hazy, T. E., Frank, M. J., & O'Reilly, R. C. (in press). Banishing the homunculus: Making working memory work. Neuroscience.
83. Hernandez-Lopez, S., Bargas, J., Surmeier, D. J., Reyes, A., & Galarraga, E. (1997). Dl receptor activation enhances evoked discharge in neostriatal medium spiny neurons by modulating an L-type Ca2+ conductance. Journal of Neuroscience, 17, 3334-3342.
84. Hernandez-Lopez, S., Tkatch, T., Perez-Garci, E., Galarraga, E., Bargas, J., Hamm, H., & Surmeier, D. J. (2000). D2 dopamine receptors in striatal medium spiny neurons reduce L-type Ca2+ currents and excitability via a novel PLCβl-IP3-calcineurin-signaling cascade. Journal of Neuroscience, 20, 8987-8995.
85. Hikosaka, K., & Watanabe, M. (2000). Delay activity of orbital and lateral prefrontal neurons of the monkey varying with different rewards. Cerebral Cortex, 10, 263-271.
86. Hikosaka, K., & Watanabe, M. (2004). Long- and short-range reward expectancy in the primate orbitofrontal cortex. European Journal of Neuroscience, 19, 1046-1054.
87. Hikosaka, O. (1994). Role of basal ganglia in control of innate movements, learned behaviour and cognition. In G. Percheron, J. McKenzie. & J. Feger (Eds.), The basal ganglia IV: New ideas and data on structure and function (pp. 589-596). New York: Plenum Press.
88. Hikosaka, O. (1.998). Neural systems for control of voluntary action-A hypothesis. Advances in Biophysics, 35, 81-102.
89. Hinson, J. M., & Staddon, J. E. (1983). Matching, maximizing and hill climbing. Journal of the Experimental Analysis of Behavior, 40, 321-331.
90. Holland, P. C., & Gallagher, M. (2004). Amygdala-frontal interactions and reward expectancy. Current Opinion in Neurobiology, 14, 148-155.
91. Hollerman, J. R., & Schultz, W. (1998). Dopamine neurons report an error in the temporal prediction of reward during learning. Nature Neuro-science, 1, 304-309.
92. Holroyd, C. B., & Coles, M. G. H. (2002). The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109, 679-709.
93. Horn, N. R., Dolan, M., Elliott, R., Deakin, J. F., & Woodruff, P. W. (2003). Response inhibition and impulsivity: An fMRI study. Neuropsychologia, 41, 1959-1966.
94. Houlc, J. C., & Wise, S. P. (1995). Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: Their role in planning and controlling action. Cerebral Cortex, 5, 95-110.
95. Iversen, S., & Mishkin, M. (1970). Perseverative interference in monkeys following selective lesions of the inferior prefrontal convexity. Experimental Brain Research, 11, 376-386.
96. Izquierdo, A., Suda, R. K., & Murray, E. A. (2004). Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. Journal of Neuroscience, 24, 7540-7548.
97. Joel, D., Doljansky, J., Roz, N., & Rehavi, M. (2005). Role of the orbital cortex and of the serotonergic system in a rat model of obsessive compulsive disorder. Neuroscience, 130, 25-36.
98. Joel, D., & Weiner, I. (1999). Striatal contention scheduling and the split circuit scheme of basal ganglia-thalamocortical circuitry: From anatomy to behaviour. In R. Miller & J. R. Wickens (Eds.), Conceptual advances in brain research: Brain dynamics and the striatal complex (pp. 209-236). Amsterdam: Harwood Academic.
99. Jog, M. S., Kubota, Y., Connolly, C. I., Hillegaart, V., & Graybiel, A. M. (1999, November 26). Building neural representations of habits. Science, 286, 1745-1749.
100. Johansen, J. P., & Fields, H. L. (2004). Glutamatergic activation of anterior cingulate cortex produces an aversive teaching signal. Nature Neuroscience, 7, 398-403.
101. Jones, B., & Mishkin, M. (1972). Limbic lesions and the problem of stimulus-reinforcement associations. Experimental Neurology, 36, 362-377.
102. Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47, 263-291.
103. Kamin, L. J. (1968). "Attention-like" processes in classical conditioning. In M. R. Jones (Ed.), Miami symposium on the prediction of behavior (pp. 9-31). Miami, FL: University of Miami Press.
104. Kapur, S., & Remington, G. (1996). Serotonin-dopamine interaction and its relevance to schizophrenia. American Journal of Psychiatry, 153, 466-476.
105. Kemp, J. M., & Powell, T. P. (1970). The cortico-striate projections in the monkey. Brain, 93, 525-546.
106. Kerr, J. N., & Wickens, J. R. (2001). Dopamine D-1/D-5 receptor activation is required for long-term potentiation in the rat neostriatum in vitro. Journal of Neurophysiology, 85, 117-124.
107. Knowlton, B. J., Mangels, J. A., & Squire, L. R. (1996, September 6). A neostriatal habit learning system in humans. Science, 273, 1399-1402.
108. Knutson, B., Fong, G. W., Bennett, S. M., Adams, C. M., & Hommer, D. (2003). A region of mesial prefrontal cortex tracks monetarily rewarding outcomes: Characterization with rapid event-related fMRI. NeuroImage, 18, 263-272.
109. Kringelbach, M. L., & Rolls, E. T. (2004). The functional neuroanatomy of the human orbitofrontal cortex: Evidence from neuroimaging and neuropsychology. Progress in Neurobiology, 72, 341-372.
110. LeDoux, J. E. (2000). Emotion circuits in the brain. Annual Review of Neuroscience, 23, 155-184.
111. Lindvall, O., Bjorklund, A., & Divac, I. (1978). Organization of the catecholamine neurons projecting to the frontal cortex in the rat. Brain Research, 142, 1-24.
112. Major, G., & Tank, D. (2004). Persistent neural activity: Prevalence and mechanisms. Current Opinion in Neurobiology, 14, 675-684.
113. Markowitsch, H. J., Vandekerckhovel, M. M., Lanfermann, H., & Russ, M. O. (2003). Engagement of lateral and medial prefrontal areas in the ecphory of sad and happy autobiographical memories. Cortex, 39, 643-665.
114. Mazzanti, C. M., Lappalainen, J., Long, J. C. Bengel, D., Naukkarinen, H., Eggert, M., et al. (1998). Role of the serotonin transporter promoter polymorphism in anxiety-related traits. Archives of General Psychiatry, 55, 936-940.
115. McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004, October 15). Separate neural systems value immediate and delayed rewards, Science, 306, 503-507.
116. McCoy, A. N., & Platt, M. L. (2005). Expectations and outcomes: Decision making in the primate brain. Journal of Comparative Physiology A: Sensory, Neural, and Behavioral Physiology, 191, 201-211.
117. Mehta, M. A., Swainson, R., Ogilvie, A. D., Sahakian, B. J., & Robbins, T. W. (2000). Improved short-term spatial memory but impaired reversal learning following the dopamine D2 agonist bromocriptine in human volunteers. Psychopharmacology, 159, 10-20.
118. Mesulam, M.-M. (2000). Paralimbic (mesocortical) areas. In M.-M. Mesulam (Ed.), Principles of behavioral and cognitive neurology (pp. 49-54). New York: Oxford University Press.
119. Milham, M. P., Crowley, T. J., Thompson, L. L., Raymond, K. M., Claus, E. D., & Banich, M. T. (2006). Reduction in orbitofrontal brain volume in adolescents with severe substance and conduct problems. Manuscript submitted for publication.
120. Miller, E. K., & Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. Annual Review of Neuroscience, 24, 167-202.
121. Miller, E. K., Erickson, C. A., & Desimone, R. (1996). Neural mechanisms of visual working memory in prefrontal cortex of the macaque. Journal of Neuroscience, 16, 5154-5167.
122. Mink, J. W. (1996). The basal ganglia: Focused selection and inhibition of competing motor programs. Progress in Neurobiology, 50, 381-425.
123. Mobini, S., Body, S., Ho, M. Y., Bradshaw, C. M., Szabadi, E., Deakin, J. F., & Anderson, I. M. (2002). Effects of lesions of the orbitofrontal cortex on sensitivity to delayed and probabilistic reinforcement. Psychopharmacology, 160, 290-298.
124. Morris, G., Arkadir, D., Nevet, A., Vaadia, E., & Bergman, H. (2004). Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron, 43, 133-143.
125. Morton, J. B., & Munakata, Y. (2002). Active versus latent representations: A neural network model of perseveration and dissociation in early childhood. Developmental Psychobiology, 40, 255-265.
126. Muller, U., Wachter. T., Barthel, H., Reuter, M., & von Cramon, D. Y. (2000). Striatal [123I]beta-CIT SPECT and prefrontal cognitive functions in Parkinson's disease. Journal of Neural Transmission, 107, 303-319.
127. Murphy, F. C., Smith, K. A., Cowen, P. J., Robbins, T. W., & Sahakian, B. J. (2002). The effects of tryptophan depletion on cognitive and affective processing in healthy volunteers. Psychopharmacology, 163, 42-53.
128. Nishi, A., Snyder, G. L., & Greengard, P. (1997). Bidirectional regulation of DARPP-32 phosphorylation by dopamine. Journal of Neuroscience, 17, 8147-8155.
129. Nocjar, C., Roth, B. L., & Pehek, E. A. (2002). Localization of 5-HT(2A) receptors on dopamine cells in subnuclei of the midbrain A10 cell group. Neuroscience, 111, 163-176.
130. Norman, D. A., & Shallice, T. (1986). Attention to action: Willed and automatic control of behavior. In R. J. Davidson, G. E. Schwartz, & D. Shapiro (Eds.), Consciousness and self-regulation: Advances in research and theory (Vol. 4, pp. 1-18). New York: Plenum Press.
131. O'Doherty, J., Critchley, H., Deichmann, R., & Dolan, R. J. (2003). Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices. Journal of Neuroscience, 23, 7931-7939.
132. O'Doherty, J., Kringelback, M. L., Rolls, E. T., Hornak, J., & Andrews, C. (2001). Abstract reward and punishment in the human orbitofrontal cortex. Nature Neuroscience, 4, 95-102.
133. Oja, E. (1982). A simplified neuron model as a principal component analyzer. Journal of Mathematical Biology, 15, 267-273.
134. Onn, S. P., West, A. R., & Grace, A. A. (2000). Dopamine-mediated regulation of striatal neuronal and network interactions. Trends in Neuroscience, 25(Suppl.1), S48-S56.
135. O'Reilly, R. C. (1996). Biologically plausible error-driven learning using local activation differences: The generalized recirculation algorithm. Neural Computation, 8, 895-938.
136. O'Reilly, R. C. (1998). Six principles for biologically-based computational models of cortical cognition. Trends in Cognitive Sciences, 2, 455-462.
137. O'Reilly, R. C., & Frank, M. J. (2006). Making working memory work: A computational model of learning in the frontal cortex and basal ganglia. Neural Computation, 18, 283-328.
138. O'Reilly, R. C., Frank, M. J., Hazy, T. E., & Watz, B. (2005). Rewards are timeless: The primary value and learned value (PVLV) Pavlovian learning algorithm. Manuscript submitted for publication.
139. O'Reilly, R. C., & Munakata, Y. (2000). Computational explorations in cognitive neuroscience: Understanding the mind by simulating the brain. Cambridge. MA: MIT Press.
140. O'Reilly, R. C., Noelle, D., Braver, T. S., & Cohen, J. D. (2002). Prefrontal cortex and dynamic categorization tasks: Representational organization and neuromodulatory control. Cerebral Cortex, 12, 246-257.
141. Packard, M. G., & Knowlton, B. J. (2002). Learning and memory functions of the basal ganglia. Annual Review of Neuroscience, 25, 563-593.
142. Pan, W.-X., Schmidt, R., Wickens, J. R., & Hyland, B. I. (2005). Dopamine cells respond to predicted events during classical conditioning: Evidence for eligibility traces in the reward-learning network. Journal of Neuro-science, 25, 6235-6242.
143. Parkinson, J. A., Dalley, J. W., Cardinal, R. N., Bamford, A., Fehnert, B., Lachenal, G., et al. (2002). Nucleus accumbens dopamine depletion impairs both acquisition and performance of appetitive Pavlovian approach behaviour: Implications for mesoaccumbens dopamine function. Behavioral Brain Research 137, 149-163.
144. Pasupathy, A., & Miller, E. K. (2005, February 24). Different time courses for learning-related activity in the prefrontal cortex and striatum. Nature, 433, 873-876.
145. Paulus, M. P., Rogalsky, C., Simmons, A., Feinstein, J. S., & Stein, M. B. (2003). Increased activation in the right insula during risk-taking decision making is related to harm avoidance and neuroticism. NeuroImage, 19, 1439-1448.
146. Payne, J. W., Bettman, J. R., & Johnson, E. J. (1993). The adaptive decision maker. New York: Cambridge University Press.
147. Peters, E., & Slovic, P. (2000). The springs of action: Affective and analytical information processing in choice. Personality and Social Psychology Bulletin, 26, 1465-1475.
148. Pickens, C. L., Saddoris, M. P., Gallagher, M., & Holland, P. C. (2005). Orbitofrontal lesions impair use of cue-outcome associations in a devaluation task. Behavioral Neuroscience, 119, 317-322.
149. Pickens, C. L., Saddoris, M. P., Setlow, B., Gallagher, M., Holland, P. C., & Schoenbaum, G. (2003). Different roles for orbitofrontal cortex and basolateral amygdala in a reinforcer devaluation task. Journal of Neuroscience, 23, 11078-11084.
150. Platt, M. L. (2002). Neural correlates of decisions. Current Opinion in Neurobiology, 12, 141-148.
151. Plous, S. (1993). The psychology of judgment and decision making. New York: McGraw-Hill.
152. Remijnse, P. L., Nielen, M. M. A., Uylings, H. B. M., & Veltman, D. J. (2005). Neural correlates of a reversal learning task with an affectively neutral baseline: An event-related: fMRI study. NeuroImage, 26, 609-618.
153. Remy, P., Jackson, P. L., & Ribeiro, M. J. (2000). Relationships between cognitive deficits and dopaminergic function in the striatum of Parkinson's disease patients: A PET study. Neurology, 54 (Suppl. 3), A372.
154. Robertson, G. S., Vincent, S. R., & Fibiger, H. C. (1992). Dl and D2 dopamine receptors differentially regulate c-fos expression in striatonigral and striatopallidal neurons. Neuroscience, 49, 285-296.
155. Robinson, R. G., & Starkstein, S. E. (1989). Mood disorders following stroke: New findings and future directions. Journal of Geriatric Psychiatry, 22, 1-15.
156. Robinson, R. G., & Szetela, B. (1981). Mood change following left hemispheric brain injury. Annals of Neurology, 9, 447-453.
157. Roesch, M. R., & Olson, C. R. (2004, April 9). Neuronal activity related to reward value and motivation in primate frontal cortex. Science, 304, 307-310.
158. Rogers, R. D. Owen, A. M., Middleton, H. C., Williams, E. J., Pickard, J. D., Sahakian, B. J., & Robbins, T. W. (1999). Choosing between small, likely rewards and large, unlikely rewards activates inferior and orbital prefrontal cortex. Journal of Neuroscience, 19, 9029-9038.
159. Rolls, E. T. (1996). The orbitofrontal cortex. Philosophical Transactions of the Royal Society of London, Series B, 351, 1433-1444.
160. Rolls, E. T. (1999). The brain and emotion. Oxford University Press.
161. Rolls, E. T. (2004). Convergence of sensory systems in the orbitofrontal cortex in primates and brain design for emotion. The Anatomical Record Part A: Discoveries in Molecular, Cellular, and Evolutionary Biology, 281A, 1212-1225.
162. Rolls, E. T., & Kringelbach, M. L. (2003). Different representations of pleasant and unpleasant odours in the human brain. European. Journal of Neuroscience, 18, 695-703.
163. Rosenkilde, C. E., Bauer, R. H., & Fuster, J. M. (1981). Single cell activity in ventral prefrontal cortex of behaving monkeys. Brain Research, 209, 375-394.
164. Rubchinsky, L. L. Kopell, N., & Sigvardt, K. A. (2003). Modeling facilitation and inhibition of competing motor programs in basal ganglia subthalamic nucleus-pallidal circuits. Proceedings of the National Academy of Sciences, USA, 100, 14427-14432.
165. Satoh, T., Nakai, S., Sato, T., & Kimura, M. (2003). Correlated coding of motivation and outcome of decision by dopamine neurons. Journal of Neuroscience, 23, 9913-9923.
166. Schoenbaum, G., Chiba, A. A., & Gallagher, M. (1999). Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning. Journal of Neuroscience, 19, 1876-1884.
167. Schoenbaum, G., Chiba, A. A., & Gallagher, M. (2000). Changes in functional connectivity in orbitofrontal cortex and basolateral amygdala during learning and reversal training. Journal of Neuroscience, 20, 5179-5189.
168. Schoenbaum, G., & Roesch, M. (2005). Orbitofrontal cortex, associative-learning, and expectancies. Neuron, 47, 633-636.
169. Schoenbaum, G., & Setlow, B. (2001). Integrating orbitofrontal cortex into prefrontal theory: Common processing themes across species and sub-divisions. Learning and Memory, 8, 134-147.
170. Schoenbaum, G., & Setlow, B. (2003). Lesions of nucleus accumbens disrupt learning about aversive outcomes. Journal of Neuroscience, 23, 9833-9841.
171. Schoenbaum, G., & Setlow, B. (2005). Cocaine makes actions insensitive to outcomes but not extinction: Implications for altered orbitofrontal-amygdalar function. Cerebral Cortex, 15, 1162-1169.
172. Schoenbaum, G., Setlow, B., Saddoris, M. P., & Gallagher, M. (2003). Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala. Neuron, 39, 855-867.
173. Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of Neurophysiology, 80, 1-27.
174. Schultz, W. (1999). The reward signal of midbrain dopamine neurons. News in Physiological Sciences, 14, 249-255.
175. Schultz, W. (2002). Getting formal with dopamine and reward. Neuron, 36, 241-263.
176. Schultz, W., Dayan, P., & Montague, P. R. (1997. March 14). A neural substrate of prediction and reward. Science, 275, 1593-1599.
177. Seamans, J. K., Durslewitz, D., Christie, B. R., Stevens, C. F., & Sejnowski, T. J. (2001). Dopamine D1/D5 receptor modulation of excitatory synaptic inputs to layer V prefrontal cortex neurons. Proceedings of the National Academy of Sciences, USA, 98, 301-306.
178. Seamans, J. K., & Yang, C. R. (2004). The principal features and mechanisms of dopamine modulation in the prefrontal cortex. Progress in Neurobiology, 74, 1-57.
179. Setlow, B., Schoenbaum, G., & Gallagher, M. (2003). Neural encoding in ventral striatum during olfactory discrimination learning. Neuron, 38, 625-636.
180. Shanks, D. R., Tunney, R. J., & McCarthy, J. D. (2002). A re-examination of probability matching and rational choice. Journal of Behavioral Decision Making, 15, 233-250.
181. Shima, K., & Tanji, J. (1998, November 13). Role for cingulate motor area cells in voluntary movement selection based on reward. Science, 282, 1335-1338.
182. Shiv, B., Loewenstein, G., & Bechara, A. (2005). The dark side of emotion in decision-making: When individuals with decreased emotional reactions make more advantageous decisions. Cognitive Brain Research, 23, 85-92.
183. Shohamy, D., Myers, C. E., Grossman, S., Sage, J., Gluck, M. A., & Poldrack, R. A. (2004). Cortico-striatal contributions to feedback-based learning: Converging data from neuroimaging and neuropsychology. Brain, 127, 851-859.
184. Small, D. M., Zatorre, R. J., Dagher, A., Evans, A. C., & Jones-Gotman, M. (2001). Changes in brain activity related to eating chocolate: From pleasure to aversion. Brain, 124(Pt. 9), 1720-1733.
185. Smith-Roe, S. L., & Kelley, A. E. (2000). Coincident activation of NMDA and dopamine Dl receptors within the nucleus accumbens core is required for appetitive instrumental learning. Journal of Neuroscience, 22, 7737-7742.
186. Stout, J. C., Busemeyer, J. R., Lin, A., Grant, S. J., & Bonson, K. R. (2004). Cognitive modeling analysis of decision-making processes in cocaine abusers. Psychonomic Bulletin & Review, 11, 742-747.
187. Sutton. R. S. (1988). Learning to predict by the method of temporal differences. Machine Learning, 3, 9-44.
188. Swainson, R., Rogers, R. D., Sahakian, B. J., Summers, B. A., Polkey, C. E., & Robbins, T. W. (2000). Probabilistic learning and reversal deficits in patients with Parkinson's disease or frontal or temporal lobe lesions: Possible adverse effects of dopaminergic medication. Neuropsychologia, 38, 596-612.
189. Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005, March 11). Adaptive coding of reward value by dopamine neurons. Science, 307, 1642-1645.
190. Tremblay, L., & Schultz, W. (1999, April 22). Relative reward preference in primate orbitofrontal cortex. Nature, 398, 704-708.
191. Tremblay, L., & Schultz, W. (2000). Modifications of reward expectation-related neuronal activity during learning in primate orbitofrontal cortex. Journal of Neurophysiology, 83, 1877-1885.
192. Tversky, A., & Kahneman, D. (1974, September 27). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124-1131.
193. Tversky, A., & Kahneman, D. (1981, January 30). The framing of decisions and the psychology of choice. Science, 211, 453-458.
194. Tversky, A., & Kahneman, D. (1986). Rational choice and the framing of decisions. Journal of Business, 59, S251-S278.
195. Tversky, A., & Kahneman, D. (1991). Loss aversion in riskless choice: A reference dependent model. Quarterly Journal of Economics, 107, 1039-1061.
196. Ungless, M. A., Magill, P. J., & Bolam, J. P. (2004, March 26). Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science, 303, 2040-2042.
197. Ursu, S., & Carter, C. S. (2005). Outcome representations, counterfactual comparisons and the human orbitofrontal cortex: Implications for neuroimaging studies of decision-making. Brain Research, 23, 51-60.
198. Volkow, N. D., Chang, L., Wang, G. J., Fowler, J. S., Ding, Y. S., Sedler, M., et al. (2001). Low level of brain dopamine D2 receptors in methamphetamine abusers: Association with metabolism in the orbitofrontal cortex. American Journal of Psychiatry, 158, 2015-2021.
199. Volkow, N. D., Fowler, J. S., & Wang, G. J. (2003). The addicted human brain: Insights from imaging studies. Journal of Clinical Investigation, 111 1444-1451.
200. Waelti, P., Dickinson, A., & Schultz, W. (2001, July 5). Dopamine responses comply with basic assumptions of formal learning theory. Nature, 412, 43-48.
201. Wagar, B. M., & Thagard, P. (2004). Spiking Phineas Gage: A neurocomputational theory of cognitive-affective integration in decision making. Psychological Review, 111, 67-79.
202. Walderhaug, E., Lunde, H., Nordvik, J. E., Landro, N. I. Refsum, H., & Magnusson, A. (2002). Lowering of serotonin by rapid tryptophan depletion increases impulsiveness in normal individuals. Psychopharmacology, 164, 385-391.
203. Wallis, J. D., & Miller, E. K. (2003). From rale to response: Neuronal processes in the premotor and prefrontal cortex. Journal of Neurophysiology, 90, 1790-1806.
204. Wang, J., & O'Donnell, P. (2001). Dl dopamine receptors potentiate NMDA-mediated excitability increase in layer V prefrontal cortical pyramidal neurons. Cerebral Cortex, 11, 452-462.
205. Watanabe, K., & Hikosaka, O. (2005). Immediate changes in anticipatory activity of caudate neurons associated with reversal of position-reward contingency. Journal of Neurophysiology, 94, 1879-1887.
206. Wickens, J. (1997). Basal ganglia: Structure and computations. Network: Computation in Neural Systems, 8, R77-R109.
207. Wise, S. P., Murray, E. A., & Gerfen, C. R. (1996). The frontal cortex-basal ganglia system in primates. Critical Reviews in Neurobiology, 10, 317-356.
208. Yeung, N., Botvinick, M. M., & Cohen, J. D. (2004). The neural basis of error detection: Conflict monitoring and the error-related negativity. Psychological Review, 111, 931-959.
209. Yin, H. H., Knowlton, B. J., & Balleine, B. W. (2004). Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. European Journal of Neuroscience, 19, 181-189.