Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons

Journal of Neuroscience

Volumn 23, Issue 30, 2003, Pages 9913-9923

  Satoh, Takemasa ^a   Nakai, Sadamu ^a   Sato, Tatsuo ^a   Kimura, Minoru ^a  

^a KYOTO PREFECTURAL UNIVERSITY OF MEDICINE   (Japan)

Author keywords

Decision making; Dopamine neuron; Learning; Reward prediction error; Teaching signal; Trial and error

Indexed keywords

ANIMAL EXPERIMENT; ARTICLE; ASSOCIATION; BEHAVIOR; CONDITIONING; CONTROLLED STUDY; CORRELATION ANALYSIS; DECISION MAKING; DOPAMINERGIC NERVE CELL; DOPAMINERGIC SYSTEM; ERROR; EXPECTATION; LEARNING; MALE; MESENCEPHALON; MONKEY; MOTIVATION; NONHUMAN; PRIORITY JOURNAL; REINFORCEMENT; REWARD; TASK PERFORMANCE; TEACHING;

EID: 0242440823     PISSN: 02706474     EISSN: None     Source Type: Journal    
DOI: 10.1523/jneurosci.23-30-09913.2003     Document Type: Article

References (49)

1. Aosaki T, Graybiel AM, Kimura M (1994) Effect of the nigrostriatal dopamine system on acquired neural responses in the striatum of behaving monkeys. Science 265:412-415.
2. Arnauld A, Nichole P (1982) The art of thinking: Port-Royal logic (Dickoff J, James P, translators). Indianapolis: Bobbs-Merrill.
3. Barto AG (1995) Adaptive critics and the basal ganglia. In. Models of information processing in the basal ganglia (Houk JC, Davis JL, Beiser DG, eds), pp 215-232. Cambridge, MA: MIT.
4. Bindra D (1978) How adaptive behavior is produced: a perceptual-motivation alternative to response reinforcement. Behav Brain Sci 1:41-91.
5. Bolles RC (1972) Reinforcement, expectancy, and learning. Psychol Rev 79:394-409.
6. Brown J, Bullock D, Grossberg S (1999) How the basal ganglia use parallel excitatory and inhibitory learning pathways to selectively respond to unexpected rewarding cues. J Neurosci 19:10502-10511.
7. Dayan P, Balleine BW (2002) Reward, motivation, and reinforcement learning. Neuron 36:285-298.
8. Dayan P, Kakade S, Montague PR (2000) Learning and selective attention. Nat Neurosci [Suppl] 3:1218-1223.
9. Dickinson A (1980) Contemporary animal learning theory. Cambridge, UK: Cambridge UP.
10. Dickinson A, Balleine B (1994) Motivational control of goal-directed action. Anim Learn Behav 22:1-18.
11. Dormont JF, Conde H, Farin D (1998) The role of the pedunculopontine tegmental nucleus in relation to conditioned motor performance in the cat. I. Context-dependent and reinforcement-related single unit activity. Exp Brain Res 121:401-410.
12. Fiorillo CD, Tobler PN, Schultz W (2003) Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299:1898-1902.
13. Futami T, Takakusaki K, Kitai ST (1995) Glutamatergic and cholinergic inputs from the pedunculopontine tegmental nucleus to dopamine neurons in the substantia nigra pars compacta. Neurosci Res 21:331-342.
14. Grace AA, Bunney BS (1983) Intracellular and extracellular electrophysiology of nigral dopaminergic neurons-1. Identification and characterization. Neuroscience 10:301-315.
15. Herrnstein RJ, Vaughn WJ (1980) The allocation of individual behavior. In. Limits to action (Staddon JER, ed), pp 143-176. New York: Academic.
16. Horvitz JC (2000) Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience 96:651-656.
17. Horvitz JC, Stewart T, Jacobs BL (1997) Burst activity of ventral tegmental dopamine neurons is elicited by sensory stimuli in the awake cat. Brain Res 759:251-258.
18. Houk JC, Adams JL, Barto AG (1995) A model of how the basal ganglia generate and use neural signals that predict reinforcement. In: Models of information processing in the basal ganglia (Houk JC, Davis JL, Beiser DG, eds), pp 249-270. Cambridge, MA: MIT.
19. Hull CL (1943) Principles of behavior, an introduction to behavior theory, Chap 6, 7. New York: Appleton-Century.
20. Jacobs BL (1986) Single unit activity of brain monoamine-containing neurons in freely moving animals. Ann NY Acad Sci 473:70-77.
21. Kimura M (1986) The role of primate putamen neurons in the association of sensory stimuli with movement. Neurosci Res 3:436-443.
22. Kobayashi Y, Inoue Y, Yamamoto M, Isa T, Aizawa H (2002) Contribution of pedunculopontine tegmental nucleus neurons to performance of visually guided saccade tasks in monkeys. J Neurophysiol 88:715-731.
23. Koepp MJ, Gunn RN, Lawrence AD, Cunningham VJ, Dagher A, Jones T, Brooks DJ, Bench CJ, Grasby PM (1998) Evidence for striatal dopamine release during a video game. Nature 393:266-268.
24. Konorski J (1967) Integrative activity of the brain. Chicago: Chicago UP.
25. Ljungberg T, Apicella P, Schultz W (1992) Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol 67:145-163.
26. Matsumoto N, Hanakawa T, Maki S, Graybiel AM, Kimura M (1999) Role of nigrostriatal dopamine system in learning to perform sequential motor tasks in a predictive manner. J Neurophysiol 82:978-998.
27. Medina JF, Nores WL, Mauk MD (2002) Inhibition of climbing fibres is a signal for the extinction of conditioned eyelid responses. Nature 416:330-333.
28. Mirenowicz J, Schultz W (1994) Importance of unpredictability for reward responses in primate dopamine neurons. J Neurophysiol 72:1024-1027.
29. Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16:1936-1947.
30. Olds J, Milner P (1954) Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. J Comp Physiol Psychol 47:419-427.
31. Redgrave P, Prescott TJ, Gurney K (1999) Is the short-latency dopamine response too short to signal reward error? Trends Neurosci 22:146-151.
32. Rescorla RA, Wagner AR (1972) Current research and theory. In: Classical conditioning II (Black AH, Prokasy WF, eds), pp 64-99. New York: Appleton Century Crofts.
33. Robbins TW, Everitt BJ (1996) Neurobehavioural mechanisms of reward and motivation. Curr Opin Neurobiol 6:228-236.
34. Salamone JD, Correa M (2002) Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine. Behav Brain Res 137:3-25.
35. Schultz W (1986) Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. J Neurophysiol 56:1439-1461.
36. Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80:1-27.
37. Schultz W, Romo R (1987) Responses of nigrostriatal dopamine neurons to high-intensity somatosensory stimulation in the anesthetized monkey. J Neurophysiol 57:201-217.
38. Schultz W, Apicella P, Ljungberg T (1993) Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci 13:900-913.
39. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593-1599.
40. Shidara M, Aigner TG, Richmond BJ (1998) Neuronal signals in the monkey ventral striatum related to progress through a predictable series of trials. J Neurosci 18:2613-2625.
41. Spanagel R, Weiss F (1999) The dopamine hypothesis of reward: past and current status. Trends Neurosci 22:521-527.
42. Suri RE, Schultz W (1998) Learning of sequential movements by neural network model with dopamine-like reinforcement signal. Exp Brain Res 121:350-354.
43. Sutton RS (1988) Learning to predict by the method of temporal differences. Mach Learn 3:9-44.
44. Sutton RS, Barto AG (1998) Reinforcement learning, pp 87-160. Cambridge, MA: MIT.
45. Takikawa Y, Kawagoe R, Itoh H, Nakahara H, Hikosaka O (2002) Modulation of saccadic eye movements by predicted reward outcome. Exp Brain Res 142:284-291.
46. Thorndike EL (1911) Animal intelligence. New York: Macmillan.
47. Toates F (1986) Motivational systems. Cambridge, UK: Cambridge UP.
48. Watanabe M, Cromwell HC, Tremblay L, Hollerman JR, Hikosaka K, Schultz W (2001) Behavioral reactions reflecting differential reward expectations in monkeys. Exp Brain Res 140:511-518.
49. Wise RA (2002) Brain reward circuitry: insights from unsensed incentives. Neuron 36:229-240.