What is reinforced by phasic dopamine signals?

Brain Research Reviews

Volumn 58, Issue 2, 2008, Pages 322-339

  Redgrave, Peter ^a   Gurney, Kevin ^a   Reynolds, John ^b  

^a UNIVERSITY OF SHEFFIELD   (United Kingdom)

^b UNIVERSITY OF OTAGO   (New Zealand)

Author keywords

Agency; Basal ganglia; Dopamine; Reinforcement; Reward; Selection

Indexed keywords

BASAL GANGLION; DOPAMINERGIC SYSTEM; DOPAMINERGIC TRANSMISSION; LEARNING; MESENCEPHALON; MONKEY; NONHUMAN; PRIORITY JOURNAL; REINFORCEMENT; REVIEW; SENSORY NERVE; SIGNAL TRANSDUCTION; STIMULUS RESPONSE; VISION;

ANIMALS; DOPAMINE; HUMANS; NEURAL PATHWAYS; NEURONS; REINFORCEMENT (PSYCHOLOGY); SUBSTANTIA NIGRA;

EID: 49449095638     PISSN: 01650173     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.brainresrev.2007.10.007     Document Type: Review

References (165)

1. Alexander G.E., DeLong M.R., and Strick P.L. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Ann. Rev. Neurosci. 9 (1986) 357-381
2. Apicella P., Legallet E., and Trouche E. Responses of tonically discharging neurons in the monkey striatum to primary rewards delivered during different behavioral states. Exp. Brain Res. 116 (1997) 456-466
3. Arbuthnott G.W., and Wickens J. Space, time and dopamine. Trends Neurosci. 30 2 (2007) 62-69
4. Balleine B.W., and Killcross S. Parallel incentive processing: an integrated view of amygdala function. Trends Neurosci. 29 5 (2006) 272-279
5. Bardo M.T., Bowling S.L., Robinet P.M., Rowlett J.K., Lacy M., and Mattingly B.A. Role of D1 and D2 receptors in novelty-maintained place preference. Exp. Clin. Psychopharm. 1 (1993) 101-109
6. BarGad I., Morris G., and Bergman H. Information processing, dimensionality reduction and reinforcement learning in the basal ganglia. Prog. Neurobiol. 71 6 (2003) 439-473
7. Bayer H.M., and Glimcher P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47 1 (2005) 129-141
8. Berridge K.C. The debate over dopamine's role in reward: the case for incentive salience. Psychopharmacol. (Berl.) 191 3 (2007) 391-431
9. Berridge K.C., Aldridge J.W., Houchard K.R., and Zhuang X.X. Sequential super-stereotypy of an instinctive fixed action pattern in hyper-dopaminergic mutant mice: a model of obsessive compulsive disorder and Tourette's. BMC Biol. 3 (2005)
10. Bevins R.A., Besheer J., Palmatier M.I., Jensen H.C., Pickett K.S., and Eurek S. Novel-object place conditioning: behavioral and dopaminergic processes in expression of novelty reward. Behav. Brain Res. 129 (2002) 41-50
11. Bickford M.E., and Hall W.C. Collateral projections of predorsal bundle cells of the superior colliculus in the rat. J. Comp. Neurol. 283 (1989) 86-106
12. Black J., Belluzzi J.D., and Stein L. Reinforcement delay of one second severely impairs acquisition of brain self-stimulation. Brain Res. 359 1-2 (1985) 113-119
13. Blatter K., and Schultz W. Rewarding properties of visual stimuli. Exp. Brain Res. 168 4 (2006) 541-546
14. Calabresi P., Gubellini P., Centonze D., Picconi B., Bernardi G., Chergui K., Svenningsson P., Fienberg A.A., and Greengard P. Dopamine and cAMP-regulated phosphoprotein 32 kDa controls both striatal long-term depression and long-term potentiation, opposing forms of synaptic plasticity. J. Neurosci. 20 22 (2000) 8443-8451
15. Calabresi P., Picconi B., Tozzi A., and Di Filippo M. Dopamine-mediated regulation of corticostriatal synaptic plasticity. Trends Neurosci. 30 5 (2007) 211-219
16. Centonze D., Picconi B., Gubellini P., Bernardi G., and Calabresi P. Dopaminergic control of synaptic plasticity in the dorsal striatum. Eur. J. Neurosci. 13 (2001) 1071-1077
17. Chevalier G., and Deniau J.M. Disinhibition as a basic process in the expression of striatal functions. Trends Neurosci. 13 (1990) 277-281
18. Coizet V., Comoli E., Westby G.W.M., and Redgrave P. Phasic activation of substantia nigra and the ventral tegmental area by chemical stimulation of the superior colliculus: an electrophysiological investigation in the rat. Eur. J. Neurosci. 17 (2003) 28-40
19. Coizet V., Dommett E.J., Redgrave P., and Overton P.G. Nociceptive responses of midbrain dopaminergic neurones are modulated by the superior colliculus in the rat. Neuroscience 139 4 (2006) 1479-1493
20. Coizet V., Overton P.G., and Redgrave P. Collateralization of the tectonigral projection with other major output pathways of superior colliculus in the rat. J. Comp. Neurol. 500 6 (2007) 1034-1049
21. Comoli E., Coizet V., Boyes J., Bolam J.P., Canteras N.S., Quirk R.H., Overton P.G., and Redgrave P. A direct projection from superior colliculus to substantia nigra for detecting salient visual events. Nat. Neurosci. 6 9 (2003) 974-980
22. Corbit L.H., and Balleine B.W. The role of prelimbic cortex in instrumental conditioning. Behav. Brain Res. 146 1-2 (2003) 145-157
23. Corbit L.H., Ostlund S.B., and Balleine B.W. Sensitivity to instrumental contingency degradation is mediated by the entorhinal cortex and its efferents via the dorsal hippocampus. J. Neurosci. 22 24 (2002) 10976-10984
24. Corlett P.R., Aitken M.R.F., Dickinson A., Shanks D.R., Honey G.D., Honey R.A.E., Robbins T.W., Bullmore E.T., and Fletcher P.C. Prediction error during retrospective revaluation of causal associations in humans: fMRI evidence in favor of an associative model of learning. Neuron 44 5 (2004) 877-888
25. Costa R.M. Plastic corticostriatal circuits for action learning: what's dopamine got to do with it?. Ann. N. Y. Acad. Sci. 1104 (2007) 172-191
26. Craig A.D.B. A new view of pain as a homeostatic emotion. Trends Neurosci. 26 (2003) 303-307
27. Crossman A.R. Functional anatomy of movement disorders. J. Anat. 196 (2000) 519-525
28. Crutcher M.D., and DeLong M.R. Single cell studies of the primate putamen. II. Relations to direction of movement and pattern of muscular activity. Exp. Brain Res. 53 2 (1984) 244-258
29. Dayan P., and Balleine B.W. Reward, motivation, and reinforcement learning. Neuron 36 (2002) 285-298
30. de Gelder B., and Hadjikhani N. Non-conscious recognition of emotional body language. Neuroreport 17 6 (2006) 583-586
31. Dean P., Redgrave P., and Westby G.W.M. Event or emergency? Two response systems in the mammalian superior colliculus. Trends Neurosci. 12 (1989) 137-147
32. Deaner R.O., Khera A.V., and Platt M.L. Monkeys pay per view: adaptive valuation of social images by rhesus macaques. Curr. Biol. 15 6 (2005) 543-548
33. Dickinson A. The 28th Bartlett memorial lecture causal learning: an associative analysis. Quart. J. Exp. Psychol. B. Com. Phycol. P. 54 (2001) 3-25
34. Ding L., and Hikosaka O. Comparison of reward modulation in the frontal eye field and caudate of the macaque. J. Neurosci. 26 25 (2006) 6695-6703
35. Dommett E., Coizet V., Blaha C.D., Martindale J., Lefebvre V., Walton N., Mayhew J.E., Overton P.G., and Redgrave P. How visual stimuli activate dopaminergic neurons at short latency. Science 307 5714 (2005) 1476-1479
36. Elsner B., and Hommel B. Contiguity and contingency in action-effect learning. Psychol. Res. 68 2-3 (2004) 138-154
37. Everitt B.J., and Robbins T.W. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat. Neurosci. 8 11 (2005) 1481-1489
38. Fanselow M.S., and Poulos A.M. The neuroscience of mammalian associative learning. Annu. Rev. Psychol. 56 (2005) 207-234
39. Fields H. State-dependent opioid control of pain. Nat. Rev. Neurosci. 5 7 (2004) 565-575
40. Fiorillo C.D., Tobler P.N., and Schultz W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299 (2003) 1898-1902
41. Floresco S.B., West A.R., Ash B., Moore H., and Grace A.A. Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission. Nat. Neurosci. 6 9 (2003) 968-973
42. Freeman A.S. Firing properties of substantia nigra dopaminergic neurons in freely moving rats. Life Sci. 36 (1985) 1983-1994
43. Fudge J.L., Kunishio K., Walsh P., Richard C., and Haber S.N. Amygdaloid projections to ventromedial striatal subterritories in the primate. Neuroscience 110 (2002) 257-275
44. Fuxe K., Manger P., Genedani S., and Agnati L. The nigrostriatal DA pathway and Parkinson's disease. J. Neural Transm.-Suppl. 70 (2006) 71-83
45. Grace A.A. The tonic phasic model of dopamine system regulation-its relevance for understanding how stimulant abuse can alter basal ganglia function. Drug Alcohol Depend. 37 (1995) 111-129
46. Grantyn R. Gaze control through superior colliculus: structure and function. In: Buttner-Ennever J.A. (Ed). Neuroanatomy of the Oculomotor System (1988), Elsevier, Amsterdam 273-333
47. Greengard P., Allen P.B., and Nairn A.C. Beyond the dopamine receptor: the DARPP-32/protein phosphatase-1 cascade. Neuron 23 3 (1999) 435-447
48. Grillner S., Helligren J., Menard A., Saitoh K., and Wikstrom M.A. Mechanisms for selection of basic motor programs - roles for the striatum and pallidum. Trends Neurosci. 28 7 (2005) 364-370
49. Guarraci F.A., and Kapp B.S. An electrophysiological characterization of ventral tegmental area dopaminergic neurons during differential Pavlovian fear conditioning in the awake rabbit. Behav. Brain Res. 99 (1999) 169-179
50. Gurney K., Prescott T.J., and Redgrave P. A computational model of action selection in the basal ganglia. I. A new functional anatomy. Biol. Cybern. 84 (2001) 401-410
51. Gurney K., Prescott T.J., and Redgrave P. A computational model of action selection in the basal ganglia. II. Analysis and simulation of behaviour. Biol. Cybern. 84 (2001) 411-423
52. Heien M., Khan A.S., Ariansen J.L., Cheer J.F., Phillips P.E.M., Wassum K.M., and Wightman R.M. Real-time measurement of dopamine fluctuations after cocaine in the brain of behaving rats. Proc. Natl. Acad. Sci. U. S. A. 102 29 (2005) 10023-10028
53. Hikosaka O. GABAergic output of the basal ganglia. Prog. Brain Res. 160 (2007) 209-226
54. Hikosaka O., and Wurtz R.H. Visual and oculomotor function of monkey substantia nigra pars reticulata. I. Relation of visual and auditory responses to saccades. J. Neurophysiol. 49 (1983) 1230-1253
55. Hikosaka O., Sakamoto M., and Usui S. Functional properties of monkey caudate neurons II. Visual and auditory responses. J. Neurophysiol. 61 (1989) 799-813
56. Hikosaka O., Nakamura K., and Nakahara H. Basal ganglia orient eyes to reward. J. Neurophysiol. 95 2 (2006) 567-584
57. Horn G., and Hill R.M. Effect of removing the neocortex on the response to repeated sensory stimulation of neurones in the mid-brain. Nature 211 50 (1966) 754-755
58. Horvitz J.C., Stewart T., and Jacobs B.L. Burst activity of ventral tegmental dopamine neurons is elicited by sensory stimuli in the awake cat. Brain Res. 759 (1997) 251-258
59. Houk J.C. Agents of the mind. Biol. Cybern. 92 6 (2005) 427-437
60. Humphries M.D., Stewart R.D., and Gurney K.N. A physiologically plausible model of action selection and oscillatory activity in the basal ganglia. J. Neurosci. 26 50 (2006) 12921-12942
61. Hyland B.I., Reynolds J.N.J., Hay J., Perk C.G., and Miller R. Firing modes of midbrain dopamine cells in the freely moving rat. Neuroscience 114 (2002) 475-492
62. Ikeda T., and Hikosaka O. Reward-dependent gain and bias of visual responses in primate superior colliculus. Neuron 39 (2003) 693-700
63. Izhikevich E.M. Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb. Cortex 17 (2007) 2443-2452
64. Jay M.F., and Sparks D.L. Sensorimotor integration in the primate superior colliculus. I Motor convergence. J. Neurophysiol. 57 (1987) 22-34
65. Johnson M.H. Subcortical face processing. Nat. Rev. Neurosci. 6 10 (2005) 766-774
66. Kakade S., and Dayan P. Dopamine: generalization and bonuses. Neural. Netw. 15 4-6 (2002) 549-559
67. Katsuta H., and Isa T. Release from GABA(A) receptor-mediated inhibition unmasks interlaminar connection within superior colliculus in anesthetized adult rats. Neurosci. Res. 46 1 (2003) 73-83
68. Kerr J.N., and Wickens J.R. Dopamine D-1/D-5 receptor activation is required for long-term potentiation in the rat neostriatum in vitro. J. Neurophysiol. 85 1 (2001) 117-124
69. Kim H., Shimojo S., and O'Doherty J.P. Is avoiding an aversive outcome rewarding? Neural substrates of avoidance learning in the human brain. PLoS Biol. 4 8 (2006) 1453-1461
70. Klop E.M., Mouton L.J., Hulsebosch R., Boers J., and Holstege G. In cat four times as many lamina I neurons project to the parabrachial nuclei and twice as many to the periaqueductal gray as to the thalamus. Neuroscience 134 1 (2005) 189-197
71. Kobayashi S., Lauwereyns J., Koizumi M., Sakagami M., and Hikosaka O. Influence of reward expectation on visuospatial processing in macaque lateral prefrontal cortex. J. Neurophysiol. 87 (2002) 1488-1498
72. Kobayashi S., Nomoto K., Watanabe M., Hikosaka O., Schultz W., and Sakagami M. Influences of rewarding and aversive outcomes on activity in macaque lateral prefrontal cortex. Neuron 51 6 (2006) 861-870
73. Levesque M., Charara A., Gagnon S., Parent A., and Deschenes M. Corticostriatal projections from layer V cells in rat are collaterals of long-range corticofugal axons. Brain Res. 709 (1996) 311-315
74. Lindvall O., and Bjorklund A. The organization of the ascending catecholamine neuron systems in the rat brain as revealed by the glyoxylic acid fluoresence method. Acta. Physiol. Scand., Suppl. 412 (1974) 1-48
75. Lisman J.E., and Grace A.A. The hippocampal-VTA loop: controlling the entry of information into long-term memory. Neuron 46 5 (2005) 703-713
76. Ljungberg T., Apicella P., and Schultz W. Responses of monkey dopamine neurons during learning of behavioural reactions. J. Neurophysiol. 67 (1992) 145-163
77. Maeda H., and Mogenson G.J. Effects of peripheral stimulation on the activity of neurons in the ventral tegmental area, substantia nigra and midbrain reticular formation of rats. Brain Res. Bull. 8 1 (1982) 7-14
78. Mahon S., Deniau J.M., and Charpier S. Corticostriatal plasticity: life after the depression. Trends Neurosci. 27 8 (2004) 460-467
79. Marin O., Smeets W.J.A.J., and Gonzalez A. Evolution of the basal ganglia in tetrapods: a new perspective based on recent studies in amphibians. Trends Neurosci. 21 (1998) 487-494
80. Matsumoto M., and Hikosaka O. Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 447 7148 (2007) 1111-1115
81. Matsumoto N., Minamimoto T., Graybiel A.M., and Kimura M. Neurons in the thalamic CM-Pf complex supply striatal neurons with information about behaviorally significant sensory events. J. Neurophysiol. 85 (2001) 960-976
82. Matsumura M., Kojima J., Gardiner T.W., and Hikosaka O. Visual and oculomotor functions of monkey subthalamic nucleus. J. Neurophysiol. 67 (1992) 1615-1632
83. May P.J., McHaffie J.G., Jiang H., Stanford T.R., Costello M.G., Haber S.N., Coizet V., and Redgrave P. Projections from the Superior Colliculus to Substantia Nigra Pars Compacta in a Primate. Program No 4502 Abstract Viewer/Itinerary Planner (2006), Society for Neuroscience, Washington, DC Online
84. McHaffie J.G., Stanford T.R., Stein B.E., Coizet V., and Redgrave P. Subcortical loops through the basal ganglia. Trends Neurosci. 28 8 (2005) 401-407
85. McHaffie J.G., Jiang H., May P.J., Coizet V., Overton P.G., Stein B.E., and Redgrave P. A direct projection from superior colliculus to substantia nigra pars compacta in the cat. Neuroscience 138 1 (2006) 221-234
86. Medina L., and Reiner A. Neurotransmitter organization and connectivity of the basal ganglia in vertebrates: implications for evolution of basal ganglia. Brain Behav. Evol. 46 (1995) 235-258
87. Mink J.W. The basal ganglia: focused selection and inhibition of competing motor programs. Prog. Neurobiol. 50 (1996) 381-425
88. Mirenowicz J., and Schultz W. Importance of unpredictability for reward responses in primate dopamine neurons. J. Neurophysiol. 72 (1994) 1024-1027
89. Montague P.R., Dayan P., and Sejnowski T.J. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 16 5 (1996) 1936-1947
90. Montague P.R., Hyman S.E., and Cohen J.D. Computational roles for dopamine in behavioural control. Nature 431 7010 (2004) 760-767
91. Morris G., Arkadir D., Nevet A., Vaadia E., and Bergman H. Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron 43 1 (2004) 133-143
92. Morris G., Nevet A., Arkadir D., Vaadia E., and Bergman H. Midbrain dopamine neurons encode decisions for future action. Nat. Neurosci. 9 8 (2006) 1057-1063
93. Nakahara H., Itoh H., Kawagoe R., Takikawa Y., and Hikosaka O. Dopamine neurons can represent context-dependent prediction error. Neuron 41 2 (2004) 269-280
94. Nakamura K., Mikami A., and Kubota K. Activity of single neurons in the monkey amygdala during performance of a visual discrimination task. J. Neurophysiol. 67 6 (1992) 1447-1463
95. O'Doherty J.P., Dayan P., Friston K., Critchley H., and Dolan R.J. Temporal difference models and reward-related learning in the human brain. Neuron 38 2 (2003) 329-337
96. Ohman A. The role of the amygdala in human fear: automatic detection of threat. Psychoneuroendocrinology 30 10 (2005) 953-958
97. Omelchenko N., and Sesack S.R. Glutamate synaptic inputs to ventral tegmental area neurons in the rat derive primarily from subcortical sources. Neuroscience 146 3 (2007) 1259-1274
98. Overton P.G., Coizet V., Dommett E.J., and Redgrave P. The Parabrachial Nucleus is a Source of Short Latency Nociceptive Input to Midbrain Dopaminergic Neurones in Rat (2005), Society for Neuroscience, Washington, DC 301.5 online
99. Padoa-Schioppa C., and Assad J.A. Neurons in the orbitofrontal cortex encode economic value. Nature 441 7090 (2006) 223-226
100. Pan W.X., Schmidt R., Wickens J.R., and Hyland B.I. Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J. Neurosci. 25 26 (2005) 6235-6242
101. Paton J.J., Belova M.A., Morrison S.E., and Salzman C.D. The primate amygdala represents the positive and negative value of visual stimuli during learning. Nature 439 (2006) 865-870
102. Potts G.F., Martin L.E., Burton P., and Montague P.R. When things are better or worse than expected: the medial frontal cortex and the allocation of processing resources. J. Cogn. Neurosci. 18 7 (2006) 1112-1119
103. Prescott T.J., Gonzalez F.M.M., Gurney K., Humphries M.D., and Redgrave P. A robot model of the basal ganglia: behavior and intrinsic processing. Neural Netw. 19 1 (2006) 31-61
104. Ravel S., and Richmond B.J. Dopamine neuronal responses in monkeys performing visually cued reward schedules. Eur. J. Neurosci. 24 1 (2006) 277-290
105. Redgrave P. Basal ganglia. Scholarpedia (2007) 13246
106. Redgrave P., and Gurney K. The short-latency dopamine signal: a role in discovering novel actions?. Nat. Rev. Neurosci. 7 12 (2006) 967-975
107. Redgrave P., Prescott T., and Gurney K.N. The basal ganglia: a vertebrate solution to the selection problem. Neuroscience 89 (1999) 1009-1023
108. Redgrave P., Prescott T.J., and Gurney K. Is the short latency dopamine response too short to signal reward error. Trends Neurosci. 22 (1999) 146-151
109. Reiner A. Functional circuitry of the avian basal ganglia: implications for basal ganglia organization in stem amniotes. Brain Res. Bull. 57 (2002) 513-528
110. Reiner A., Jiao Y., DelMar N., Laverghetta A.V., and Lei W.L. Differential morphology of pyramidal tract-type and intratelencephalically projecting-type corticostriatal neurons and their intrastriatal terminals in rats. J. Comp. Neurol. 457 (2003) 420-440
111. Reynolds J.N., and Wickens J.R. Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw. 15 4-6 (2002) 507-521
112. Reynolds J.N.J., Hyland B.I., and Wickens J.R. A cellular mechanism of reward-related learning. Nature 413 (2001) 67-70
113. Reynolds J.N.J., Schulz J.M., and Wickens J.R. Visual responsiveness of striatal spiny neurons in anaesthetised rats: an in vivo intracellular study. Proceedings of the International Australasian Winter Conference on Brain Research (2005), Queenstown, NZ
114. Ring H.A., and SerraMestres J. Neuropsychiatry of the basal ganglia. J. Neurol. Neurosurg. Psychiat. 72 (2002) 12-21
115. Robbins T.W., and Sahakian B.J. Behavioral and neurochemical determinants of drug-induced stereotypy. In: Rose F.C. (Ed). Metabolic Disorder of the Nervous System (1981), Pitman Pr, London 244-291
116. Roesch M.R., and Olson C.R. Neuronal activity related to reward value and motivation in primate frontal cortex. Science 304 5668 (2004) 307-310
117. Roitman M.F., Stuber G.D., Phillips P.E.M., Wightman R.M., and Carelli R.M. Dopamine operates as a subsecond modulator of food seeking. J. Neurosci. 24 6 (2004) 1265-1271
118. Rousselet G.A., Thorpe S.J., and Fabre-Thorpe M. How parallel is visual processing in the ventral pathway. Trends Cogn. Sci. 8 8 (2004) 363-370
119. Saka E., Goodrich C., Harlan P., Madras B.K., and Graybiel A.M. Repetitive behaviors in monkeys are linked to specific striatal activation patterns. J. Neurosci. 24 34 (2004) 7557-7565
120. Salamone J.D., and Correa M. Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine. Behav. Brain Res. 137 (2002) 3-25
121. Samejima K., Ueda Y., Doya K., and Kimura M. Representation of action-specific reward values in the striatum. Science 310 5752 (2005) 1337-1340
122. Satoh T., Nakai S., Sato T., and Kimura M. Correlated coding of motivation and outcome of decision by dopamine neurons. J. Neurosci. 23 30 (2003) 9913-9923
123. Schall J.D. Neural correlates of decision processes: neural and mental chronometry. Curr. Opin. Neurobiol. 13 (2003) 182-186
124. Schoenbaum G., Setlow B., Saddoris M.P., and Gallagher M. Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala. Neuron 39 5 (2003) 855-867
125. Schultz W. Predictive reward signal of dopamine neurons. J. Neurophysiol. 80 (1998) 1-27
126. Schultz W. Multiple reward signals in the brain. Nat. Rev. Neurosci. 1 (2000) 199-207
127. Schultz W. Behavioral theories and the neurophysiology of reward. Annu. Rev. Psychol. 57 (2006) 87-115
128. Schultz W. Multiple dopamine functions at different time courses. Annu. Rev. Neurosci. 30 (2007) 259-288
129. Schultz W., and Romo R. Responses of nigrostriatal dopamine neurons to high-intensity somatosensory stimulation in the anesthetized monkey. J. Neurophysiol. 57 1 (1987) 201-217
130. Schultz W., and Romo R. Dopamine neurons of the monkey midbrain: contingencies of responses to stimuli eliciting immediate behavioural reactions. J. Neurophysiol. 63 (1990) 607-624
131. Schultz W., Apicella P., Romo R., and Scarnati E. Context-dependent activity in primate striatum reflecting past and future behavioural events. In: Houk J.C., Davis J.L., and Beiser D.G. (Eds). Models of Information Processing in the Basal Ganglia (1995), MIT Press, Cambridge, MA 11-27
132. Schultz W., Dayan P., and Montague P.R. A neural substrate of prediction and reward. Science 275 (1997) 1593-1599
133. Seymour B., O'Doherty J.P., Koltzenburg M., Wiech K., Frackowiak R., Friston K., and Dolan R. Opponent appetitive-aversive neural processes underlie predictive learning of pain relief. Nat. Neurosci. 8 9 (2005) 1234-1240
134. Sheldon A.B. Preference for familiar versus novel stimuli as a function of the familiarity of the environment. J. Comp. Physiol. Psychol. 67 (1969) 516-521
135. Shepard P.D., Holcomb H.H., and Gold J.M. The presence of absence: habenular regulation of dopamine neurons and the encoding of negative outcomes. Schizophr. Bull. 32 3 (2006) 417-421
136. Singh S.P., Barto A.G., and Chentanez N. Intrinsically motivated reinforcement learning. In: Saul L.K., Weiss H., and Bottou L. (Eds). Advances in Neural Information Processing Systems vol 17 (2005), MIT Press, Cambridge, MA 1281-1288
137. Smeets W.J.A.J., Marin O., and Gonzalez A. Evolution of the basal ganglia: new perspectives through a comparative approach. J. Anat. 196 (2000) 501-517
138. Smith Y., Raju D.V., Pare J.F., and Sidibe M. The thalamostriatal system: a highly specific network of the basal ganglia circuitry. Trends Neurosci. 27 9 (2004) 520-527
139. Snaith S., and Holland O. An investigation of two mediation strategies suitable for behavioural control in animals and animats. In: Meyer J.-A., and Wilson S. (Eds). From Animals to Animats: Proceedings of the First International Conference on the Simulation of Adaptive Behaviour (1990), MIT Press, Cambridge, MA 255-262
140. Sparks D.L. Translation of sensory signals into commands for control of saccadic eye movements: role of the primate superior colliculus. Physiol. Rev. 66 (1986) 118-171
141. Sprague J.M., Marchiafava P.L., and Rixxolatti G. Unit responses to visual stimuli in the superior colliculus of the unanesthetized, mid-pontine cat. Arch. Ital. Biol. 106 3 (1968) 169-193
142. Stein B.E., and Meredith M.A. The Merging of the Senses (1993), The MIT Press, Cambridge, MA
143. Sugase-Miyamoto Y., and Richmond B.J. Neuronal signals in the monkey basolateral amygdala during reward schedules. J. Neurosci. 25 48 (2005) 11071-11083
144. Sutton R.S., and Barto A.G. Reinforcement Learning - an Introduction (1998), MIT Press, Cambridge, MA.
145. Takikawa Y., Kawagoe R., and Hikosaka O. A possible role of midbrain dopamine neurons in short-and long-term adaptation of saccades to position-reward mapping. J. Neurophysiol. 92 4 (2004) 2520-2529
146. Tang K., Low M.J., Grandy D.K., and Lovinger D.M. Dopamine-dependent synaptic plasticity in striatum during in vivo development. Proc. Natl. Acad. Sci. U. S. A. 98 3 (2001) 1255-1260
147. Thorndike E.L. Animal Intelligence (1911), Macmillan, New York
148. Thorpe S.J., and Fabre-Thorpe M. Seeking categories in the brain. Science 291 (2001) 260-263
149. Tobler P.N., Dickinson A., and Schultz W. Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. J. Neurosci. 23 32 (2003) 10402-10410
150. Tobler P.N., Fiorillo C.D., and Schultz W. Adaptive coding of reward value by dopamine neurons. Science 307 5715 (2005) 1642-1645
151. Tobler P.N., O'Doherty J.P., Dolan R.J., and Schultz W. Human neural learning depends on reward prediction errors in the blocking paradigm. J. Neurophysiol. 95 1 (2006) 301-310
152. Tsai C.-T., Nakamura S., and Iwama K. Inhibition of neuronal activity of the substantia nigra by noxious stimuli and its modification by the caudate nucleus. Brain Res. 195 (1980) 299-311
153. Ungless M.A. Dopamine: the salient issue. Trends Neurosci. 27 12 (2004) 702-706
154. Ungless M.A., Magill P.J., and Bolam J.P. Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science 303 5666 (2004) 2040-2042
155. Venton B.J., and Wightman R.M. Pharmacologically induced, subsecond dopamine transients in the caudate-putamen of the anesthetized rat. Synapse 61 1 (2007) 37-39
156. Waelti P., Dickinson A., and Schultz W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 412 (2001) 43-48
157. Watanabe M., Hikosaka K., Sakagami M., and Shirakawa S. Coding and monitoring of motivational context in the primate prefrontal cortex. J. Neurosci. 22 (2002) 2391-2400
158. Wickens J. A Theory of the Striatum (1993), Pergamon, Oxford
159. Wickens J.R., Reynolds J.N.J., and Hyland B.I. Neural mechanisms of reward-related motor learning. Curr. Opin. Neurobiol. 13 6 (2003) 685-690
160. Wightman R.M., and Robinson D.L. Transient changes in mesolimbic dopamine and their association with 'reward'. J. Neurochem. 82 (2002) 721-735
161. Williams G.V., and Millar J. Concentration-dependent actions of stimulated dopamine release on neuronal activity in rat striatum. Neuroscience 39 1 (1990) 1-16
162. Windmann S., Kirsch P., Mier D., Stark R., Walter B., Gunturkun O., and Vaitl D. On framing effects in decision making: linking lateral versus medial orbitofrontal cortex activation to choice outcome processing. J. Cogn. Neurosci. 18 7 (2006) 1198-1211
163. Wise R.A. Dopamine, learning and motivation. Nat. Rev. Neurosci. 5 6 (2004) 483-494
164. Wurtz R.H., and Albano J.E. Visual-motor function of the primate superior colliculus. Ann Rev Neurosci 3 (1980) 189-226
165. Young A.M.J. Increased extracellular dopamine in nucleus accumbens in response to unconditioned and conditioned aversive stimuli: studies using 1 min microdialysis in rats. J. Neurosci. Meth. 138 1-2 (2004) 57-63