Dopamine reward prediction-error signalling: A two-component response

Nature Reviews Neuroscience

Volumn 17, Issue 3, 2016, Pages 183-195

  Schultz, Wolfram ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

DOPAMINE;

AVERSIVE BEHAVIOR; BEHAVIOR; DECISION MAKING; DOPAMINE RELEASE; DOPAMINE UPTAKE; DOPAMINERGIC NERVE CELL; DOPAMINERGIC SYSTEM; HUMAN; NERVE CELL PLASTICITY; NONHUMAN; PREDICTION; PREDICTION ERROR SIGNAL; PRIORITY JOURNAL; REVIEW; REWARD; SALIENCE NETWORK; SIGNAL TRANSDUCTION; STIMULUS RESPONSE; ANIMAL; METABOLISM; NERVE CELL; PHYSIOLOGY;

ANIMALS; DOPAMINE; HUMANS; NEURONS; REWARD; SIGNAL TRANSDUCTION;

EID: 84959086568     PISSN: 1471003X     EISSN: 14710048     Source Type: Journal    
DOI: 10.1038/nrn.2015.26     Document Type: Review

References (169)

1. Schultz, W. Multiple dopamine functions at different time courses. Ann. Rev. Neurosci. 30, 259-288 (2007)
2. Ljungberg, T., Apicella, P., & Schultz, W. Responses of monkey dopamine neurons during learning of behavioral reactions. J. Neurophysiol. 67, 145-163 (1992)
3. Schultz, W., Dayan, P., & Montague, R. R. A neural substrate of prediction and reward. Science 275, 1593-1599 (1997)
4. Schultz, W. Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1-27 (1998)
5. Waelti, P., Dickinson, A., & Schultz, W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 412, 43-48 (2001)
6. Tobler, P. N., Dickinson, A., & Schultz, W. Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. J. Neurosci. 23, 10402-10410 (2003)
7. Bayer, H. M., & Glimcher, P. W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129-141 (2005)
8. Pan, W. X., Schmidt, R., Wickens, J. R., & Hyland, B. I. Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J. Neurosci. 25, 6235-6242 (2005)
9. Lak, A., Stauffer, W. R., & Schultz, W. Dopamine prediction error responses integrate subjective value from different reward dimensions. Proc. Natl Acad. Sci. USA 111, 2343-2348 (2014)
10. Fiorillo, C. D., Tobler, P. N., & Schultz, W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299, 1898-1902 (2003)
11. Schultz, W., Ruffieux, A., & Aebischer, P. The activity of pars compacta neurons of the monkey substantia nigra in relation to motor activation. Exp. Brain Res. 51, 377-387 (1983)
12. Schultz, W. Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. J. Neurophysiol. 56, 1439-1462 (1986)
13. DeLong, M. R., Crutcher, M. D., & Georgopoulos, A. P. Relations between movement and single cell discharge in the substantia nigra of the behaving monkey. J. Neurosci. 3, 1599-1606 (1983)
14. Romo, R., & Schultz, W. Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. J. Neurophysiol. 63, 592-606 (1990)
15. Chiodo, L. A., Antelman, S. M., Caggiula, A. R., & Lineberry, C. G. Sensory stimuli alter the discharge rate of dopamine (DA) neurons: evidence for two functional types of DA cells in the substantia nigra. Brain Res. 189, 544-549 (1980)
16. Steinfels, G. F., Heym, J., Strecker, R. E., & Jacobs, B. L. Behavioral correlates of dopaminergic unit activity in freely moving cats. Brain Res. 258, 217-228 (1983)
17. Schultz, W., & Romo, R. Responses of nigrostriatal dopamine neurons to high intensity somatosensory stimulation in the anesthetized monkey. J. Neurophysiol. 57, 201-217 (1987)
18. Schultz, W., & Romo, R. Dopamine neurons of the monkey midbrain: contingencies of responses to stimuli eliciting immediate behavioral reactions. J. Neurophysiol. 63, 607-624 (1990)
19. Mirenowicz, J., & Schultz, W. Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379, 449-451 (1996)
20. Horvitz, J. C., Stewart, T., & Jacobs, B. L. Burst activity of ventral tegmental dopamine neurons is elicited by sensory stimuli in the awake cat. Brain Res. 759, 251-258 (1997)
21. Guarraci, F. A., & Kapp, B. S. An electrophysiological characterization of ventral tegmental area dopaminergic neurons during differential pavlovian fear conditioning in the awake rabbit. Behav. Brain Res. 99, 169-179 (1999)
22. Joshua, M., Adler, A., Mitelman, R., Vaadia, E., & Bergman, H. Midbrain dopaminergic neurons and striatal cholinergic interneurons encode the difference between reward and aversive events at different epochs of probabilistic classical conditioning trials. J. Neurosci. 28, 11673-11684 (2008)
23. Brischoux, F., Chakraborty, S., Brierley, D. I., & Ungless, M. A. Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc. Natl Acad. Sci. USA 106, 4894-4899 (2009)
24. Matsumoto, M., & Hikosaka, O. Two types of dopamine neuron distinctively convey positive and negative motivational signals. Nature 459, 837-841 (2009)
25. Fiorillo, C. D., Song, M. R., & Yun, S. R. Multiphasic temporal dynamics in responses of midbrain dopamine neurons to appetitive and aversive stimuli. J. Neurosci. 33, 4710-4725 (2013)
26. Fiorillo, C. D., Yun, S. R., & Song, M. R. Diversity and homogeneity in responses of midbrain dopamine neurons. J. Neurosci. 33, 4693-4709 (2013)
27. Fiorillo, C. D. Two dimensions of value: dopamine neurons represent reward but not aversiveness. Science 341, 546-549 (2013)
28. Thorpe, S. J., Rolls, E. T., & Maddison, S. The orbitofrontal cortex: neuronal activity in the behaving monkey. Exp. Brain Res. 49, 93-115 (1983)
29. Ravel, S., Legallet, E., & Apicella, P. Responses of tonically active neurons in the monkey striatum discriminate between motivationally opposing stimuli. J. Neurosci. 23, 8489-8497 (2003)
30. Roitman, M. F., Wheeler, R. A., & Carelli, R. M. Nucleus accumbens neurons are innately tuned for rewarding and aversive taste stimuli, encode their predictors, and are linked to motor output. Neuron 45, 587-597 (2005)
31. Paton, J. J., Belova, M. A., Morrison, S. E., & Salzman, C. D. The primate amygdala represents the positive and negative value of visual stimuli during learning. Nature 439, 865-870 (2006)
32. Amemori, K.-I., & Graybiel, A. M. Localized microstimulation of primate pregenual cingulate cortex induces negative decision-making. Nat. Neurosci. 15, 776-785 (2012)
33. Nomoto, K., Schultz, W., Watanabe, T., & Sakagami, M. Temporally extended dopamine responses to perceptually demanding reward-predictive stimuli. J. Neurosci. 30, 10692-10702 (2010)
34. Kobayashi, S., & Schultz, W. Reward contexts extend dopamine signals to unrewarded stimuli. Curr. Biol. 24, 56-62 (2014)
35. Stauffer, W. R., Lak, A., & Schultz, W. Dopamine reward prediction error responses reflect marginal utility. Curr. Biol. 24, 2491-2500 (2014)
36. Bushnell, M. C., Goldberg, M. E., & Robinson, D. L. Behavioral enhancement of visual responses in monkey cerebral cortex. I. Modulation in posterior parietal cortex related to selective visual attention. J. Neurophysiol. 46, 755-772 (1981)
37. Treue, S., & Maunsell, J. H. R. Attentional modulation of visual motion processing in cortical areas MT and MST. Nature 382, 539-541 (1996)
38. Womelsdorf, T., Anton-Erxleben, K., Pieper, F., & Treue, S. Dynamic shifts of visual receptive fields in cortical area MT by spatial attention. Nat. Neurosci. 9, 1156-1160 (2006)
39. Nardo, D., Santangelo, V., & Macaluso, E. Stimulus-driven orienting of visuo-spatial attention in complex dynamic environments. Neuron 69, 1015-1028 (2011)
40. Annic, A., Bocquillon, P., Bourriez, J. L., Derambure, P., & Dujardin, K. Effects of stimulus-driven and goal-directed attention on prepulse inhibition of the cortical responses to an auditory pulse. Clin. Neurophysiol. 125, 1576-1588 (2014)
41. Thompson, K. G., Hanes, D. P., Bichot, N. P., & Schall, J. D. Perceptual and motor processing stages identified in the activity of macaque frontal eye field neurons during visual search. J. Neurophysiol. 76, 4040-4055 (1996)
42. Ipata, A. E., Gee, A. L., Bisley, J. W., & Goldberg, M. E. Neurons in the lateral intraparietal area create a priority map by the combination of disparate signals. Exp. Brain Res. 192, 479-488 (2009)
43. Ipata, A. E., Gee, A. L., & Goldberg, M. E. Feature attention evokes task-specific pattern selectivity in V4 neurons. Proc. Natl Acad. Sci. USA 109, 16778-16785 (2012)
44. Pooresmaeili, A., Poort, J., & Roelfsema, P. R. Simultaneous selection by object-based attention in visual and frontal cortex. Proc. Natl Acad. Sci. USA 111, 6467-6472 (2014)
45. Shadlen, M. N., & Newsome, W. T. Neural basis of a perceptual decision in the parietal cortex (Area LIP) of the rhesus monkey. J. Neurophysiol. 86, 1916-1936 (2001)
46. Roitman, J. D., & Shadlen, M. N. Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J. Neurosci. 22, 9475-9489 (2002)
47. Ringach, D. L., Hawken, M. J., & Shapley, R. Dynamics of orientation tuning in macaque primary visual cortex. Nature 387, 281-284 (1997)
48. Sugase, Y., Yamane, S., Ueno, S., & Kawano, K. Global and fine information coded by single neurons in the temporal visual cortex. Nature 400, 869-873 (1999)
49. Bredfeldt, C. E., & Ringach, D. L. Dynamics of spatial frequency tuning in macaque V1. J. Neurosci. 22, 1976-1984 (2002
50. Hedg , J., & Van Essen, D. C. Temporal dynamics of shape analysis in macaque visual area V2. J. Neurophysiol. 92, 3030-3042 (2004)
51. Roelfsema, P. R., Tolboom, M., & Khayat, P. S. Different processing phases for features, figures, and selective attention in the primary visual cortex. Neuron 56, 785-792 (2007)
52. Hedg , J. Time course of visual perception: Coarse-To fine processing and beyond. Prog. Neurobiol. 84, 405-439 (2008)
53. Lak, A., Arabzadeh, E., Harris, J. A., & Diamond, M. E. Correlated physiological and perceptual effects of noise in a tactile stimulus. Proc. Natl Acad. Sci. USA 107, 7981-7986 (2010)
54. Hung, C. P., Kreiman, G., Poggio, T., & DiCarlo, J. J. Fast readout of object identity from macaque inferior temporal cortex. Science 310, 863-866 (2005)
55. Ambroggi, F., Ishikawa, A., Fields, H. L., & Nicola, S. M. Basolateral amygdala neurons facilitate reward-seeking behavior by exciting nucleus accumbens neurons. Neuron 59, 648-661 (2008)
56. Peck, C. J., Lau, B., & Salzman, C. D. The primate amygdala combines information about space and value. Nat. Neurosci. 16, 340-348 (2013)
57. Mogami, T., & Tanaka, K. Reward association affects neuronal responses to visual stimuli in macaque TE and perirhinal cortices. J. Neurosci. 26, 6761-6770 (2006)
58. Stanisor, L., van der Togt, C., Pennartz, C. M. A., & Roelfsema, P. R. A unified selection signal for attention and reward in primary visual cortex. Proc. Natl Acad. Sci. USA 110, 9136-9141 (2013)
59. Morris, G., Arkadir, D., Nevet, A., Vaadia, E., & Bergman, H. Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron 43, 133-143 (2004)
60. Day, J. J., Roitman, M. F., & Wightman, R. M., & Carelli, R. M. Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat. Neurosci. 10, 1020-1028 (2007)
61. Kobayashi, S., & Schultz, W. Influence of reward delays on responses of dopamine neurons. J. Neurosci. 28, 7837-7846 (2008)
62. Fiorillo, C. D., Newsome, W. T., & Schultz, W. The temporal precision of reward prediction in dopamine neurons. Nat. Neurosci. 11, 966-973 (2008)
63. Budygin, E. A., et al. Aversive stimulus differentially triggers subsecond dopamine release in reward regions. Neuroscience 201, 331-337 (2012)
64. Redgrave, P., Prescott, T J., & Gurney, K. Is the short-latency dopamine response too short to signal reward?. Trends Neurosci. 22, 146-151 (1999)
65. Redgrave, P., & Gurney, K. The short-latency dopamine signal: a role in discovering novel actions?. Nat. Rev. Neurosci. 7, 967-975 (2006)
66. Schultz, W., Apicella, P., & Ljungberg, T. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J. Neurosci. 13, 900-913 (1993)
67. Mitchell, D. S., & Gormezano, I. Effects of water deprivation on classical appetitive conditioning of the rabbit's jaw movement response. Learn. Motivat. 1, 199-206 (1970)
68. Mackintosh, N J. The Psychology of Animal Learning (Academic Press, 1974)
69. Ljungberg, T., Apicella, P., & Schultz, W. Responses of monkey midbrain dopamine neurons during delayed alternation performance. Brain Res. 586, 337-341 (1991)
70. Pearce, J. M., & Hall, G. A model for Pavlovian conditioning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol. Rev. 87, 532-552 (1980)
71. Chelazzi, L., et al. Altering spatial priority maps via reward-based learning. J. Neurosci. 34, 8594-8604 (2014)
72. Darwin, C. On the Origin of Species by Natural Selection, or the Preservation of Favoured Races in the Struggle for Life (John Murray, 1859)
73. Montague, P. R., Dayan, P., & Sejnowski, T. J. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 16, 1936-1947 (1996)
74. Tobler, P. N., Fiorillo, C. D., & Schultz, W. Adaptive coding of reward value by dopamine neurons. Science 307, 1642-1645 (2005)
75. Enomoto, K., et al. Dopamine neurons learn to encode the long-Term value of multiple future rewards. Proc. Natl Acad. Sci. USA 108, 15462-15467 (2011)
76. Richards, J. B., Mitchell, S. H., de Wit, H., & Seiden, L. S. Determination of discount functions in rats with an adjusting-Amount procedure. J. Exp. Anal. Behav. 67, 353-366 (1997)
77. Day, J. J., Jones, J. L., Wightman, R. M., & Carelli, R. M. Phasic nucleus accumbens dopamine release encodes effort-And delay-related costs. Biol. Psychiat. 68, 306-309 (2010)
78. Pascal, B. Penss (1658-1662) (Hackett, 2004)
79. Bernoulli, D. Specimen theoriae novae de mensura sortis. Comentarii Academiae Scientiarum Imperialis Petropolitanae 5, 175-192 (1738); English translation available in Exposition of a new theory on the measurement of risk. Econometrica 22, 23-36 (1954)
80. Bentham, J An Introduction to the Principle of Morals and Legislations (Blackwell, 1948)
81. von Neumann, J., & Morgenstern, O. The Theory of Games and Economic Behavior (Princeton Univ. Press, 1944)
82. Kahneman, D., & Tversky, A. Prospect theory: an analysis of decision under risk. Econometrica 47, 263-291 (1979)
83. Luce, R. D. Individual Choice Behavior: A Theoretical Analysis (Wiley, 1959)
84. Kagel, J. H., Battalio, R. C., & Green, L. Economic Choice Theory: An Experimental Analysis of Animal Behavior (Cambridge Univ. Press, 1995)
85. Mas-Colell, A., Whinston, M., & Green, J. Microeconomic Theory (Oxford Univ. Press, 1995)
86. Savage, L J. The Foundations of Statistics (Wiley, 1954)
87. Debreu, G. Cardinal utility for even-chance mixtures of pairs of sure prospects. Rev. Econ. Stud. 26, 174-177 (1959)
88. Rothschild, M., & Stiglitz, J. E. Increasing risk: I. A definition. J. Econ. Theory 2, 225-243 (1970)
89. Caraco, T., Martindale, S., & Whitham, T. S. An empirical demonstration of risk-sensitive foraging preferences. Anim. Behav. 28, 820-830 (1980)
90. Machina, M. J. Choice under uncertainty: problems solved and unsolved. J. Econ. Perspect. 1, 121-154 (1987)
91. McCoy, A. N., & Platt, M. L. Risk-sensitive neurons in macaque posterior cingulate cortex. Nat. Neurosci. 8, 1220-1227 (2005)
92. O'Neill, M., & Schultz, W. Coding of reward risk distinct from reward value by orbitofrontal neurons. Neuron 68, 789-800 (2010)
93. Prelec, D., & Loewenstein, G. Decision making over time and under uncertainty: a common approach. Management Sci. 37, 770-786 (1991)
94. Weber, B. J., & Chapman, G. B. Playing for peanuts: why is risk seeking more common for low-stakes gambles?. Organ. Behav. Hum. Decis. Process. 97, 31-46 (2005)
95. Fehr-Duda, H., Bruhin, A., Epper, T., & Schubert, R. Rationality on the rise: why relative risk aversion increases with stake size. J. Risk Uncertain. 40, 147-180 (2010)
96. Kreps, D M. A Course in Microeconomic Theory (Pearson Education, 1990)
97. Friedman, M., & Savage, L. J. The utility analysis of choices involving risk. J. Polit. Econ. 56, 279-304 (1948)
98. Markowitz, H. The utility of wealth. J. Polit. Econ. 6, 151-158 (1952)
99. Jones, S. R., Garris, P. A., & Wightman, R. M. Different effects of cocaine and nomifensine on dopamine uptake in the caudate-putamen and nucleus accumbens. J. Pharmacol. Exp. Ther. 274, 396-403 (1995)
100. Calabresi, P., et al. Dopamine and cAMP-regulated phosphoprotein 32 kDa controls both striatal long-Term depression and long-Term potentiation, opposing forms of synaptic plasticity. J. Neurosci. 20, 8443-8451 (2000)
101. Gurden, H., Takita, M., & Jay, T. M. Essential role of D1 but not D2 receptors in the NMDA receptor-dependent long-Term potentiation at hippocampal-prefrontal cortex synapses in vivo. J. Neurosci. 20, RC106 (2000)
102. Reynolds, J. N. J., Hyland, B. I., & Wickens, J. R. A cellular mechanism of reward-related learning. Nature 413, 67-70 (2001)
103. Pawlak, V., & Kerr, J. N. D. Dopamine receptor activation is required for corticostriatal spike-Timing-dependent plasticity. J. Neurosci. 28, 2435-2446 (2008)
104. Shen, W., Flajolet, M., Greengard, P., & Surmeier, D. J. Dichotomous dopaminergic control of striatal synaptic plasticity. Science 321, 848-851 (2008)
105. Zhang, J. C., Lau, P. M., & Bi, G. Q. Gain in sensitivity and loss in temporal contrast of STDP by dopaminergic modulation at hippocampal synapses. Proc. Natl Acad. Sci. USA 106, 1328-1333 (2009)
106. Yagishita, S., et al. A critical time window for dopamine actions on the structural plasticity of dendritic spines. Science 345, 1616-1620 (2014)
107. Tsai, H. C., et al. Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science 324, 1080-1084 (2009)
108. Witten, I. B., et al. Recombinase-driver rat lines: tools, techniques, and optogenetic application to dopamine-mediated reinforcement. Neuron 72, 721-733 (2011)
109. Adamantidis, A. R., et al. Optogenetic interrogation of dopaminergic modulation of the multiple phases of reward-seeking behavior. J. Neurosci. 31, 10829-10835 (2011)
110. Kim, K. M., et al. Optogenetic mimicry of the transient activation of dopamine neurons by natural reward is sufficient for operant reinforcement. PLoS ONE 7, e33612 (2012)
111. Steinberg, E. E., et al. A causal link between prediction errors, dopamine neurons and learning. Nat. Neurosci. 16, 966-973 (2013)
112. Ilango, A., et al. Similar roles of substantia nigra and ventral tegmental dopamine neurons in reward and aversion. J. Neurosci. 34, 817-822 (2014)
113. Kravitz, A. V., Tye, L. D., & Kreitzer, A. C. Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nat. Neurosci. 15, 816-818 (2012)
114. Schultz, W. Neuronal reward and decision signals: from theories to data. Physiol. Rev. 95, 853-951 (2015)
115. Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. Midbrain dopamine neurons encode decisions for future action. Nat. Neurosci. 9, 1057-1063 (2006)
116. Padoa-Schioppa, C., & Assad, J. A. Neurons in the orbitofrontal cortex encode economic value. Nature 441, 223-226 (2006)
117. Cai, X., Kim, S., & Lee, D. Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice. Neuron 69, 170-182 (2011)
118. Kennerley, S. W., Behrens, T. E. J., & Wallis, J. D. Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nat. Neurosci. 14, 1581-1589 (2011)
119. So, N. Y., & Stuphorn, V. Supplementary eye field encodes reward prediction error. J. Neurosci. 32, 2950-2963 (2012)
120. Samejima, K., Ueda, Y., Doya, K., & Kimura, M. Representation of action-specific reward values in the striatum. Science 310, 1337-1340 (2005)
121. Lau, B., & Glimcher, P. W. Value representations in the primate striatum during matching behavior. Neuron 58, 451-463 (2008)
122. Ito, M., & Doya, K. Validation of decision-making models and analysis of decision variables in the rat basal ganglia. J. Neurosci. 29, 9861-9874 (2009)
123. Kim, H., Sul, J. H., Huh, N., Lee, D., & Jung, M. W. Role of striatum in updating values of chosen actions. J. Neurosci. 29, 14701-14712 (2009)
124. Seo, M., Lee, E., & Averbeck, B. B. Action selection and action value in frontal-striatal circuits. Neuron 74, 947-960 (2012)
125. Brown, J. R., & Arbuthnott, G. W. The electrophysiology of dopamine (D2) receptors: a study of the actions of dopamine on corticostriatal transmission. Neuroscience 10, 349-355 (1983)
126. Toan, D. L., & Schultz, W. Responses of rat pallidum cells to cortex stimulation and effects of altered dopaminergic activity. Neuroscience 15, 683-694 (1985)
127. Mink, J. W. The basal ganglia: focused selection and inhibition of competing motor programs. Prog. Neurobiol. 50, 381-425 (1996)
128. Zweifel, L. S., et al. Disruption of NMDAR-dependent burst firing by dopamine neurons provides selective assessment of phasic dopamine-dependent behavior. Proc. Natl Acad. Sci. USA 106, 7281-7288 (2009)
129. Aosaki, T., Graybiel, A. M., & Kimura, M. Effect of the nigrostriatal dopamine system on acquired neural responses in the striatum of behaving monkeys. Science 265, 412-415 (1994)
130. Herández-Lépz, S., Bargas, J., Surmeier, D. J., Reyes, A., & Galarraga, E. D1 receptor activation enhances evoked discharge in neostriatal medium spiny neurons by modulating an L type Ca2+ conductance. J. Neurosci. 17, 3334-3342 (1997
131. Hernαndez-L?pez, S., et al. D2 dopamine receptors in striatal medium spiny neurons reduce L type Ca2+ currents and excitability via a novel PLCβ1 IP3-calcineurin-signaling cascade. J. Neurosci. 20, 8987-8995 (2000)
132. Tai, L. H., & Lee, A. M., Benavidez, N., Bonci, A., & Wilbrecht, L. Transient stimulation of distinct subpopulations of striatal neurons mimics changes in action value. Nat. Neurosci. 15, 1281-1289 (2012)
133. Badrinarayan, A., et al. Aversive stimuli differentially modulate real-Time dopamine transmission dynamics within the nucleus accumbens core and shell. J. Neurosci. 32, 15779-15790 (2012)
134. Lerner, T., et al. Intact-brain analyses reveal distinct information carried by SNc dopamine subcircuits. Cell 162, 635-647 (2015)
135. Solomon, R. L., & Corbit, J. D. An opponent-process theory of motivation. Psychol. Rev. 81, 119-145 (1974)
136. Oleson, E. B., Gentry, R. N., Chioma, V. C., & Cheer, J. F. Subsecond dopamine release in the nucleus accumbens predicts conditioned punishment and its successful avoidance. J. Neurosci. 32, 14804-14808 (2012)
137. Lammel, S., et al. Input-specific control of reward and aversion in the ventral tegmental area. Nature 491, 212-217 (2012)
138. Stopper, C. M., Tse, M. T. L., Montes, D. R., Wiedman, C. R., & Floresco, S. B. Overriding phasic dopamine signals redirects action selection during risk/reward decision making. Neuron 84, 177-189 (2014)
139. Christoph, G. R., Leonzio, R. J., & Wilcox, K. S. Stimulation of the lateral habenula inhibits dopamine-containing neurons in the substantia nigra and ventral tegmental area of the rat. J. Neurosci. 6, 613-619 (1986)
140. Ji, H., & Shepard, P. D. Lateral habenula stimulation inhibits rat midbrain dopamine neurons through a GABAA receptor-mediated mechanism. J. Neurosci. 27, 6923-6930 (2007)
141. Matsumoto, M., & Hikosaka, O. Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 447, 1111-1115 (2007)
142. Kapur, S. Psychosis as a state of aberrant salience: a framework linking biology, phenomenology, and pharmacology in schizophrenia. Am. J. Psychiatry 160, 13-23 (2003)
143. Robinson, T. E., & Berridge, K. C. The neural basis for drug craving: an incentive-sensitization theory of addiction. Brain Res. Rev. 18, 247-291 (1993)
144. Saunders, B. T., & Robinson, T. E. The role of dopamine in the accumbens core in the expression of Pavlovian-conditioned responses. Eur. J. Neurosci. 36, 2521-2532 (2012)
145. Rao, R. P. N., & Ballard, D. H. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci. 2, 79-87 (1999)
146. Roeper, J. Dissecting the diversity of midbrain dopamine neurons. Trends Neurosci. 36, 336-342 (2013)
147. Fuster, J. M. Unit activity of prefrontal cortex during delayed-response performance: neuronal correlates of transient memory. J. Neurophysiol. 36, 61-78 (1973)
148. Apicella, P., Scarnati, E., Ljungberg, T., & Schultz, W. Neuronal activity in monkey striatum related to the expectation of predictable environmental events. J. Neurophysiol. 68, 945-960 (1992)
149. Hollerman, J. R., Tremblay, L., & Schultz, W. Influence of reward expectation on behavior-related neuronal activity in primate striatum. J. Neurophysiol. 80, 947-963 (1998)
150. Matsumoto, M., & Takada, M. Distinct representations of cognitive and motivational signals in midbrain dopamine neurons. Neuron 79, 1011-1024 (2013)
151. Satoh, T., Nakai, S., Sato, T., & Kimura, M. Correlated coding of motivation and outcome of decision by dopamine neurons. J. Neurosci. 23, 9913-9923 (2003)
152. Bromberg-Martin, E. S., & Hikosaka, O. Lateral habenula neurons signal errors in the prediction of reward information. Nature Neurosci. 14, 1209-1216 (2011)
153. de Lafuente, O., & Romo, R. Dopamine neurons code subjective sensory experience and uncertainty of perceptual decisions. Proc. Natl Acad. Sci. USA 49, 19767-19771 (2011)
154. Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., & Uchida, N. Neuron-Type-specific signals for reward and punishment in the ventral tegmental area. Nature 482, 85-88 (2012)
155. Stuber, G. D., Wightman, R. M., & Carelli, R. M. Extinction of cocaine self-Administration reveals functionally and temporally distinct dopaminergic signals in the nucleus accumbens. Neuron 46, 661-669 (2005)
156. Howe, M. W., Tierney, P. L., Sandberg, S. G., Phillips, P. E. M., & Graybiel, A. M. Prolonged dopamine signalling in striatum signals proximity and value of distant rewards. Nature 500, 575-579 (2013)
157. Chesselet, M. F. Presynaptic regulation of neurotransmitter release in the brain: facts and hypothesis. Neuroscience 12, 347-375 (1984)
158. Sugam, J. A., Day, J. J., Wightman, R. M., & Carelli, R. M. Phasic nucleus accumbens dopamine encodes risk-based decision-making behavior. Biol. Psychiat. 71, 199-205 (2012)
159. Threlfell, S., et al. Striatal dopamine release is triggered by synchronized activity in cholinergic interneurons. Neuron 75, 58-64 (2012)
160. Chuhma, N., Mingote, S., Moore, H., & Rayport, S. Dopamine neurons control striatal cholinergic neurons via regionally heterogeneous dopamine and glutamate signaling. Neuron 81, 901-912 (2014)
161. Brimblecombe, K. R., & Cragg, S. J. Substance P weights striatal dopamine transmission differently within the striosome-matrix axis. J. Neurosci. 35, 9017-9023 (2015)
162. Robbins, T. W., & Arnsten, A. F. T. The neuropsychopharmacology of fronto-executive function: monoaminergic modulation. Ann. Rev. Neurosci. 32, 267-287 (2009)
163. Young, A. M. J., Joseph, M. H., & Gray, J. A. Increased dopamine release in vivo in nucleus accumbens and caudate nucleus of the rat during drinking: a microdialysis study. Neuroscience 48, 871-876 (1992)
164. Grace, A. A. Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: a hypothesis for the etiology of schizophrenia. Neuroscience 41, 1-24 (1991)
165. Datla, K. P., Ahier, R. G., Young, A. M. J., Gray, J. A., & Joseph, M. H. Conditioned appetitive stimulus increases extracellular dopamine in the nucleus accumbens of the rat. Eur. J. Neurosci. 16, 1987-1993 (2002)
166. Cheng, J. J., de Bruin, J. P. C., & Feenstra, M. G. P. Dopamine efflux in nucleus accumbens shell and core in response to appetitive classical conditioning. Eur. J. Neurosci. 18, 1306-1314 (2003)
167. Young, A. M. J. Increased extracellular dopamine in nucleus accumbens in response to unconditioned and conditioned aversive stimuli: studies using 1 min microdialysis in rats. J. Neurosci. Meth. 138, 57-63 (2004)
168. Anzalone, A., et al. Dual control of dopamine synthesis and release by presynaptic and postsynaptic dopamine D2 receptors. J. Neurosci. 32, 9023-9034 (2012)
169. Grace, A. A., Floresco, S. B., Goto, Y., & Lodge, D. J. Regulation of firing of dopaminergic neurons and control of goal-directed behaviors. Trends Neurosci. 30, 220-227 (2007