Electrophysiological correlates of reward processing in dopamine neurons

Handbook of Reward and Decision Making

Volumn , Issue , 2009, Pages 29-50

  Tobler, Philippe N ^a   Kobayashi, Shunsuke ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 79960206048     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1016/B978-0-12-374620-7.00002-9     Document Type: Chapter

References (117)

1. Batson J.D., Best M.R. Single-element assessment of conditioned inhibition. Bull. Psychon. Soc. 1981, 18:328-330.
2. Dickinson A. Contemporary animal learning theory 1980, Cambridge, Cambridge University Press.
3. Thorndike E.L. Animal intelligence: experimental studies 1911, Macmillan, New York.
4. Watson J.B. Behavior: an introduction to comparative psychology 1914, Holt, Rinehart and Winston, New York.
5. Hull C.L. Principles of behavior 1943, Appleton-Century-Crofts, New York.
6. Mackintosh N.J. Conditioning and associative learning 1983, Oxford University Press, Oxford.
7. Pavlov I.P. Conditional Reflexes 1927/1960, Dover Publications, New York, (the 1960 edition is an unaltered republication of the 1927 translation by Oxford University Press).
8. Tolman E.C. Purposive behavior in animals and men 1932, Century, New York.
9. Tolman E.C. There is more than one kind of learning. Psychol. Rev. 1949, 56:144-155.
10. Konorski J. Conditioned reflexes and neuron organization 1948/1968, Hafner Publishing, London, (the edition of 1968 is a facsimile reprint of the 1948 edition by Cambridge University Press).
11. Konorski J. Integrative action of the brain 1967, University of Chicago Press, Chicago, IL.
12. Dickinson A., Balleine B. Motivational control of goal-directed action. Anim. Learn. Behav. 1994, 22:1-18.
13. Gallistel C.R., Gibbon J. Time, rate, and conditioning. Psychol. Rev. 2000, 107:289-344.
14. Hume D. A treatise of human nature 1739/2000, John Noon, London, (edited in 2000 by D. F. Norton & M. J. Norton. Oxford: Oxford University Press).
15. Pearce J.M. A model for stimulus generalization in Pavlovian conditioning. Psychol. Rev. 1987, 94:61-73.
16. Rescorla R.A., Furrow D.R. Stimulus similarity as a determinant of Pavlovian conditioning. J. Exp. Psychol. Anim. Behav. Process. 1977, 3:203-215.
17. Trobalon J.B., Miguelez D., McLaren I.P., Mackintosh N.J. Intradimensional and extradimensional shifts in spatial learning. J. Exp. Psychol. Anim. Behav. Process. 2003, 29:143-152.
18. Gibbon J., Baldock M.D., Locurto C., Gold L., Terrace H.S. Trial and intertrial durations in autoshaping. J. Exp. Psychol. Anim. Behav. Process. 1977, 3:264-284.
19. Williams B.A. Information effects on the response-reinforcer association. Anim. Learn. Behav. 1978, 6:371-379.
20. Schneiderman N. Response system divergencies in aversive classical conditioning. Classical conditioning II: current research and theory 1972, 341-376. Appleton-Century-Crofts, New York. A. Black, W.F. Prokasy (Eds.).
21. Heth C.D., Rescorla R.A. Simultaneous and backward fear conditioning in the rat. J. Comp. Physiol. Psychol. 1973, 82:434-443.
22. Hellstern F., Malaka R., Hammer M. Backward inhibitory learning in honeybees: a behavioral analysis of reinforcement processing. Learn. Mem. 1998, 4:429-444.
23. Plotkin H.C., Oakley D.A. Backward conditioning in the rabbit (Oryctolagus cuniculus). J. Comp. Physiol. Psychol. 1975, 88:586-590.
24. Rescorla R.A. Pavlovian conditioning and its proper control procedures. Psychol. Rev. 1967, 74:71-80.
25. Rescorla R.A. Probability of shock in the presence and absence of CS in fear conditioning. J. Comp. Physiol. Psychol. 1968, 66:1-5.
26. Rescorla R.A. Conditioned inhibition of fear resulting from negative CS-US contingencies. J. Comp. Physiol. Psychol. 1969, 67:504-509.
27. Kamin L.J. Predictability, surprise, attention and conditioning. Punishment and aversive behavior 1969, 279-296. Appleton-Century-Crofts, New York. B.A. Campbell, R.M. Church (Eds.).
28. Rescorla R.A. Variation in the effectiveness of reinforcement and nonreinforcement following prior inhibitory conditioning. Learn. Motiv. 1971, 2:113-123.
29. Rescorla R.A. Establishment of a positive reinforcer through contrast with shock. J. Comp. Physiol. Psychol. 1969, 67:260-263.
30. Reiss S., Wagner A.R. CS habituation produces a "latent inhibition effect" but no active "conditioned inhibition". Learn. Motiv. 1972, 3:237-245.
31. Rescorla R.A. Summation and retardation tests of latent inhibition. J. Comp. Physiol. Psychol. 1971, 75:77-81.
32. Rescorla R.A., Wagner A.R. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. Classical conditioning II: current research and theory 1972, 64-99. Appleton-Century-Crofts, New York. A. Black, W.F. Prokasy (Eds.).
33. Mackintosh N.J. A theory of attention: variations in the associability of stimuli with reinforcement. Psychol. Rev. 1975, 82:276-298.
34. Wagner A.R. Expectancies and the priming of STM. Cognitive processes in animal behavior 1978, 177-209. Lawrence Erlbaum, Hillsdale, NJ. S.H. Hulse, H. Fowler, W.K. Honig (Eds.).
35. Pearce J.M., Hall G. A model of Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol. Rev. 1980, 87:532-552.
36. Sutton R.S., Barto A.G. Toward a modern theory of adaptive networks: expectation and prediction. Psychol. Rev. 1981, 88:135-170.
37. Sutton R.S., Barto A.G. Time-derivative models of Pavlovian reinforcement. Learning and computational neuroscience: foundations of adaptive networks 1990, 497-537. MIT Press, Boston, MA. M. Gabriel, J. Moore (Eds.).
38. Sutton R.S. Learning to predict by the method of temporal difference. Mach. Learn. 1988, 3:9-44.
39. Gallistel C.R. The organization of learning 1990, MIT Press, Boston, MA.
40. Bower G.H. A contrast effect in differential conditioning. J. Exp. Psychol. 1961, 62:196-199.
41. Harper D.G.C. Competitive foraging in mallards: ideal free ducks. Anim. Behav. 1982, 30:575-584.
42. Cromwell H.C., Schultz W. Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. J. Neurophysiol. 2003, 89:2823-2838.
43. Glimcher P.W. Decisions, uncertainty, and the brain: the science of neuroeconomics 2003, MIT Press, Boston, MA.
44. Bernoulli D. Exposition of a new theory on the measurement of risk. Econometrica 1738/1954, 22:23-36. (translated from Latin).
45. Kahneman D., Tversky A. Prospect theory: an analysis of decision under risk. Econometrica 1979, 47:263-291.
46. von Neumann J., Morgenstern O. Theory of games and economic behavior 1944, Princeton University Press, Princeton, NJ.
47. Tversky A., Kahneman D. The framing of decisions and the psychology of choice. Science 1981, 211:453-458.
48. Hertwig R., Barron G., Weber E.U., Erev I. Decisions from experience and the effect of rare events. Psychol. Sci. 2004, 15:534-539.
49. Samuelson P.A. Some aspects of the pure theory of capital. Q. J. Economics 1937, 51:469-496.
50. Ainslie G. Specious rewards: a behavioral theory of impulsiveness and impulse control. Psychol. Bull. 1975, 82:463-496.
51. Mazur J.E. An adjusting procedure for studying delayed reinforcement. Quantitative analyses of behavior 1987, pp., vol. 5. Erlbaum, Hillsdale, NJ. M.L. Commons, J.E. Mazur, J.A. Nevin, H. Rachlin (Eds.).
52. Richards J.B., Mitchell S.H., de Wit H., Seiden L.S. Determination of discount functions in rats with an adjusting-amount procedure. J. Exp. Anal. Behav. 1997, 67:353-366.
53. DeLong M.R., Crutcher M.D., Georgopoulos A.P. Relations between movement and single cell discharge in the substantia nigra of the behaving monkey. J. Neurosci. 1983, 3:1599-1606.
54. Grace A.A., Bunney B.S. Nigral dopamine neurons: intracellular recording and identification with L-dopa injection and histofluorescence. Science 1980, 210:654-656.
55. Schultz W., Romo R. Responses of nigrostriatal dopamine neurons to high-intensity somatosensory stimulation in the anesthetized monkey. J. Neurophysiol. 1987, 57:201-217.
56. Lammel S., Hetzel A., Häckel O., Jones I., Liss B., Roeper J. Unique properties of mesoprefrontal neurons within a dual mesocorticolimbic dopamine system. Neuron 2008, 57:760-773.
57. Sesack S.R., Pickel V.M. Prefrontal cortical efferents in the rat synapse on unlabeled neuronal targets of catecholamine terminals in the nucleus accumbens septi and on dopamine neurons in the ventral tegmental area. J. Comp. Neurol. 1992, 320:145-160.
58. Futami T., Takakusaki K., Kitai S.T. Glutamatergic and cholinergic inputs from the pedunculopontine tegmental nucleus to dopamine neurons in the substantia nigra pars compacta. Neurosci. Res. 1995, 21:331-342.
59. Fudge J.L., Haber S.N. Bed nucleus of the stria terminalis and extended amygdala inputs to dopamine subpopulations in primates. Neuroscience 2001, 104:807-827.
60. Georges F., Aston-Jones G. Activation of ventral tegmental area cells by the bed nucleus of the stria terminalis: a novel excitatory amino acid input to midbrain dopamine neurons. J. Neurosci. 2002, 22:5173-5187.
61. Geisler S., Zahm D.S. Afferents of the ventral tegmental area in the rat-anatomical substratum for integrative functions. J. Comp. Neurol. 2005, 490:270-294.
62. McHaffie J.G., Jiang H., May P.J., Coizet V., Overton P.G., Stein B.E., Redgrave P. A direct projection from superior colliculus to substantia nigra pars compacta in the cat. Neuroscience 2006, 138:221-234.
63. Araki M., McGeer P.L., Kimura H. The efferent projections of the rat lateral habenular nucleus revealed by the PHA-L anterograde tracing method. Brain Res. 1988, 441:319-330.
64. Floresco S.B., West A.R., Ash B., Moore H., Grace A.A. Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission. Nat. Neurosci. 2003, 6:968-973.
65. Lodge D.J., Grace A.A. The laterodorsal tegmentum is essential for burst firing of ventral tegmental area dopamine neurons. Proc. Natl. Acad. Sci. USA 2006, 103:5167-5172.
66. Wu M., Hrycyshyn A.W., Brudzynski S.M. Subpallidal outputs to the nucleus accumbens and ventral tegmental area: anatomical and electrophysiological studies. Brain Res. 1996, 740:151-161.
67. Haber S.N., Fudge J.L. The primate substantia nigra and VTA: integrative circuitry and function. Crit. Rev. Neurobiol. 1997, 11:323-342.
68. Gonon F. Nonlinear relationship between impulse flow and dopamine released by rat midbrain dopaminergic neurons as studied by in vivo electrochemistry. Neuroscience 1988, 24:19-28.
69. Kawagoe K.T., Garris P.A., Wiedemann D.J., Wightman R.M. Regulation of transient dopamine concentration gradients in the microenvironment surrounding nerve terminals in the rat striatum. Neuroscience 1992, 51:55-64.
70. Cepeda C., Buchwald N.A., Levine M.S. Neuromodulatory actions of dopamine in the neostriatum are dependent upon the excitatory amino acid receptor subtypes activated. Proc. Natl. Acad. Sci. USA 1993, 90:9576-9580.
71. Gonon F., Sundstrom L. Excitatory effects of dopamine released by impulse flow in the rat nucleus accumbens in vivo. Neuroscience 1996, 75:13-18.
72. Steinfels G.F., Heym H., Jacobs B.L. Single unit activity of dopaminergic neurons in freely moving animals. Life Sci. 1981, 29:1435-1442.
73. Horvitz J.C., Stewart T., Jacobs B.L. Burst activity of ventral tegmental dopamine neurons is elicited by sensory stimuli in the awake cat. Brain Res. 1997, 759:251-258.
74. Comoli E., Coizet V., Boyes J., Bolam J.P., Canteras N.S., Quirk R.H., Overton P.G., Redgrave P. A direct projection from superior colliculus to substantia nigra for detecting salient visual events. Nat. Neurosci. 2003, 6:974-980.
75. Dommett E., Coizet V., Blaha C.D., Martindale J., Lefebvre V., Walton N., Mayhew J.E., Overton P.G., Redgrave P. How visual stimuli activate dopaminergic neurons at short latency. Science 2005, 307:1476-1479.
76. Redgrave P., Mitchell I.J., Dean P. Descending projections from the superior colliculus in rat: a study using orthograde transport of wheatgerm-agglutinin conjugated horseradish peroxidase. Exp. Brain Res. 1987, 68:147-167.
77. Beninato M., Spencer R.F. A cholinergic projection to the rat substantia nigra from the pedunculopontine tegmental nucleus. Brain Res. 1987, 412:169-174.
78. Beninato M., Spencer R.F. The cholinergic innervation of the rat substantia nigra: a light and electron microscopic immunohistochemical study. Exp. Brain Res. 1988, 72:178-184.
79. Bolam J.P., Francis C.M., Henderson Z. Cholinergic input to dopaminergic neurons in the substantia nigra: a double immunocytochemical study. Neuroscience 1991, 41:483-494.
80. Oakman S.A., Faris P.L., Kerr P.E., Cozzari C., Hartman B.K. Distribution of pontomesencephalic cholinergic neurons projecting to substantia nigra differs significantly from those projecting to ventral tegmental area. J. Neurosci. 1995, 15:5859-5869.
81. Pan W.X., Hyland B.I. Pedunculopontine tegmental nucleus controls conditioned responses of midbrain dopamine neurons in behaving rats. J. Neurosci. 2005, 25:4725-4732.
82. Kobayashi Y., Okada K. Reward prediction error computation in the pedunculopontine tegmental nucleus neurons. Ann. NY Acad. Sci. 2007, 1104:310-323.
83. Lokwan S.J., Overton P.G., Berry M.S., Clark D. Stimulation of the pedunculopontine tegmental nucleus in the rat produces burst firing in A9 dopaminergic neurons. Neuroscience 1999, 92:245-254.
84. Inglis W.L., Olmstead M.C., Robbins T.W. Pedunculopontine tegmental nucleus lesions impair stimulus-reward learning in autoshaping and conditioned reinforcement paradigms. Behav. Neurosci. 2000, 114:285-294.
85. Alderson H.L., Brown V.J., Latimer M.P., Brasted P.J., Robertson A.H., Winn P. The effect of excitotoxic lesions of the pedunculopontine tegmental nucleus on performance of a progressive ratio schedule of reinforcement. Neuroscience 2002, 112:417-425.
86. Tsai C.T., Nakamura S., Iwama K. Inhibition of neuronal activity of the substantia nigra by noxious stimuli and its modification by the caudate nucleus. Brain Res. 1980, 195:299-311.
87. Maeda H., Mogenson G.J. Effects of peripheral stimulation on the activity of neurons in the ventral tegmental area, substantia nigra and midbrain reticular formation of rats. Brain Res. Bull. 1982, 8:7-14.
88. Mirenowicz J., Schultz W. Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 1996, 379:449-451.
89. Brischoux F., Chakraborty S. Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc. Natl. Acad. Sci. U. S. A. 2009, 106:4894-4899.
90. Sorg B.A., Kalivas P.W. Effects of cocaine and footshock stress on extracellular dopamine levels in the ventral striatum. Brain Res. 1991, 559:29-36.
91. Young A.M., Joseph M.H., Gray J.A. Latent inhibition of conditioned dopamine release in rat nucleus accumbens. Neuroscience 1993, 54:5-9.
92. Romo R., Chéramy A., Godeheu G., Glowinski J. In vivo presynaptic control of dopamine release in the cat caudate nucleus - III. Further evidence for the implication of corticostriatal glutamatergic neurons,. Neuroscience 1986, 19:1091-1099.
93. Keefe K.A., Sved A.F., Zigmond M.J., Abercrombie E.D. Stress-induced dopamine release in the neostriatum: evaluation of the role of action potentials in nigrostriatal dopamine neurons or local initiation by endogenous excitatory amino acids. J. Neurochem. 1993, 61:1943-1952.
94. Romo R., Schultz W. Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. J. Neurophysiol. 1990, 63:592-606.
95. Ljungberg T., Apicella P., Schultz W. Responses of monkey dopamine neurons during learning of behavioral reactions. J. Neurophysiol. 1992, 67:145-163.
96. Mirenowicz J., Schultz W. Importance of unpredictability for reward responses in primate dopamine neurons. J. Neurophysiol. 1994, 72:1024-1027.
97. Miller J.D., Sanghera M.K., German D.C. Mesencephalic dopaminergic unit activity in the behaviorally conditioned rat. Life Sci. 1981, 29:1255-1263.
98. Schultz W. Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. J. Neurophysiol. 1986, 56:1439-1461.
99. Schultz W., Romo R. Dopamine neurons of the monkey midbrain: contingencies of responses to stimuli eliciting immediate behavioral reactions. J. Neurophysiol. 1990, 63:607-624.
100. Ljungberg T., Apicella P., Schultz W. Responses of monkey midbrain dopamine neurons during delayed alternation performance. Brain Res. 1991, 567:337-341.
101. Hollerman J.R., Schultz W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci. 1998, 1:304-309.
102. Day J.J., Roitman M.F., Wightman R.M., Carelli R.M. Associative learning mediates dynamic shifts in dopamine signalling in the nucleus accumbens. Nat. Neurosci. 2007, 10:1020-1028.
103. Schultz W., Dayan P., Montague P.R. A neural substrate of prediction and reward. Science 1997, 275:1593-1599.
104. Waelti P., Dickinson A., Schultz W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 2001, 412:43-48.
105. Matsumoto M., Hikosaka O. Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 2007, 447:1111-1115.
106. Fiorillo C.D., Newsome W.T., Schultz W. The temporal precision of reward prediction in dopamine neurons. Nat. Neurosci. 2008, 11:966-973.
107. Kobayashi S., Schultz W. Influence of reward delays on responses of dopamine neurons. J. Neurosci. 2008, 28:7837-7846.
108. Church R.M., Gibbon J. Temporal generalization. J. Exp. Psychol. Anim. Behav. Process. 1982, 8:165-186.
109. Rescorla R.A. Pavlovian conditioned inhibition. Psychol. Bull. 1969, 72:77-94.
110. Tobler P.N., Dickinson A., Schultz W. Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. J. Neurosci. 2003, 23:10402-10410.
111. Tobler P.N., Fiorillo C.D., Schultz W. Adaptive coding of reward value by dopamine neurons. Science 2005, 307:1642-1645.
112. Fiorillo C.D., Tobler P.N., Schultz W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 2003, 299:1898-1902.
113. Satoh T., Nakai S., Sato T., Kimura M. Correlated coding of motivation and outcome of decision by dopamine neurons. J. Neurosci. 2003, 23:9913-9923.
114. Roesch M.R., Calu D.J., Schoenbaum G. Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat. Neurosci. 2007, 10:1615-1624.
115. Morris G., Nevet A., Arkadir D., Vaadia E., Bergman H. Midbrain dopamine neurons encode decisions for future action. Nat. Neurosci. 2006, 9:1057-1063.
116. Bayer H.M., Glimcher P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 2005, 47:129-141.
117. Bayer H.M., Lau B., Glimcher P.W. Statistics of midbrain dopamine neuron spike trains in the awake primate. J. Neurophysiol. 2007, 98:1428-1439.