Brief optogenetic inhibition of dopamine neurons mimics endogenous negative reward prediction errors

Nature Neuroscience

Volumn 19, Issue 1, 2015, Pages 111-116

  Chang, Chun Yun ^a   Esber, Guillem R ^b   Marrero Garcia, Yasmin ^a   Yau, Hau Jie ^a   Bonci, Antonello ^a,c   Schoenbaum, Geoffrey ^a,c,d  

^a NATIONAL INSTITUTE ON DRUG ABUSE   (United States)

^b CITY UNIVERSITY OF NEW YORK   (United States)

^c JOHNS HOPKINS UNIVERSITY SCHOOL OF MEDICINE   (United States)

^d UNIVERSITY OF MARYLAND SCHOOL OF MEDICINE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CONTROLLED STUDY; DOPAMINERGIC NERVE CELL; ERROR; FEMALE; MALE; NERVE CELL INHIBITION; NONHUMAN; OPTOGENETICS; PREDICTION; PREDICTION ERROR; PRIORITY JOURNAL; RAT; REWARD; TASK PERFORMANCE; VENTRAL TEGMENTUM; ANIMAL; ANIMAL BEHAVIOR; ANTICIPATION; CONDITIONED REFLEX; ELECTROPHYSIOLOGY; LONG EVANS RAT; PHYSIOLOGY; TRANSGENIC RAT;

ANIMALS; ANTICIPATION, PSYCHOLOGICAL; BEHAVIOR, ANIMAL; CONDITIONING, CLASSICAL; DOPAMINERGIC NEURONS; ELECTROPHYSIOLOGICAL PHENOMENA; FEMALE; MALE; NEURAL INHIBITION; OPTOGENETICS; RATS; RATS, LONG-EVANS; RATS, TRANSGENIC; REWARD; VENTRAL TEGMENTAL AREA;

EID: 84953837909     PISSN: 10976256     EISSN: 15461726     Source Type: Journal    
DOI: 10.1038/nn.4191     Document Type: Article

References (50)

1. Rescorla R.A., & Wagner A.R. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. in Classical Conditioning II: Current Research and Theory (eds. Black A.H., & Prokasy W.F.) 64-99 (Appleton-Century-Crofts, New York, 1972).
2. Sutton R.S. Learning to predict by the method of temporal difference. Mach. Learn. 3, 9-44 (1988).
3. Mirenowicz J., & Schultz W. Importance of unpredictability for reward responses in primate dopamine neurons. J. Neurophysiol. 72, 1024-1027 (1994).
4. Roesch M.R., Calu D.J., & Schoenbaum G. Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat. Neurosci. 10, 1615-1624 (2007).
5. Waelti P., Dickinson A., & Schultz W. Dopamine responses comply with basic assumptions of formal learning theory. Nature. 412, 43-48 (2001).
6. Pan W.-X., Schmidt R., Wickens J.R., & Hyland B.I. Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J. Neurosci. 25, 6235-6242 (2005).
7. Dardenne K., McClure S.M., Nystrom L.E., & Cohen J.D. BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science. 319, 1264-1267 (2008).
8. Day J.J., Roitman M.F., Wightman R.M., & Carelli R.M. Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat. Neurosci. 10, 1020-1028 (2007).
9. Hart A.S., Rutledge R.B., Glimcher P.W., & Phillips P.E. Phasic dopamine release in the rat nucleus accumbens symmetrically encodes a reward prediction error term. J. Neurosci. 34, 698-704 (2014).
10. Schultz W., Dayan P., & Montague P.R. A neural substrate of prediction and reward. Science. 275, 1593-1599 (1997).
11. Redgrave P., Gurney K., & Reynolds J. What is reinforced by phasic dopamine signals? Brain Res. Rev. 58, 322-339 (2008).
12. Zweifel L.S., et al. Disruption of NMDAR-dependent burst firing by dopamine neurons provides selective assessment of phasic dopamine-dependent behavior. Proc. Natl. Acad. Sci. USA. 106, 7281-7288 (2009).
13. Steinberg E.E., et al. A causal link between prediction errors, dopamine neurons and learning. Nat. Neurosci. 16, 966-973 (2013).
14. Tsai H.C., et al. Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science. 324, 1080-1084 (2009).
15. Frank M.J., Moustafa A.A., Haughey H.M., Curran T., & Hutchison K.E. Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc. Natl. Acad. Sci. USA. 104, 16311-16316 (2007).
16. Kim K.M., et al. Optogenetic mimicry of the transient activation of dopamine neurons by natural reward is sufficient for operant reinforcement. PLoS One 7, e33612 (2012).
17. Stopper C.M., Tse M.T., Montes D.R., Wiedman C.R., & Floresco S.B. Overriding phasic dopamine signals redirects action selection during risk/reward decision making. Neuron. 84, 177-189 (2014).
18. Shumake J., Ilango A., Scheich H., Wetzel W., & Ohl F.W. Differential neuromodulation of acquisition and retrieval of avoidance learning by the lateral habenula and ventral tegmental area. J. Neurosci. 30, 5876-5883 (2010).
19. Stamatakis A.M., & Stuber G.D. Activation of lateral habenula inputs to the ventral midbrain promotes behavioral avoidance. Nat. Neurosci. 15, 1105-1107 (2012).
20. Danna C.L., Shepard P.D., & Elmer G.I. The habenula governs the attribution of incentive salience to reward predictive cues. Front. Hum. Neurosci. 7, 781 (2013).
21. Bayer H.M., Lau B., & Glimcher P.W. Statistics of midbrain dopamine neuron spike trains in the awake primate. J. Neurophysiol. 98, 1428-1439 (2007).
22. Glimcher P.W. Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. Proc. Natl. Acad. Sci. USA 108 (suppl. 3), 15647-15654 (2011).
23. Kakade S., & Dayan P. Dopamine: generalization and bonuses. Neural Netw. 15, 549-559 (2002).
24. Matsumoto M., & Hikosaka O. Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature. 459, 837-841 (2009).
25. Pearce J.M., Kaye H., & Hall G. Predictive accuracy and stimulus associability: development of a model for Pavlovian learning. in Quantitative Analyses of Behavior (eds. Commons M.L., Herrnstein R.J., & Wagner A.R.) 241-255 (Ballinger, Cambridge, Massachusetts, USA, 1982).
26. Esber G.R., & Haselgrove M. Reconciling the influence of predictiveness and uncertainty on stimulus salience: a model of attention in associative learning. Proc. R. Soc. Lond. B. 278, 2553-2561 (2011).
27. Rescorla R.A. Renewal after overexpectation. Learn. Behav. 35, 19-26 (2007).
28. Rescorla R.A. Spontaneous recovery from overexpectation. Learn. Behav. 34, 13-20 (2006).
29. Niv Y., Daw N.D., Joel D., & Dayan P. Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl.) 191, 507-520 (2007).
30. Salamone J.D., & Correa M. The mysterious motivational functions of mesolimbic dopamine. Neuron. 76, 470-485 (2012).
31. Berridge K.C., & Robinson T.E. What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res. Brain Res. Rev. 28, 309-369 (1998).
32. Ikemoto S., Yang C., & Tan A. Basal ganglia circuit loops, dopamine and motivation: a review and enquiry. Behav. Brain Res. 290, 17-31 (2015).
33. Takahashi Y.K., et al. The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes. Neuron. 62, 269-280 (2009).
34. Takahashi Y.K., et al. Neural estimates of imagined outcomes in the orbitofrontal cortex drive behavior and learning. Neuron. 80, 507-518 (2013).
35. Rescorla R.A. Reduction in the effectiveness of reinforcement after prior excitatory conditioning. Learn. Motiv. 1, 372-381 (1970).
36. Witten I.B., et al. Recombinase-driver rat lines: tools, techniques, and optogenetic application to dopamine-mediated reinforcement. Neuron. 72, 721-733 (2011).
37. Lammel S., et al. Diversity of transgenic mouse models for selective targeting of midbrain dopamine neurons. Neuron. 85, 429-438 (2015).
38. Stamatakis A.M., et al. A unique population of ventral tegmental area neurons inhibits the lateral habenula to promote reward. Neuron. 80, 1039-1053 (2013).
39. Stuber G.D., Stamatakis A.M., & Kantak P.A. Considerations when using cre-driver rodent lines for studying ventral tegmental area circuitry. Neuron. 85, 439-445 (2015).
40. Yamaguchi T., Qi J., Wang H.-L., Zhang S., & Morales M. Glutamatergic and dopaminergic neurons in the mouse ventral tegmental area. Eur. J. Neurosci. 41, 760-772 (2015).
41. Li X., Qi J., Yamaguchi T., Wang H.-L., & Morales M. Heterogeneous composition of dopamine neurons of the rat A10 region: molecular evidence for diverse signaling properties. Brain Struct. Funct. 218, 1159-1176 (2013).
42. Root D.H., et al. Norepinephrine activates dopamine D4 receptors in the rat lateral habenula. J. Neurosci. 35, 3460-3469 (2015).
43. Tecuapetla F., et al. Glutamatergic signaling by mesolimbic dopamine neurons in the nucleus accumbens. J. Neurosci. 30, 7105-7110 (2010).
44. Stuber G.D., Hnasko T.S., Britt J.P., Edwards R.H., & Bonci A. Dopaminergic terminals in the nucleus accumbens but not the dorsal striatum corelease glutamate. J. Neurosci. 30, 8229-8233 (2010).
45. Zhang S., et al. Dopaminergic and glutamatergic microdomains in a subset of rodent mesoaccumbens axons. Nat. Neurosci. 18, 386-392 (2015).
46. Matsumoto M., & Hikosaka O. Lateral habenula as a source of negative reward signals in dopamine neurons. Nature. 447, 1111-1115 (2007).
47. Hong S., Jhou T.C., Smith M., Saleem K.S., & Hikosaka O. Negative reward signals from the lateral habenula to dopamine neurons are mediated by rostromedial tegmental nucleus in primates. J. Neurosci. 31, 11457-11471 (2011).
48. Ji H., & Shepard P.D. Lateral habenula stimulation inhibits rat midbrain dopamine neurons through a GABAA receptor-mediated mechanism. J. Neurosci. 27, 6923-6930 (2007).
49. Mileykovskiy B., & Morales M. Duration of inhibition of ventral tegmental area dopamine neurons encodes a level of conditioned fear. J. Neurosci. 31, 7471-7476 (2011).
50. Ilango A., et al. Similar roles of substantia nigra and ventral tegmental dopamine neurons in reward and aversion. J. Neurosci. 34, 817-822 (2014).