Reward and choice encoding in terminals of midbrain dopamine neurons depends on striatal target

Nature Neuroscience

Volumn 19, Issue 6, 2016, Pages 845-854

  Parker, Nathan F ^a   Cameron, Courtney M ^a   Taliaferro, Joshua P ^a   Lee, Junuk ^a   Choi, Jung Yoon ^a   Davidson, Thomas J ^b   Daw, Nathaniel D ^a   Witten, Ilana B ^a  

^a PRINCETON UNIVERSITY   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ANIMAL CELL; ANIMAL EXPERIMENT; ARTICLE; CALCIUM SIGNALING; CELL BODY; CELL SUBPOPULATION; CONTROLLED STUDY; DECISION MAKING; DOPAMINERGIC NERVE CELL; DORSAL STRIATUM; DORSOMEDIAL STRIATUM; ELECTROPHYSIOLOGICAL PROCEDURES; INSTRUMENTAL REVERSAL LEARNING; MESENCEPHALON; MOUSE; NONHUMAN; PRIORITY JOURNAL; REVERSAL LEARNING; REWARD; TASK PERFORMANCE; VENTRAL STRIATUM; ANIMAL; C57BL MOUSE; CORPUS STRIATUM; LEARNING; MALE; METABOLISM; PHYSIOLOGY; TRANSGENIC MOUSE;

DOPAMINE;

ANIMALS; CHOICE BEHAVIOR; CORPUS STRIATUM; DOPAMINE; DOPAMINERGIC NEURONS; LEARNING; MALE; MESENCEPHALON; MICE, INBRED C57BL; MICE, TRANSGENIC; REWARD;

EID: 84964311329     PISSN: 10976256     EISSN: 15461726     Source Type: Journal    
DOI: 10.1038/nn.4287     Document Type: Article

References (52)

1. Morris, G., Nevet, A., Arkadir, D., Vaadia, E. & Bergman, H. Midbrain dopamine neurons encode decisions for future action. Nat. Neurosci. 9, 1057-1063 (2006).
2. Schultz, W., Dayan, P. & Montague, P.R. A neural substrate of prediction and reward. Science 275, 1593-1599 (1997).
3. Starr, B.S. & Starr, M.S. Differential effects of dopamine D1 and D2 agonists and antagonists on velocity of movement, rearing and grooming in the mouse. Implications for the roles of D1 and D2 receptors. Neuropharmacology 25, 455-463 (1986).
4. Wise, R.A. Dopamine, learning and motivation. Nat. Rev. Neurosci. 5, 483-494 (2004).
5. Marshall, J.F. & Berrios, N. Movement disorders of aged rats: reversal by dopamine receptor stimulation. Science 206, 477-479 (1979).
6. Arbuthnott, G.W. & Crow, T.J. Relation of contraversive turning to unilateral release of dopamine from the nigrostriatal pathway in rats. Exp. Neurol. 30, 484-491 (1971).
7. Hollerman, J.R. & Schultz, W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci. 1, 304-309 (1998).
8. Schultz, W. Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1-27 (1998).
9. Witten, I.B. et al. Recombinase-driver rat lines: tools, techniques, and optogenetic application to dopamine-mediated reinforcement. Neuron 72, 721-733 (2011).
10. Tsai, H.-C. et al. Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science 324, 1080-1084 (2009).
11. Nestler, E.J. & Carlezon, W.A. Jr. The mesolimbic dopamine reward circuit in depression. Biol. Psychiatry 59, 1151-1159 (2006).
12. Niv, Y., Daw, N.D., Joel, D. & Dayan, P. Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl.) 191, 507-520 (2007).
13. Nicola, S.M., Surmeier, J. & Malenka, R.C. Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. Annu. Rev. Neurosci. 23, 185-215 (2000).
14. Graybiel, A.M. The basal ganglia. Curr. Biol. 10, R509-R511 (2000).
15. Domesick, V.B. Neuroanatomical organization of dopamine neurons in the ventral tegmental area. Ann. NY Acad. Sci. 537, 10-26 (1988).
16. Lammel, S., Ion, D.I., Roeper, J. & Malenka, R.C. Projection-specific modulation of dopamine neuron synapses by aversive and rewarding stimuli. Neuron 70, 855-862 (2011).
17. Lynd-Balta, E. & Haber, S.N. The organization of midbrain projections to the striatum in the primate: sensorimotor-related striatum versus ventral striatum. Neuroscience 59, 625-640 (1994).
18. Lammel, S. et al. Unique properties of mesoprefrontal neurons within a dual mesocorticolimbic dopamine system. Neuron 57, 760-773 (2008).
19. Lerner, T.N. et al. Intact-brain analyses reveal distinct information carried by SNc dopamine subcircuits. Cell 162, 635-647 (2015).
20. Tai, L.-H., Lee, A.M., Benavidez, N., Bonci, A. & Wilbrecht, L. Transient stimulation of distinct subpopulations of striatal neurons mimics changes in action value. Nat. Neurosci. 15, 1281-1289 (2012).
21. Balleine, B.W., Delgado, M.R. & Hikosaka, O. The role of the dorsal striatum in reward and decision-making. J. Neurosci. 27, 8161-8165 (2007).
22. Graybiel, A.M., Aosaki, T., Flaherty, A.W. & Kimura, M. The basal ganglia and adaptive motor control. Science 265, 1826-1831 (1994).
23. Roitman, M.F., Wheeler, R.A. & Carelli, R.M. Nucleus accumbens neurons are innately tuned for rewarding and aversive taste stimuli, encode their predictors, and are linked to motor output. Neuron 45, 587-597 (2005).
24. Ikemoto, S. & Panksepp, J. The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking. Brain Res. Brain Res. Rev. 31, 6-41 (1999).
25. Chen, T.-W. et al. Ultrasensitive fluorescent proteins for imaging neuronal activity. Nature 499, 295-300 (2013).
26. Gunaydin, L.A. et al. Natural neural projection dynamics underlying social behavior. Cell 157, 1535-1551 (2014).
27. Lau, B. & Glimcher, P.W. Value representations in the primate striatum during matching behavior. Neuron 58, 451-463 (2008).
28. Lau, B. & Glimcher, P.W. Dynamic response-by-response models of matching behavior in rhesus monkeys. J. Exp. Anal. Behav. 84, 555-579 (2005).
29. Lammel, S. et al. Diversity of transgenic mouse models for selective targeting of midbrain dopamine neurons. Neuron 85, 429-438 (2015).
30. Cui, G. et al. Concurrent activation of striatal direct and indirect pathways during action initiation. Nature 494, 238-242 (2013).
31. Hyland, B.I., Reynolds, J.N.J., Hay, J., Perk, C.G. & Miller, R. Firing modes of midbrain dopamine cells in the freely moving rat. Neuroscience 114, 475-492 (2002).
32. Pan, W.-X., Schmidt, R., Wickens, J.R. & Hyland, B.I. Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J. Neurosci. 25, 6235-6242 (2005).
33. Cohen, J.Y., Haesler, S., Vong, L., Lowell, B.B. & Uchida, N. Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482, 85-88 (2012).
34. Roesch, M.R., Calu, D.J. & Schoenbaum, G. Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat. Neurosci. 10, 1615-1624 (2007).
35. Bayer, H.M. & Glimcher, P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129-141 (2005).
36. Day, J.J., Roitman, M.F., Wightman, R.M. & Carelli, R.M. Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat. Neurosci. 10, 1020-1028 (2007).
37. Ilango, A. et al. Similar roles of substantia nigra and ventral tegmental dopamine neurons in reward and aversion. J. Neurosci. 34, 817-822 (2014).
38. Gershman, S.J., Pesaran, B. & Daw, N.D. Human reinforcement learning subdivides structured action spaces by learning effector-specific values. J. Neurosci. 29, 13524-13531 (2009).
39. O'Reilly, R.C. & Frank, M.J. Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Comput. 18, 283-328 (2006).
40. Diuk, C., Tsai, K., Wallis, J., Botvinick, M. & Niv, Y. Hierarchical learning induces two simultaneous, but separable, prediction errors in human basal ganglia. J. Neurosci. 33, 5797-5805 (2013).
41. Matsumoto, M. & Hikosaka, O. Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459, 837-841 (2009).
42. Mirenowicz, J. & Schultz, W. Importance of unpredictability for reward responses in primate dopamine neurons. J. Neurophysiol. 72, 1024-1027 (1994).
43. Mirenowicz, J. & Schultz, W. Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379, 449-451 (1996).
44. Stefani, M.R. & Moghaddam, B. Rule learning and reward contingency are associated with dissociable patterns of dopamine activation in the rat prefrontal cortex, nucleus accumbens, and dorsal striatum. J. Neurosci. 26, 8810-8818 (2006).
45. Howe, M.W., Tierney, P.L., Sandberg, S.G., Phillips, P.E.M. & Graybiel, A.M. Prolonged dopamine signalling in striatum signals proximity and value of distant rewards. Nature 500, 575-579 (2013).
46. Phillips, P.E.M., Stuber, G.D., Heien, M.L.A.V., Wightman, R.M. & Carelli, R.M. Subsecond dopamine release promotes cocaine seeking. Nature 422, 614-618 (2003).
47. Hart, A.S., Rutledge, R.B., Glimcher, P.W. & Phillips, P.E.M. Phasic dopamine release in the rat nucleus accumbens symmetrically encodes a reward prediction error term. J. Neurosci. 34, 698-704 (2014).
48. Hamid, A.A. et al. Mesolimbic dopamine signals the value of work. Nat. Neurosci. 19, 117-126 (2016).
49. Willuhn, I., Burgeno, L.M., Everitt, B.J. & Phillips, P.E.M. Hierarchical recruitment of phasic dopamine signaling in the striatum during the progression of cocaine use. Proc. Natl. Acad. Sci. USA 109, 20703-20708 (2012).
50. Brown, H.D., McCutcheon, J.E., Cone, J.J., Ragozzino, M.E. & Roitman, M.F. Primary food reward and reward-predictive stimuli evoke different patterns of phasic dopamine signaling throughout the striatum. Eur. J. Neurosci. 34, 1997-2006 (2011).
51. Bates, D., Mächler, M., Bolker, B. & Walker, S. Fitting linear mixed-effects models using lme4. Preprint at http://arxiv.org/abs/1406.5823 (2014).
52. Friston, K.J., Stephan, K.E., Lund, T.E., Morcom, A. & Kiebel, S. Mixed-effects and fMRI studies. Neuroimage 24, 244-252 (2005).