Dynamic mesolimbic dopamine signaling during action sequence learning and expectation violation

Scientific Reports

Volumn 6, Issue , 2016, Pages

  Collins, Anne L ^a   Greenfield, Venuz Y ^a   Bye, Jeffrey K ^a   Linker, Kay E ^a   Wang, Alice S ^a   Wassum, Kate M ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DOPAMINE;

ANIMAL; LEARNING; MALE; METABOLISM; NUCLEUS ACCUMBENS; PERCEPTIVE DISCRIMINATION; REWARD; SIGNAL TRANSDUCTION; SPRAGUE DAWLEY RAT; STEREOTYPY; TASK PERFORMANCE;

ANIMALS; DISCRIMINATION (PSYCHOLOGY); DOPAMINE; LEARNING; MALE; NUCLEUS ACCUMBENS; RATS, SPRAGUE-DAWLEY; REWARD; SIGNAL TRANSDUCTION; STEREOTYPED BEHAVIOR; TASK PERFORMANCE AND ANALYSIS;

EID: 84958078589     PISSN: None     EISSN: 20452322     Source Type: Journal    
DOI: 10.1038/srep20231     Document Type: Article

References (80)

1. Schultz, W. Getting formal with dopamine and reward. Neuron 36, 241-263 (2002).
2. Schultz, W. Updating dopamine reward signals. Curr Opin Neurobiol 23, 229-238 (2013).
3. Waelti, P., Dickinson, A., Schultz, W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 412, 43-48 (2001).
4. Keiflin, R., Janak, P. H. Dopamine Prediction Errors in Reward Learning and Addiction: From Theory to Neural Circuitry. Neuron 88, 247-263 (2015).
5. Day, J. J., Roitman, M. F., Wightman, R. M., Carelli, R. M. Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat Neurosci 10, 1020-1028 (2007).
6. Hart, A. S., Rutledge, R. B., Glimcher, P. W., Phillips, P. E. Phasic dopamine release in the rat nucleus accumbens symmetrically encodes a reward prediction error term. J Neurosci 34, 698-704 (2014).
7. Wassum, K. M., Ostlund, S. B., Maidment, N. T. Phasic Mesolimbic Dopamine Signaling Precedes and Predicts Performance of a Self-Initiated Action Sequence Task. Biol Psychiatry 71, 846-854 (2012).
8. Takikawa, Y., Kawagoe, R., Hikosaka, O. A possible role of midbrain dopamine neurons in short-and long-term adaptation of saccades to position-reward mapping. J Neurophysiol 92, 2520-2529 (2004).
9. Schultz, W., Apicella, P., Ljungberg, T. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci 13, 900-913 (1993).
10. Enomoto, K. et al. Dopamine neurons learn to encode the long-term value of multiple future rewards. Proc Natl Acad Sci USA 108 15462-15467 (2011).
11. Howe, M. W., Tierney, P. L., Sandberg, S. G., Phillips, P. E., Graybiel, A. M. Prolonged dopamine signalling in striatum signals proximity and value of distant rewards. Nature 500, 575-579 (2013).
12. Ostlund, S. B., Wassum, K. M., Murphy, N. P., Balleine, B. W., Maidment, N. T. Extracellular Dopamine Levels in Striatal Subregions Track Shifts in Motivation and Response Cost during Instrumental Conditioning. Journal of Neuroscience 31, 200-207 (2011).
13. McCullough, L. D., Cousins, M. S., Salamone, J. D. The role of nucleus accumbens dopamine in responding on a continuous reinforcement operant schedule: a neurochemical and behavioral study. Pharmacol Biochem Behav 46, 581-586 (1993).
14. Cousins, M. S., Trevitt, J., Atherton, A., Salamone, J. D. Different behavioral functions of dopamine in the nucleus accumbens and ventrolateral striatum: a microdialysis and behavioral investigation. Neuroscience 91, 925-934 (1999).
15. Sokolowski, J. D., Conlan, A. N., Salamone, J. D. A microdialysis study of nucleus accumbens core and shell dopamine during operant responding in the rat. Neuroscience 86, 1001-1009 (1998).
16. Salamone, J. D., Correa, M., Farrar, A., Mingote, S. M. Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology (Berl) 191, 461-482 (2007).
17. Nicola, S. M. The flexible approach hypothesis: unification of effort and cue-responding hypotheses for the role of nucleus accumbens dopamine in the activation of reward-seeking behavior. J Neurosci 30, 16585-16600 (2010).
18. Berridge, K. C. The debate over dopamine's role in reward: the case for incentive salience. Psychopharmacology (Berl) 191, 391-431 (2007).
19. Clark, J. J. et al. Chronic microsensors for longitudinal, subsecond dopamine detection in behaving animals. Nat Methods 7, 126-129 (2010).
20. Jankovic, J. Parkinson's disease: clinical features and diagnosis. J Neurol Neurosurg Psychiatry 79, 368-376 (2008).
21. Hyman, S. E. The neurobiology of addiction: implications for voluntary control of behavior. Am J Bioeth 7, 8-11 (2007).
22. Wassum, K. M., Ostlund, S. B., Maidment, N. T., Balleine, B. W. Distinct opioid circuits determine the palatability and the desirability of rewarding events. Proc Natl Acad Sci USA 106, 12512-12517 (2009).
23. Davis, J. D., Smith, G. P. Analysis of the microstructure of the rhythmic tongue movements of rats ingesting maltose and sucrose solutions. Behav Neurosci 106, 217-228 (1992).
24. Kaplan, J. M., Roitman, M. F., Grill, H. J. Ingestive taste reactivity as licking behavior. Neurosci Biobehav Rev 19, 89-98 (1995).
25. McDannald, M. A. et al. Orbitofrontal neurons acquire responses to 'valueless' Pavlovian cues during unblocking. Elife 3, e02653 (2014).
26. Roitman, M. F., Stuber, G. D., Phillips, P. E., Wightman, R. M., Carelli, R. M. Dopamine operates as a subsecond modulator of food seeking. J Neurosci 24, 1265-1271 (2004).
27. Owesson-White, C. A. et al. Neural encoding of cocaine-seeking behavior is coincident with phasic dopamine release in the accumbens core and shell. Eur J Neurosci 30, 1117-1127 (2009).
28. Saddoris, M. P. et al. Mesolimbic dopamine dynamically tracks, and is causally linked to, discrete aspects of value-based decision making. Biol Psychiatry 77, 903-911 (2015).
29. Oleson, E. B., Gentry, R. N., Chioma, V. C., Cheer, J. F. Subsecond dopamine release in the nucleus accumbens predicts conditioned punishment and its successful avoidance. J Neurosci 32, 14804-14808 (2012).
30. Wanat, M. J., Kuhnen, C. M., Phillips, P. E. Delays conferred by escalating costs modulate dopamine release to rewards but not their predictors. J Neurosci 30, 12020-12027 (2010).
31. Nasrallah, N. A. et al. Risk preference following adolescent alcohol use is associated with corrupted encoding of costs but not rewards by mesolimbic dopamine. Proc Natl Acad Sci USA 108, 5466-5471 (2011).
32. Hollon, N. G., Arnold, M. M., Gan, J. O., Walton, M. E., Phillips, P. E. Dopamine-associated cached values are not sufficient as the basis for action selection. Proc Natl Acad Sci USA 111, 18357-18362 (2014).
33. Stuber, G. D., Roitman, M. F., Phillips, P. E., Carelli, R. M., Wightman, R. M. Rapid dopamine signaling in the nucleus accumbens during contingent and noncontingent cocaine administration. Neuropsychopharmacology 30, 853-863 (2005).
34. Phillips, P. E., Stuber, G. D., Heien, M. L., Wightman, R. M., Carelli, R. M. Subsecond dopamine release promotes cocaine seeking. Nature 422, 614-618 (2003).
35. Willuhn, I., Burgeno, L. M., Everitt, B. J., Phillips, P. E. Hierarchical recruitment of phasic dopamine signaling in the striatum during the progression of cocaine use. Proc Natl Acad Sci USA 109, 20703-20708 (2012).
36. Satoh, T., Nakai, S., Sato, T., Kimura, M. Correlated coding of motivation and outcome of decision by dopamine neurons. J Neurosci 23, 9913-9923 (2003).
37. Romo, R., Schultz, W. Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. J Neurophysiol 63, 592-606 (1990).
38. Jin, X., Costa, R. M. Start/stop signals emerge in nigrostriatal circuits during sequence learning. Nature 466, 457-462 (2010).
39. du Hoffmann, J., Nicola, S. M. Dopamine invigorates reward seeking by promoting cue-evoked excitation in the nucleus accumbens. J Neurosci 34, 14349-14364 (2014).
40. McGinty, V. B., Lardeux, S., Taha, S. A., Kim, J. J., Nicola, S. M. Invigoration of reward seeking by cue and proximity encoding in the nucleus accumbens. Neuron 78, 910-922 (2013).
41. Clark, J. J., Collins, A. L., Sanford, C. A., Phillips, P. E. Dopamine encoding of Pavlovian incentive stimuli diminishes with extended training. J Neurosci 33, 3526-3532 (2013).
42. Choi, W. Y., Balsam, P. D., Horvitz, J. C. Extended habit training reduces dopamine mediation of appetitive response expression. J Neurosci 25, 6729-6733 (2005).
43. Saddoris, M. P., Cacciapaglia, F., Wightman, R. M., Carelli, R. M. Differential Dopamine Release Dynamics in the Nucleus Accumbens Core and Shell Reveal Complementary Signals for Error Prediction and Incentive Motivation. J Neurosci 35, 11572-11582 (2015).
44. de Araujo, I. E. et al. Food reward in the absence of taste receptor signaling. Neuron 57, 930-941 (2008).
45. Tobler, P. N., Fiorillo, C. D., Schultz, W. Adaptive coding of reward value by dopamine neurons. Science 307, 1642-1645 (2005).
46. Bayer, H. M., Glimcher, P. W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129-141 (2005).
47. Lak, A., Stauffer, W. R., Schultz, W. Dopamine prediction error responses integrate subjective value from different reward dimensions. Proc Natl Acad Sci USA 111, 2343-2348 (2014).
48. McDannald, M. A., Lucantonio, F., Burke, K. A., Niv, Y., Schoenbaum, G. Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. J Neurosci 31, 2700-2705 (2011).
49. Hamid, A. A. et al. Mesolimbic dopamine signals the value of work. Nat Neurosci (2015).
50. Hollerman, J. R., Schultz, W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 1, 304-309 (1998).
51. Schultz, W. Predictive reward signal of dopamine neurons. J Neurophysiol 80, 1-27 (1998).
52. Schultz, W., Dickinson, A. Neuronal coding of prediction errors. Annu Rev Neurosci 23, 473-500 (2000).
53. Gershman, S. J. Dopamine ramps are a consequence of reward prediction errors. Neural Computation 26, 467-471 (2014).
54. Morita, K., Kato, A. Striatal dopamine ramping may indicate flexible reinforcement learning with forgetting in the cortico-basal ganglia circuits. Front Neural Circuits 8, 36 (2014).
55. Schultz, W. Multiple dopamine functions at different time courses. Annu Rev Neurosci 30, 259-288 (2007).
56. Floresco, S. B., West, A. R., Ash, B., Moore, H., Grace, A. A. Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission. Nat Neurosci 6, 968-973 (2003).
57. Grace, A. A. Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: a hypothesis for the etiology of schizophrenia. Neuroscience 41, 1-24 (1991).
58. Schultz, W. Neuronal Reward and Decision Signals: From Theories to Data. Physiol Rev 95, 853-951 (2015).
59. Cachope, R., Cheer, J. F. Local control of striatal dopamine release. Front Behav Neurosci 8, 188 (2014).
60. Threlfell, S. et al. Striatal dopamine release is triggered by synchronized activity in cholinergic interneurons. Neuron 75, 58-64 (2012).
61. Cachope, R. et al. Selective activation of cholinergic interneurons enhances accumbal phasic dopamine release: setting the tone for reward processing. Cell Rep 2, 33-41 (2012).
62. Schultz, W., Apicella, P., Scarnati, E., Ljungberg, T. Neuronal activity in monkey ventral striatum related to the expectation of reward. J Neurosci 12, 4595-4610 (1992).
63. Wassum, K. M., Ostlund, S. B., Loewinger, G. C., Maidment, N. T. Phasic Mesolimbic Dopamine Release Tracks Reward Seeking During Expression of Pavlovian-to-Instrumental Transfer. Biol Psychiatry 73, 747-755 (2013).
64. Ostlund, S. B., LeBlanc, K. H., Kosheleff, A. R., Wassum, K. M., Maidment, N. T. Phasic Mesolimbic Dopamine Signaling Encodes the Facilitation of Incentive Motivation Produced by Repeated Cocaine Exposure. Neuropsychopharmacology, 39, 2441-2449 (2014).
65. Wassum, K. M., Ostlund, S. B., Balleine, B. W., Maidment, N. T. Differential dependence of Pavlovian incentive motivation and instrumental incentive learning processes on dopamine signaling. Learn Mem 18, 475-483 (2011).
66. Flagel, S. B. et al. A selective role for dopamine in stimulus-reward learning. Nature 469, 53-57 (2011).
67. Rutledge, R. B., Skandali, N., Dayan, P., Dolan, R. J. Dopaminergic Modulation of Decision Making and Subjective Well-Being. J Neurosci 35, 9811-9822 (2015).
68. Dayan, P., Niv, Y., Seymour, B., Daw, N. D. The misbehavior of value and the discipline of the will. Neural Netw 19, 1153-1160 (2006).
69. Dickinson, A., Smith, J., Mirenowicz, J. Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists. Behav Neurosci 114, 468-483 (2000).
70. Lex, A., Hauber, W. Dopamine D1 and D2 receptors in the nucleus accumbens core and shell mediate Pavlovian-instrumental transfer. Learn Mem 15, 483-491 (2008).
71. Saunders, B. T., Robinson, T. E. The role of dopamine in the accumbens core in the expression of Pavlovian-conditioned responses. Eur J Neurosci 36, 2521-2532 (2012).
72. Balleine, B. Asymmetrical interactions between thirst and hunger in Pavlovian-instrumental transfer. Q J Exp Psychol B 47, 211-231 (1994).
73. Graybiel, A. M. The basal ganglia and chunking of action repertoires. Neurobiol Learn Mem 70, 119-136 (1998).
74. Ostlund, S. B., Winterbauer, N. E., Balleine, B. W. Evidence of action sequence chunking in goal-directed instrumental conditioning and its dependence on the dorsomedial prefrontal cortex. J Neurosci 29, 8280-8287 (2009).
75. Jin, X., Tecuapetla, F., Costa, R. M. Basal ganglia subcircuits distinctively encode the parsing and concatenation of action sequences. Nat Neurosci 17, 423-430 (2014).
76. Sclafani, A. Starch and sugar tastes in rodents: an update. Brain Res Bull 27, 383-386 (1991).
77. Heien, M. L. et al. Real-time measurement of dopamine fluctuations after cocaine in the brain of behaving rats. Proc Natl Acad Sci USA 102, 10023-10028 (2005).
78. Keithley, R. B., Wightman, R. M. Assessing principal component regression prediction of neurochemicals detected with fast-scan cyclic voltammetry. ACS Chem Neurosci 2, 514-525 (2011).
79. Paxinos, G., Watson, C. The rat brain in stereotaxic coordinates. 4th edn. (Academic Press, 1998).
80. Cheer, Joseph F., Kate M. Wassum & R. Mark Wightman. Cannabinoid modulation of electrically evoked pH and oxygen transients in the nucleus accumbens of awake rats. Journal of neurochemistry 97.4, 1145-1154 (2006).