Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards

Nature Neuroscience

Volumn 10, Issue 12, 2007, Pages 1615-1624

  Roesch, Matthew R ^a   Calu, Donna J ^a   Schoenbaum, Geoffrey ^a  

^a UNIVERSITY OF MARYLAND SCHOOL OF MEDICINE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APOMORPHINE;

ANIMAL EXPERIMENT; ANIMAL MODEL; ARTICLE; ASSOCIATION; COMPUTER PROGRAM; CONTROLLED STUDY; DECISION MAKING; DIAGNOSTIC ERROR; DOPAMINE RELEASE; DOPAMINERGIC NERVE CELL; LEARNING; MALE; NONHUMAN; OUTCOMES RESEARCH; PREDICTION; PRIORITY JOURNAL; RAT; REINFORCEMENT; REWARD; VENTRAL TEGMENTUM; WAVEFORM;

ACTION POTENTIALS; ANALYSIS OF VARIANCE; ANIMALS; APOMORPHINE; BEHAVIOR, ANIMAL; BRAIN MAPPING; CHOICE BEHAVIOR; CONDITIONING, OPERANT; CUES; DOPAMINE; DOPAMINE AGONISTS; MALE; NEURONS; ODORS; PREDICTIVE VALUE OF TESTS; RATS; RATS, LONG-EVANS; REWARD; TIME PERCEPTION; VENTRAL TEGMENTAL AREA;

EID: 36448968271     PISSN: 10976256     EISSN: None     Source Type: Journal    
DOI: 10.1038/nn2013     Document Type: Article

References (47)

1. Wise, R.A. Dopamine, learning and motivation. Nat. Rev. Neurosci. 5, 483-494 (2004).
2. Schultz, W. Getting formal with dopamine and reward. Neuron 36, 241-263 (2002).
3. Dayan, P. & Balleine, B.W. Reward, motivation and reinforcement learning. Neuron 36, 285-298 (2002).
4. Day, J.J., Roitman, M.F., Wightman, R.M. & Carelli, R.M. Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat. Neurosci. 10, 1020-1028 (2007).
5. Mirenowicz, J. & Schultz, W. Importance of unpredictability for reward responses in primate dopamine neurons. J. Neurophysiol. 72, 1024-1027 (1994).
6. Fiorillo, C.D., Tobler, P.N. & Schultz, W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299, 1898-1902 (2003).
7. Tobler, P.N., Dickinson, A. & Schultz, W. Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. J. Neurosci. 23, 10402-10410 (2003).
8. Hollerman, J.R. & Schultz, W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci. 1, 304-309 (1998).
9. Waelti, P., Dickinson, A. & Schultz, W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 412, 43-48 (2001).
10. Montague, P.R., Dayan, P. & Sejnowski, T.J. A framework for mesencephalic dopamine systems based on predictive hebbian learning. J. Neurosci. 16, 1936-1947 (1996).
11. Bayer, H.M. & Glimcher, P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129-141 (2005).
12. Nakahara, H., Itoh, H., Kawagoe, R., Takikawa, Y. & Hikosaka, O. Dopamine neurons can represent context-dependent prediction error. Neuron 41, 269-280 (2004).
13. Pan, W.X., Schmidt, R., Wickens, J.R. & Hyland, B.I. Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J. Neurosci. 25, 6235-6242 (2005).
14. Morris, G., Nevet, A., Arkadir, D., Vaadia, E. & Bergman, H. Midbrain dopamine neurons encode decisions for future action. Nat. Neurosci. 9, 1057-1063 (2006).
15. Kawagoe, R., Takikawa, Y. & Hikosaka, O. Reward-predicting activity of dopamine and caudate neurons - a possible mechanism of motivational control of saccadic eye movement. J. Neurophysiol. 91, 1013-1024 (2004).
16. Cardinal, R.N., Pennicott, D.R., Sugathapala, C.L., Robbins, T.W. & Everitt, B.J. Impulsive choice induced in rats by lesions of the nucleus accumbens core. Science 292, 2499-2501 (2001).
17. Evenden, J.L. & Ryan, C.N. The pharmacology of impulsive behaviour in rats: the effects of drugs on response choice with varying delays of reinforcement. Psychopharmacology (Berl.) 128, 161-170 (1996).
18. Herrnstein, R.J. Relative and absolute strength of response as a function of frequency of reinforcement. J. Exp. Anal. Behav. 4, 267-272 (1961).
19. Ho, M.Y., Mobini, S., Chiang, T.J., Bradshaw, C.M. & Szabadi, E. Theory and method in the quantitative analysis of "impulsive choice" behaviour: implications for psychopharmacology. Psychopharmacology (Berl.) 146, 362-372 (1999).
20. Mobini, S. et al. Effects of lesions of the orbitofrontal cortex on sensitivity to delayed and probabilistic reinforcement. Psychopharmacology (Berl.) 160, 290-298 (2002).
21. Kahneman, D. & Tverskey, A. Choices, values and frames. Am. Psychol. 39, 341-350 (1984).
22. Kalenscher, T. et al. Single units in the pigeon brain integrate reward amount and time-to-reward in an impulsive choice task. Curr. Biol. 15, 594-602 (2005).
23. Lowenstein, G.E.J. Choice Over Time (Russel Sage Foundation, New York, 1992).
24. Thaler, R. Some empirical evidence on dynamic inconsistency. Econ. Lett. 8, 201-207 (1981).
25. Winstanley, C.A., Theobald, D.E., Cardinal, R.N. & Robbins, T.W. Contrasting roles of basolateral amygdala and orbitofrontal cortex in impulsive choice. J. Neurosci. 24, 4718-4722 (2004).
26. Cardinal, R.N., Winstanley, C.A., Robbins, T.W. & Everitt, B.J. Limbic corticostriatal systems and delayed reinforcement. Ann. NY Acad. Sci. 1021, 33-50 (2004).
27. Kheramin, S. et al. Effects of orbital prefrontal cortex dopamine depletion on intertemporal choice: a quantitative analysis. Psychopharmacology (Berl.) 175, 206-214 (2004).
28. Wade, T.R., deWit, H. & Richards, J.B. Effects of dopaminergic drugs on delayed reward as a measure of impulsive behavior in rats. Psychopharmacology (Berl.) 150, 90-101 (2000).
29. Cardinal, R.N., Robbins, T.W. & Everitt, B.J. The effects of d-amphetamine, chlordiazepoxide, alpha-flupenthixol and behavioural manipulations on choice of signalled and unsignalled delayed reinforcement in rats. Psychopharmacology (Berl.) 152, 362-375 (2000).
30. Roesch, M.R., Takahashi, Y., Gugsa, N., Bissonette, G.B. & Schoenbaum, G. Previous cocaine exposure makes rats hypersensitive to both delay and reward magnitude. J. Neurosci. 27, 245-250 (2007).
31. Roesch, M.R., Taylor, A.R. & Schoenbaum, G. Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation. Neuron 51, 509-520 (2006).
32. Tobler, P.N., Fiorillo, C.D. & Schultz, W. Adaptive coding of reward value by dopamine neurons. Science 307, 1642-1645 (2005).
33. Kiyatkin, E.A. & Rebec, G.V. Heterogeneity of ventral tegmental area neurons: single-unit recording and iontophoresis in awake, unrestrained rats. Neuroscience 85, 1285-1309 (1998).
34. Bunney, B.S., Aghajanian, G.K. & Roth, R.H. Comparison of effects of L-dopa, amphetamine and apomorphine on firing rate of rat dopaminergic neurones. Nat. New Biol. 245, 123-125 (1973).
35. Skirboll, L.R., Grace, A.A. & Bunney, B.S. Dopamine auto- and postsynaptic receptors: electrophysiological evidence for differential sensitivity to dopamine agonists. Science 206, 80-82 (1979).
36. Niv, Y., Daw, N.D. & Dayan, P. Choice values. Nat. Neurosci. 9, 987-988 (2006).
37. Haber, S.N., Fudge, J.L. & McFarland, N.R. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 20, 2369-2382 (2000).
38. Joel, D. & Weiner, I. The connections of the dopaminergic system with the striatum in rats and primates: an analysis with respect to the functional and compartmental organization of the striatum. Neuroscience 96, 451-474 (2000).
39. Yin, H.H., Knowlton, B.J. & Balleine, B.W. Lesions of dorsolateral striatum preserve outcome expectancy, but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 19, 181-189 (2004).
40. O'Doherty, J. et al. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452-454 (2004).
41. Knowlton, B.J., Mangels, J.A. & Squire, L. A neostriatal habit learning system in humans. Science 273, 1399-1402 (1996).
42. Hatfield, T., Han, J.S., Conley, M., Gallagher, M. & Holland, P. Neurotoxic lesions of basolateral, but not central, amygdala interfere with Pavlovian second-order conditioning and reinforcer devaluation effects. J. Neurosci. 16, 5256-5265 (1996).
43. Gallagher, M., McMahan, R.W. & Schoenbaum, G. Orbitofrontal cortex and representation of incentive value in associative learning. J. Neurosci. 19, 6610-6614 (1999).
44. Baxter, M.G., Parker, A., Lindner, C.C.C., Izquierdo, A.D. & Murray, E.A. Control of response selection by reinforcer value requires interaction of amygdala and orbitofrontal cortex. J. Neurosci. 20, 4311-4319 (2000).
45. Gottfried, J.A., O'Doherty, J. & Dolan, R.J. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 301, 1104-1107 (2003).
46. Parkinson, J.A., Cardinal, R.N. & Everitt, B.J. Limbic cortical-ventral striatal systems underlying appetitive conditioning. Prog. Brain Res. 126, 263-285 (2000).
47. Lu, L. et al. Central amygdala ERK signaling pathway is critical to incubation of cocaine craving. Nat. Neurosci. 8, 212-219 (2005).