Influence of reward delays on responses of dopamine neurons

Journal of Neuroscience

Volumn 28, Issue 31, 2008, Pages 7837-7846

  Kobayashi, Shunsuke ^a   Schultz, Wolfram ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

Dopamine; Impulsivity; Neuroeconomics; Preference reversal; Single unit recording; Temporal discounting

Indexed keywords

DOPAMINE;

ARTICLE; BRAIN FUNCTION; CONDITIONING; DECISION MAKING; DOPAMINERGIC NERVE CELL; LEARNING; MALE; NERVE STIMULATION; NONHUMAN; PRIORITY JOURNAL; RESPONSE TIME; REWARD; STIMULUS RESPONSE; TASK PERFORMANCE; TIME PERCEPTION; ANIMAL; COMPARATIVE STUDY; MACACA; METHODOLOGY; NERVE CELL; PHOTOSTIMULATION; PHYSIOLOGY; REACTION TIME; TIME;

ANIMALS; CHOICE BEHAVIOR; CONDITIONING (PSYCHOLOGY); DOPAMINE; MACACA MULATTA; MALE; NEURONS; PHOTIC STIMULATION; REACTION TIME; REWARD; TIME FACTORS;

EID: 50349093022     PISSN: 02706474     EISSN: 02706474     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.1600-08.2008     Document Type: Article

References (41)

1. Ainslie GW (1974) Impulse control in pigeons. J Exp Anal Behav 21:485-489.
2. Ainslie GW, Herrnstein RJ (1981) Preference reversal and delayed reinforcement. Anim Learn Behav 9:476-482.
3. Cardinal RN, Pennicott DR, Sugathapala CL, Robbins TW, Everitt BJ (2001) Impulsive choice induced in rats by lesions of the nucleus accumbens core. Science 292:2499-2501.
4. Church RM, Gibbon J (1982) Temporal generalization. J Exp Psychol Anim Behav Process 8:165-186.
5. Corrado GS, Sugrue LP, Seung HS, Newsome WT (2005) Linear-nonlinear- Poisson models of primate choice dynamics. J Exp Anal Behav 84:581-617.
6. Critchfield TS, Kollins SH (2001) Temporal discounting: basic research and the analysis of socially important behavior. J Appl Behav Anal 34:101-122.
7. Dalley JW, Fryer TD, Brichard L, Robinson ESJ, Theobald DEH, Lääne K, Peña Y, Murphy ER, Shah Y, Probst K, Abakumova I, Aigbirhio FI, Richards HK, Hong Y, Baron JC, Everitt BJ, Robbins TW (2007) Nucleus accumbens D2/3 receptors predict trait impulsivity and cocaine reinforcement. Science 315:1267-1270.
8. Daw ND, Courville AC, Touretzky DS (2006) Representation and timing in theories of the dopamine system. Neural Comput 18:1637-1677.
9. Delamater AR, Holland PC (2008) The influence of CS-US interval on several different indices of learning in appetitive conditioning. J Exp Psychol Anim Behav Process 34:202-222.
10. Fiorillo CD, Tobler PN, Schultz W (2003) Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299:1898-1902.
11. Green L, Estle SJ (2003) Preference reversals with food and water reinforcers in rats. J Exp Anal Behav 79:233-242.
12. Green L, Myerson J (2004) A discounting framework for choice with delayed and probabilistic rewards. Psychol Bull 130:769-792.
13. Hayden BY, Platt ML (2007) Temporal discounting predicts risk sensitivity in rhesus macaques. Curr Biol 17:49-53.
14. Holland PC (1980) CS-US interval as a determinant of the form of pavlovian appetitive conditioned responses. J Exp Psychol Anim Behav Process 6:155-174.
15. Houk JC, Adams JL, Barto AG (1995) A model of how the basal ganglia generate and use neural signals that predict reinforcement. In: Models of information processing in the basal ganglia (Houk JC, Davis JL, Beiser DG, eds), pp. 249-270. Cambridge, MA: MIT.
16. Kable JW, Glimcher PW (2007) The neural correlates of subjective value during intertemporal choice. Nat Neurosci 10:1625-1633.
17. Kagel JH, Green L, Caraco T (1986) When foragers discount the future: constraint or adaptation? Anim Behav 34:271-283.
18. Kakade S, Dayan P (2002) Dopamine: generalization and bonuses. Neural Netw 15:549-559.
19. Kheramin S, Body S, Ho MY, Velázquez-Martinez DN, Bradshaw CM, Szabadi E, Deakin JFW, Anderson IM (2004) Effects of orbital prefrontal cortex dopamine depletion on inter-temporal choice: a quantitative analysis. Psychopharmacology (Berl) 175:206-214.
20. Kobayashi S, Kawagoe R, Takikawa Y, Koizumi M, Sakagami M, Hikosaka O (2007) Functional differences between macaque prefrontal cortex and caudate nucleus during eye movements with and without reward. Exp Brain Res 176:341-355.
21. Ljungberg T, Apicella P, Schultz W (1991) Responses of monkey midbrain dopamine neurons during delayed alternation performance. Brain Res 586:337-341.
22. Ljungberg T, Apicella P, Schultz W (1992) Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol 67:145-163.
23. Mazur JE (1987) An adjusting procedure for studying delayed reinforcement. In: Quantitative analyses of behavior, Vol 5 (Commons ML, Mazur JE, Nevin JA, Rachlin H, eds), Hillsdale, NJ: Erlbaum.
24. Mazur JE (2000) Tradeoffs among delay, rate, and amount of reinforcement. Behav Processes 49:1-10.
25. McClure SM, Laibson DI, Loewenstein G, Cohen JD (2004) Separate neural systems value immediate and delayed monetary rewards. Science 306:503-507.
26. Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16:1936-1947.
27. Myerson J, Green L (1995) Discounting of delayed rewards: models of individual choice. J Exp Anal Behav 64:263-276.
28. Richards JB, Mitchell SH, de Wit H, Seiden LS (1997) Determination of discount functions in rats with an adjusting-amount procedure. J Exp Anal Behav 67:353-366.
29. Rodriguez ML, Logue AW (1988) Adjusting delay to reinforcement: comparing choice in pigeons and humans. J Exp Psychol Anim Behav Process 14:105-117.
30. Roesch MR, Calu DJ, Schoenbaum G (2007) Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat Neurosci 10:1615-1624.
31. Samuelson PA (1937) Some aspects of the pure theory of capital. Q J Economics 51:469-496.
32. Schultz W, Romo R (1987) Responses of nigrostriatal dopamine neurons to high-intensity somatosensory stimulation in the anesthetized monkey. J Neurophysiol 57:201-217.
33. Schultz W, Romo R (1990) Dopamine neurons of the monkey midbrain: contingencies of responses to stimuli eliciting immediate behavioral reactions. J Neurophysiol 63:607-624.
34. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593-1599.
35. Suri RE, Schultz W (1999) Aneural network with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience 91:871-890.
36. Sutton RS (1988) Learning to predict by the method of temporal differences. Machine Learning 3:9-44.
37. Tanaka SC, Doya K, Okada G, Ueda K, Okamoto Y, Yamawaki S (2004) Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat Neurosci 7:887-893.
38. Tobler PN, Dickinson A, Schultz W (2003) Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. J Neurosci 23:10402-10410.
39. Tobler PN, Fiorillo CD, Schultz W (2005) Adaptive coding of reward value by dopamine neurons. Science 307:1642-1645.
40. Waelti P, Dickinson A, Schultz W (2001) Dopamine responses comply with basic assumptions of formal learning theory. Nature 412:43-48.
41. Winstanley CA, Theobald DEH, Cardinal RN, Robbins TW (2004) Contrasting roles of basolateral amygdala and orbitofrontal cortex in impulsive choice. J Neurosci 24:4718-4722.