Impact of size and delay on neural activity in the rat limbic corticostriatal system

Frontiers in Neuroscience

Volumn , Issue DEC, 2011, Pages

  Roesch, Matthew R ^a   Bryden, Daniel W ^a  

^a UNIVERSITY OF MARYLAND   (United States)

Author keywords

Amygdala; Discounting; Dopamine; Orbitofrontal; Striatum; Substantia nigra; Value

Indexed keywords

AMYGDALOID NUCLEUS; ANIMAL BEHAVIOR; CORPUS STRIATUM; DECISION MAKING; DOPAMINERGIC NERVE CELL; LEARNING; LIMBIC CORTEX; LIMBIC CORTICOSTRIATAL SYSTEM; MENTAL TASK; MESENCEPHALON; MOTIVATION; NERVE CELL EXCITABILITY; NONHUMAN; ORBITAL CORTEX; PERCEPTIVE DISCRIMINATION; RESPONSE TIME; REVIEW; REWARD; REWARD DELAY; REWARD DELIVERY; REWARD SIZE; SUBSTANTIA NIGRA PARS RETICULATA; TASK PERFORMANCE; VENTRAL TEGMENTUM;

EID: 84862159396     PISSN: 16624548     EISSN: 1662453X     Source Type: Journal    
DOI: 10.3389/fnins.2011.00130     Document Type: Review

References (142)

1. Ainslie, G. W. (1974). Impulse control in pigeons. J. Exp. Anal. Behav. 21, 485-489.
2. Ambroggi, F., Ishikawa, A., Fields, H. L., and Nicola, S. M. (2008). Baso-lateral amygdala neurons facilitate reward-seeking behavior by exciting nucleus accumbens neurons. Neuron 59,648-661.
3. Ballard, K., and Knutson, B. (2009). Dissociable neural representations of future reward magnitude and delay during temporal discounting. Neu-roimage 45,143-150.
4. Bayer, H. M., and Glimcher, P. W. (2005). Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129-141.
5. Bechara, A., Damasio, H., Dama-sio, A. R., and Lee, G. P. (1999). Different contributions of the human amygdala and ven-tromedial prefrontal cortex to decision-making. J. Neurosci. 19, 5473-5481.
6. Bechara, A., Dolan, S., and Hindes, A. (2002). Decision-making and addiction (part II): myopia for the future or hypersensitivity to reward? Neu-ropsychologia 40,1690-1705.
7. Belova, M. A., Paton, J. J., Morrison, S. E., and Salzman, C. D. (2007). Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala. Neuron 55, 970-984.
8. Bezzina, G., Cheung, T. H., Asgari, K., Hampson, C. L., Body, S., Bradshaw, C. M., Szabadi, E., Deakin, J. F., and Anderson, I. M. (2007). Effects of quinolinic acid-induced lesions of the nucleus accumbens core on inter-temporal choice: a quantitative analysis. Psychopharmacology (Berl.) 195,71-84.
9. Bizot, J., Le Bihan, C., Puech, A. J., Hamon, M.,and Thiebot, M. (1999). Serotonin and tolerance to delay of reward in rats. Psychopharmacology (Berl.) 146,400-412.
10. Bizot, J. C., Thiebot, M. H., Le Bihan, C., Soubrie, P., and Simon, P. (1988). Effects of imipramine-like drugs and serotonin uptake blockers on delay of reward in rats. Possible implication in the behavioral mechanism of action of antidepressants. J. Pharmacol. Exp. Ther. 246, 1144-1151.
11. Boettiger, C. A., Mitchell, J. M., Tavares, V. C., Robertson, M., Joslyn, G., D'Esposito, M., and Fields, H. L. (2007). Immediate reward bias in humans: fronto-parietal networks and a role for the catechol-O-methyltransferase 158(Val/Val) genotype. J. Neurosci. 27,14383-14391.
12. Bowman, E. M., Aigner, T. G., and Richmond, B. J. (1996). Neural signals in the monkey ventral striatum related to motivation for juice and cocaine rewards. J. Neurophysiol. 75, 1061-1073.
13. Breiter, H. C., Aharon, I., Kahne-man, D., Dale, A., and Shizgal, P. (2001). Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 30, 619-639.
14. Bromberg-Martin, E. S., Matsumoto, M., and Hikosaka, O. (2010). Dopamine in motivational control: rewarding, aversive, and alerting. Neuron 68,815-834.
15. Bryden, D. W., Johnson, E. E., Diao, X., and Roesch, M. R. (2011). Impact of expected value on neural activity in rat substantia nigra pars reticulata. Eur. J. Neurosci. 33, 2308-2317.
16. Bucci, D. J., and Macleod, J. E. (2007). Changes in neural activity associated with a surprising change in the predictive validity of a conditioned stimulus. Eur. J. Neurosci. 26, 2669-2676.
17. Cai, X., Kim, S., and Lee, D. (2011). Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice. Neuron 69, 170-182.
18. Calu, D. J., Roesch, M. R., Haney, R. Z., Holland, P. C., and Schoenbaum, G. (2010). Neural correlates of variations in event processing during learning in central nucleus of amygdala. Neuron 68, 991-1001.
19. Cardinal, R. N. (2006). Neural systems implicated in delayed and probabilistic reinforcement. Neural. Netw. 19,1277-1301.
20. Cardinal, R. N., Pennicott, D. R., Sugath-apala, C. L., Robbins, T. W., and Everitt, B. J. (2001). Impulsive choice induced in rats by lesions of the nucleus accumbens core. Science 292,2499-2501.
21. Cardinal, R. N., Robbins, T. W., and Everitt, B. J. (2000). The effects of d-amphetamine, chlordiazepox-ide, alpha-flupentixol and behavioural manipulations on choice of signalled and unsignalled delayed reinforcement in rats. Psychophar-macology (Berl.) 152, 362-375.
22. Cardinal, R. N., Winstanley, C. A., Robbins, T. W., and Everitt, B. J. (2004). Limbic corticostriatal systems and delayed reinforcement. Ann. N. Y. Acad. Sci. 1021,33-50.
23. Carelli, R. M. (2002). Nucleus accumbens cell firing during goal-directed behaviors for cocaine vs 'natural' reinforcement. Physiol. Behav. 76, 379-387.
24. Church, R. M., and Gibbon, J. (1982). Temporal generalization. J. Exp. Psychol. Anim. Behav. Process. 8, 165-186.
25. Churchwell, J. C., Morris, A. M., Heurtelou, N. M., and Kesner, R. P. (2009). Interactions between the prefrontal cortex and amygdala during delay discounting and reversal. Behav. Neurosci. 123, 1185-1196.
26. Coffey, S. F., Gudleski, G. D., Saladin, M. E., and Brady, K. T. (2003). Impulsivity and rapid discounting of delayed hypothetical rewards in cocaine-dependent individuals. Exp. Clin. Psychopharmacol. 11, 18-25.
27. Cousens, G. A., and Otto, T. (2003). Neural substrates of olfactory discrimination learning with auditory secondary reinforcement. I. Contributions of the basolateral amygdaloid complex and orbitofrontal cortex. Int. Physiol. Behav. Sci. 38, 272-294.
28. Cousins, M. S., Atherton, A., Turner, L., and Salamone, J. D. (1996). Nucleus accumbens dopamine depletions alter relative response allocation in a T-maze cost/benefit task. Behav. Brain Res. 74, 189-197.
29. Cromwell, H.C., and Schultz, W. (2003). Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. J. Neurophysiol. 89, 2823-2838.
30. Dalley, J. W., Mar, A. C., Economidou, D., and Robbins, T. W. (2008). Neu-robehavioral mechanisms of impulsivity: fronto-striatal systems and functional neurochemistry. Pharmacol. Biochem. Behav. 90, 250-260.
31. Day, J. J., Jones, J. L., and Carelli, R. M. (2011). Nucleus accumbens neurons encode predicted and ongoing reward costs in rats. Eur. J. Neurosci. 33,308-321.
32. Delamater, A. R., and Oakeshott, S. (2007). Learning about multiple attributes of reward in Pavlovian conditioning. Ann. N Y Acad. Sci. 1104, 1-20.
33. Denk, F., Walton, M. E., Jennings, K. A., Sharp, T., Rushworth, M. E, and Bannerman, D. M. (2005). Differential involvement of serotonin and dopamine systems in cost-benefit decisions about delay or effort. Psychopharmacology (Berl.) 179, 587-596.
34. Ernst, M., Zametkin, A. J., Matochik, J. A., Jons, P. H., and Cohen, R. M. (1998). DOPA decarboxylase activity in attention deficit hyperac-tivity disorder adults. A [fluorine-18]fluorodopa positron emission tomographic study. J. Neurosci. 18, 5901-5907.
35. Evenden, J. L., and Ryan, C. N. (1996). The pharmacology of impulsive behaviour in rats: the effects of drugs on response choice with varying delays of reinforcement. Psychopharmacology (Berl.) 128, 161-170.
36. Evenden, J. L., and Ryan, C. N. (1999). The pharmacology of impulsive behaviour in rats VI: the effects of ethanol and selective seroton-ergic drugs on response choice with varying delays of reinforcement. Psychopharmacology (Berl.) 146,413-421.
37. Figner, B., Knoch, D., Johnson, E. J., Krosch, A. R., Lisanby, S. H., Fehr, E., and Weber, E. U. (2010). Lateral prefrontal cortex and self-control in intertemporal choice. Nat. Neurosci. 13,538-539.
38. Fiorillo, C. D., Newsome, W T., and Schultz, W (2008). The temporal precision of reward prediction in dopamine neurons. Nat. Neurosci. 11,966-973.
39. Fiorillo, C. D., Tobler, P. N., and Schultz, W (2003). Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299,1856-1902.
40. Floresco, S. B., St Onge, J. R., Ghods-Sharifi, S., and Winstanley, C. A. (2008). Cortico-limbic-striatal circuits subserving different forms of cost-benefit decision making. Cogn. Affect. Behav. Neurosci. 8, 375-389.
41. Frank, M. J. (2005). Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. J. Cogn. Neurosci. 17, 51-72.
42. Gallagher, M., Graham, P. W., and Holland, P. C. (1990). The amygdala central nucleus and appetitive Pavlovian conditioning: lesions impair one class of conditioned behavior. J. Neurosci. 10, 1906-1911.
43. Ghods-Sharifi, S., St Onge, J. R., and Floresco, S. B. (2009). Fundamental contribution by the basolateral amygdala to different forms of decision making. J. Neurosci. 29, 5251-5259.
44. Hariri, A. R., Brown, S. M., Williamson, D. E., Flory, J. D., de Wit, H., and Manuck, S. B. (2006). Preference for immediate over delayed rewards is associated with magnitude of ventral striatal activity. J. Neurosci. 26, 13213-13217.
45. Harrison, A. A., Everitt, B. J., and Robbins, T. W. (1997). Central 5-HT depletion enhances impulsive responding without affecting the accuracy of attentional performance: interactions with dopamin-ergic mechanisms. Psychopharmacology (Berl.) 133, 329-342.
46. Hassani, O. K., Cromwell, H. C., and Schultz, W (2001). Influence of expectation of different rewards on behavior-related neuronal activity in the striatum. J. Neurophysiol. 85, 2477-2489.
47. Hatfield, T., Han, J. S., Conley, M., Gallagher, M., and Holland, P. (1996). Neurotoxic lesions of basolateral, but not central, amygdala interfere with Pavlovian second-order conditioning and reinforcer devaluation effects. J. Neurosci. 16, 5256-5265.
48. Heerey, E. A., Robinson, B. M., McMa-hon, R. P., and Gold, J. M. (2007). Delay discounting in schizophrenia. Cogn. Neuropsychiatry 12,213-221.
49. Herrnstein, R. J. (1961). Relative and absolute strength of response as a function of frequency of reinforcement. J. Exp. Anal. Behav. 4,267-272.
50. Ho, M. Y., Mobini, S., Chiang, T. J., Bradshaw, C. M., and Szabadi, E. (1999). Theory and method in the quantitative analysis of "impulsive choice" behaviour: implications for psychopharmacology. Psychopharmacology (Berl.) 146, 362-372.
51. Holland, P. C., and Gallagher, M. (1993a). Amygdala central nucleus lesions disrupt increments, but not decrements, in conditioned stimulus processing. Behav. Neurosci. 107, 246-253.
52. Holland, P. C., and Gallagher, M. (1993b). Effects of amygdala central nucleus lesions on blocking and unblocking. Behav. Neurosci. 107, 235-245.
53. Holland, P. C., and Gallagher, M. (1993c). Effects of amygdala central nucleus lesions on blocking and unblocking. Behav. Neurosci. 107, 235-245.
54. Holland, P. C., and Gallagher, M. (1999). Amygdala circuitry in attentional and representational processes. Trends Cogn. Sci. (Regul. Ed.) 3, 65-73.
55. Holland, P. C., and Gallagher, M. (2006). Different roles for amygdala central nucleus and substantia innominata in the surprise-induced enhancement of learning. J. Neurosci. 26, 3791-3797.
56. Holland, P. C., and Kenmuir, C. (2005). Variations in unconditioned stimulus processing in unblocking. J. Exp. Psychol. Anim. Behav. Process. 31, 155-171.
57. Hollerman, J. R., and Schultz, W. (1998a). Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci. 1,304-309.
58. Hollerman, J. R., and Schultz, W. (1998b). Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci. 1, 304-309.
59. Hong, S., and Hikosaka, O. (2011). Dopamine-mediated learning and switching in cortico-striatal circuit explain behavioral changes in reinforcement learning. Front. Behav. Neurosci. 5:15. doi: 10.3389/fnbeh.2011.00015
60. Ito, M., and Doya, K. (2009). Validation of decision-making models and analysis of decision variables in the rat basal ganglia. J. Neurosci. 29, 9861-9874.
61. Janak, P. H., Chen, M. T., and Caulder, T. (2004). Dynamics of neural coding in the accumbens during extinction and reinstatement of rewarded behavior. Behav. Brain Res. 154, 125-135.
62. Jentsch, J. D., and Taylor, J. R. (1999). Impulsivity resulting from frontos-triatal dysfunction in drug abuse: implications for the control of behavior by reward-related stimuli. Psychopharmacology (Berl.) 146, 373-390.
63. Joel, D., Niv, Y., and Ruppin, E. (2002). Actor-critic models of the basal ganglia: new anatomical and computational perspectives. Neural. Netw. 15, 535-547.
64. Jones, B., and Mishkin, M. (1972). Limbic lesions and the problem of stimulus-reinforcement associations. Exp. Neurol. 36, 362-377.
65. Kable, J. W., and Glimcher, P. W. (2007). The neural correlates of subjective value during intertem-poral choice. Nat. Neurosci. 10, 1625-1633.
66. Kahneman, D., and Tverskey, A. (1984). Choices, values, and frames. Am. Psychol. 39,341-350.
67. Kalenscher, T., and Pennartz, C. M. (2008). Is a bird in the hand worth two in the future? The neuroeconomics of intertemporal decision-making. Prog. Neurobiol. 84,284-315.
68. Kalenscher, T., Windmann, S., Diekamp, B., Rose, J., Gunturkun, O., and Colombo, M. (2005). Single units in the pigeon brain integrate reward amount and time-to-reward in an impulsive choice task. Curr. Biol. 15, 594-602.
69. Kennerley, S. W., Dahmubed, A. F., Lara, A. H., and Wallis, J. D. (2009). Neurons in the frontal lobe encode the value of multiple decision variables. J. Cogn. Neurosci. 21, 1162-1178.
70. Kennerley, S. W.,and Wallis, J.D. (2009). Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables. Eur. J. Neurosci. 29, 2061-2073.
71. Kesner, R. P., and Williams, J. M. (1995). Memory for magnitude of reinforcement: dissociation between amygdala and hippocampus. Neurobiol. Learn. Mem. 64,237-244.
72. Khamassi, M., Mulder, A. B., Tabuchi, E., Douchamps, V., and Wiener, S. I. (2008). Anticipatory reward signals in ventral striatal neurons of behaving rats. Eur. J. Neurosci. 28, 1849-1866.
73. Kheramin, S., Body, S., Ho, M. Y., Velazquez-Martinez, D. N., Brad-shaw, C. M., Szabadi, E., Deakin, J. F., and Anderson, I. M. (2004). Effects of orbital prefrontal cortex dopamine depletion on intertemporal choice: a quantitative analysis. Psychopharmacology (Berl.) 175,206-214.
74. Kim, S., Hwang, J., and Lee, D. (2008). Prefrontal coding of temporally discounted values during intertemporal choice. Neuron 59,161-172.
75. Kobayashi, S., and Schultz, W. (2008). Influence of reward delays on responses of dopamine neurons. J. Neurosci. 28, 7837-7846.
76. Kringelbach, M. L. (2005). The human orbitofrontal cortex: linking reward to hedonic experience. Nat. Rev. Neurosci. 6,691-702.
77. LeDoux, J. E. (2000). "The amygdala and emotion: a view through fear," in The Amygdala: A Functional Analysis, ed. J. P. Aggleton (NewYork: Oxford University Press), 289-310.
78. Lowenstein, G. E. J. (1992). Choice Over Time. New York: Russel Sage Foundation.
79. Malkova, L., Gaffan, D., and Murray, E. A. (1997). Excitotoxic lesions of the amygdala fail to produce impairment in visual learning for auditory secondary reinforcement but interfere with reinforcer devaluation effects in rhesus monkeys. J. Neurosci. 17,6011-6020.
80. Mar, A. C., Walker, A. L., Theobald, D. E., Eagle, D. M., and Robbins, T. W. (2011). Dissociable effects of lesions to orbitofrontal cortex sub-regions on impulsive choice in the rat. J. Neurosci. 31, 6398-6404.
81. McClure, S. M., Ericson, K. M., Laibson, D. I., Loewenstein, G., and Cohen, J. D. (2007). Time discounting for primary rewards. J. Neurosci. 27, 5796-5804.
82. McClure, S. M., Laibson, D. I., Loewenstein, G., and Cohen, J. D. (2004). Separate neural systems value immediate and delayed monetary rewards. Science 306, 503-507.
83. Mirenowicz, J., and Schultz, W. (1994). Importance of unpredictability for reward responses in primate dopamine neurons. J. Neurophysiol. 72,1024-1027.
84. Miyazaki, K., Miyazaki, K. W., and Doya, K. (2011a). Activation of dorsal raphe serotonin neurons underlies waiting for delayed rewards. J. Neurosci. 31,469-479.
85. Miyazaki, K. W., Miyazaki, K., and Doya, K. (2011b). Activation of the central serotonergic system in response to delayed but not omitted rewards. Eur. J. Neurosci. 33, 153-160.
86. Mobini, S., Body, S., Ho, M. Y., Brad-shaw, C. M., Szabadi, E., Deakin, J. F., and Anderson, I. M. (2002). Effects of lesions of the orbitofrontal cortex on sensitivity to delayed and probabilistic reinforcement. Psychopharmacology (Berl.) 160, 290-298.
87. Mobini, S., Chiang, T. J., Al-Ruwaitea, A. S., Ho, M. Y., Bradshaw, C. M., and Szabadi, E. (2000a). Effect of central 5-hydroxytryptamine depletion on inter-temporal choice: a quantitative analysis. Psychopharmacology (Berl.) 149,313-318.
88. Mobini, S., Chiang, T. J., Ho, M. Y., Bradshaw, C. M., and Szabadi, E. (2000b). Effects of central 5-hydroxytryptamine depletion on sensitivity to delayed and probabilistic reinforcement. Psychopharmacology (Berl.) 152, 390-397.
89. Mogenson, G. J., Jones, D. L., and Yim, C. Y. (1980). From motivation to action: functional interface between the limbic system and the motor system. Prog. Neurobiol. 14, 69-97.
90. Montague, P.R., and Berns, G.S. (2002). Neural economics and the biological substrates of valuation. Neuron 36, 265-284.
91. Montague, P. R., Dayan, P., and Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning.J. Neurosci. 16, 1936-1947.
92. Monterosso, J., Ehrman, R., Napier, K. L., O'Brien, C. P., and Childress, A. R. (2001). Three decision-making tasks in cocaine-dependent patients: do they measure the same construct? Addiction 96, 1825-1837.
93. Morris, G., Nevet, A., Arkadir, D., Vaa-dia, E., and Bergman, H. (2006). Midbrain dopamine neurons encode decisions for future action. Nat. Neurosci. 9, 1057-1063.
94. Murray, E. A. (2007). The amygdala, reward and emotion. Trends Cogn. Sci. (Regul. Ed.) 11,489-497.
95. Nakahara, H., Itoh, H., Kawagoe, R., Takikawa, Y., and Hikosaka, O. (2004). Dopamine neurons can represent context-dependent prediction error. Neuron 41, 269-280.
96. Nakamura, K., and Hikosaka, O. (2006). Role of dopamine in the primate caudate nucleus in reward modulation of saccades. J. Neurosci. 26, 5360-5369.
97. Nicola, S. M. (2007). The nucleus accumbens as part of a basal ganglia action selection circuit. Psychopharmacology (Berl.) 191, 521-550.
98. O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K. J., and Dolan, R. J. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304,452-454.
99. O'Neill, M., and Schultz, W. (2010). Coding of reward risk by orbitofrontal neurons is mostly distinct from coding of reward value. Neuron 68, 789-800.
100. Padoa-Schioppa, C. (2011). Neurobi-ology of economic choice: a good-based model. Annu. Rev. Neurosci. 34, 333-359.
101. Pan, W. X., Schmidt, R., Wickens, J. R., and Hyland, B. I. (2005). Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J. Neurosci. 25, 6235-6242.
102. Parkinson, J. A., Crofts, H. S., McGuigan, M., Tomic, D. L., Everitt, B. J., and Roberts, A. C. (2001). The role of the primate amygdala in conditioned reinforcement. J. Neurosci. 21, 7770-7780.
103. Pennartz, C. M., Groenewegen, H. J., and Lopes da Silva, F. H. (1994). The nucleus accumbens as a complex of functionally distinct neu-ronal ensembles: an integration of behavioural, electrophysiological and anatomical data. Prog. Neurobiol. 42,719-761.
104. Redgrave, P., Prescott, T. J., and Gurney, K. (1999). The basal ganglia: a vertebrate solution to the selection problem? Neuroscience 89, 1009-1023.
105. Richards, J. B., Mitchell, S. H., de Wit, H., and Seiden, L. S. (1997). Determination of discount functions in rats with an adjusting-amount procedure. J. Exp. Anal. Behav. 67, 353-366.
106. Rodriguez, M. L., and Logue, A. W. (1988). Adjusting delay to reinforcement: comparing choice in pigeons and humans. J. Exp. Psychol. Anim. Behav. Process. 14, 105-117.
107. Roesch, M. R., Calu, D. J., Burke, K. A., and Schoenbaum, G. (2007a). Should I stay or should I go? Transformation of time-discounted rewards in orbitofrontal cortex and associated brain circuits. Ann. N. Y. Acad. Sci. 1104,21-34.
108. Roesch, M. R., Calu, D. J., and Schoenbaum, G. (2007b). Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat. Neurosci. 10, 1615-1624.
109. Roesch, M. R., Takahashi, Y., Gugsa, N., Bissonette, G. B., and Schoen-baum, G. (2007c). Previous cocaine exposure makes rats hypersensitive to both delay and reward magnitude. J. Neurosci. 27, 245-250.
110. Roesch, M. R., Calu, D. J., Esber, G. R., and Schoenbaum, G. (2010a). All that glitters... dissociating attention and outcome expectancy from prediction errors signals. J. Neurophys-iol. 104, 587-595.
111. Roesch, M. R., Calu, D. J., Esber, G. R., and Schoenbaum, G. (2010b). Neural correlates of variations in event processing during learning in basolateral amygdala. J. Neurosci. 30, 2464-2471.
112. Roesch, M. R., and Olson, C. R. (2005a). Neuronal activity dependent on anticipated and elapsed delay in macaque prefrontal cortex, frontal and supplementary eye fields, and premotor cortex. J. Neurophys-iol. 94, 1469-1497.
113. Roesch, M. R., and Olson, C. R. (2005b). Neuronal activity in primate orbitofrontal cortex reflects the value of time. J. Neurophysiol. 94, 2457-2471.
114. Roesch, M. R., Singh, T., Brown, P. L., Mullins, S. E., and Schoenbaum, G. (2009). Ventral striatal neurons encode the value of the chosen action in rats deciding between differently delayed or sized rewards. J. Neurosci. 29, 13365-13376.
115. Roesch, M. R., Taylor, A. R., and Schoenbaum, G. (2006). Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation. Neuron 51, 509-520.
116. Rudebeck, P. H., Walton, M. E., Smyth, A. N., Bannerman, D. M., and Rush-worth, M. F. (2006). Separate neural pathways process different decision costs. Nat. Neurosci. 9,1161-1168.
117. Saddoris, M. P., Gallagher, M., and Schoenbaum, G. (2005). Rapid associative encoding in basolateral amygdala depends on connections with orbitofrontal cortex. Neuron 46,321-331.
118. Schoenbaum, G., and Roesch, M. (2005). Orbitofrontal cortex, associative learning, and expectancies. Neuron 47, 633-636.
119. Schoenbaum, G., Setlow, B., Saddoris, M. P., and Gallagher, M. (2003). Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala. Neuron 39, 855-867.
120. Schultz, W. (2010). Subjective neuronal coding of reward: temporal value discounting and risk. Eur. J. Neurosci. 31,2124-2135.
121. Sellitto, M., Ciaramelli, E., and di Pelle-grino, G. (2010). Myopic discounting of future rewards after medial orbitofrontal damage in humans. J. Neurosci. 30,16429-16436.
122. Setlow, B., Schoenbaum, G., and Gallagher, M. (2003). Neural encoding in ventral striatum during olfactory discrimination learning. Neuron 38, 625-636.
123. Shen, W., Flajolet, M., Greengard, P., and Surmeier, D. J. (2008). Dichoto-mous dopaminergic control of striatal synaptic plasticity. Science 321, 848-851.
124. Stalnaker, T. A., Calhoon, G. G., Ogawa, M., Roesch, M. R., and Schoenbaum, G. (2010). Neural correlates of stimulus-response and response-outcome associations in dorso-lateral versus dorsomedial striatum. Front. Integr. Neurosci. 4:12. doi:10.3389/fnint.2010.00012
125. Taha, S. A., Nicola, S. M., and Fields, H. L. (2007). Cue-evoked encoding of movement planning and execution in the rat nucleus accumbens. J. Physiol. (Lond.) 584, 801-818.
126. Takahashi, Y. K., Roesch, M. R., Stalnaker, T. A., Haney, R. Z., Calu, D. J., Taylor, A. R., Burke, K. A., and Schoenbaum, G. (2009). The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes. Neuron 62, 269-280.
127. Tanaka, S. C., Doya, K., Okada, G., Ueda, K., Okamoto, Y., and Yamawaki, S. (2004). Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat. Neurosci. 7, 887-893.
128. Thaler, R. (1981). Some empirical evidence on dynamic inconsistency. Econ. Lett. 8,201-207.
129. Tobler, P. N., Dickinson, A., and Schultz, W. (2003). Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. J. Neurosci. 23, 10402-10410.
130. Tye, K. M., Cone, J. J., Schairer, W. W., and Janak, P. H. (2010). Amygdala neural encoding of the absence of reward during extinction. J. Neurosci. 30, 116-125.
131. van der Meer, M. A., and Redish, A. D. (2009). Covert expectation of reward in rat ventral striatum at decision points. Front. Integr. Neurosci. 3:1. doi:10.3389/neuro.07.001.2009
132. Wade, T. R., de Wit, H., and Richards, J. B. (2000). Effects of dopaminergic drugs on delayed reward as a measure of impulsive behavior in rats. Psychopharmacology (Berl.) 150,90-101.
133. Waelti, P., Dickinson, A., and Schultz, W. (2001). Dopamine responses comply with basic assumptions of formal learning theory. Nature 412,43-48.
134. Wallis, J. D., and Kennerley, S. W. (2010). Heterogeneous reward signals in prefrontal cortex. Curr. Opin. Neurobiol. 20, 191-198.
135. Winstanley, C. A. (2007). The orbitofrontal cortex, impul-sivity, and addiction: probing orbitofrontal dysfunction at the neural, neurochemical, and molecular level. Ann. N Y. Acad. Sci. 1121, 639-655.
136. Winstanley, C. A., Dalley, J. W., Theobald, D. E., and Robbins, T. W (2004a). Fractionating impulsiv-ity: contrasting effects of central 5-HT depletion on different measures of impulsive behavior. Neuropsy-chopharmacology 29, 1331-1343.
137. Winstanley, C. A., Theobald, D. E., Cardinal, R. N., and Robbins, T. W. (2004b). Contrasting roles of basolateral amygdala and orbitofrontal cortex in impulsive choice. J. Neurosci. 24,4718-4722.
138. Winstanley, C. A., Theobald, D. E., Dalley, J. W., Glennon, J. C., and Robbins, T. W (2004c). 5-HT2A and 5-HT2C receptor antagonists have opposing effects on a measure of impulsivity: interactions with global 5-HT depletion. Psychopharmacology (Berl.) 176,376-385.
139. Winstanley, C. A., Theobald, D. E. H., Cardinal, R. N., and Robbins, T. W. (2004d). Contrasting roles of basolateral amygdala and orbitofrontal cortex in impulsive choice. J. Neurosci. 24,4718-4722.
140. Wogar, M. A., Bradshaw, C. M., and Szabadi, E. (1993). Effect of lesions of the ascending 5-hydroxytryptaminergic pathways on choice between delayed reinforcers. Psychopharmacology (Berl.) 111, 239-243.
141. Yacubian, J., Glascher, J., Schroeder, K., Sommer, T., Braus, D. F., and Buchel, C. (2006). Dissociable systems for gain- and loss-related value predictions and errors of prediction in the human brain. J. Neurosci. 26, 9530-9537.
142. Zeeb, F. D., Floresco, S. B., and Winstanley, C. A. (2010). Contributions of the orbitofrontal cortex to impulsive choice: interactions with basal levels of impulsivity, dopamine signalling, and reward-related cues. Psychopharmacology (Berl.) 211, 87-98.