Ventral striatum encodes past and predicted value independent of motor contingencies

Journal of Neuroscience

Volumn 32, Issue 6, 2012, Pages 2027-2036

  Goldstein, Brandon L ^a   Barnett, Brian R ^a   Vasquez, Gloria ^a   Tobia, Steven C ^a   Kashtelyan, Vadim ^a   Burton, Amanda C ^a   Bryden, Daniel W ^a   Roesch, Matthew R ^a  

^a UNIVERSITY OF MARYLAND   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ANIMAL EXPERIMENT; ARTICLE; ASSOCIATION; BEHAVIOR; CORPUS STRIATUM; LATENT PERIOD; MALE; MOTOR PERFORMANCE; NONHUMAN; ODOR; PRIORITY JOURNAL; RAT; REWARD; SINGLE UNIT ACTIVITY;

ANIMALS; BASAL GANGLIA; CONDITIONING, OPERANT; FORECASTING; MALE; ODORS; PSYCHOMOTOR PERFORMANCE; RATS; RATS, LONG-EVANS; REACTION TIME; REWARD;

EID: 84857000663     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.5349-11.2012     Document Type: Article

References (78)

1. Balleine BW (2005) Neural bases of food-seeking: affect, arousal and reward in corticostriatolimbic circuits. Physiol Behav 86:717-730.
2. Barto A (1995) Adaptive critics and the basal ganglia. Available at: http://works.bepress.com/andrew_barto/9.
3. Berridge KC, Robinson TE (1998) What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res Brain Res Rev 28:309-369.
4. BloklandA (1998) Reaction time responding in rats. Neurosci Biobehav Rev 22:847-864.
5. Bowman EM, Aigner TG, Richmond BJ (1996) Neural signals in the monkey ventral striatum related to motivation for juice and cocaine rewards. J Neurophysiol 75:1061-1073.
6. Brog JS, Salyapongse A, Deutch AY, Zahm DS (1993) The patterns of afferent innervation of the core and shell in the "accumbens" part of the rat ventral striatum: immunohistochemical detection of retrogradely transported fluoro-gold. J Comp Neurol 338:255-278.
7. Bryden DW, Johnson EE, Diao X, Roesch MR (2011) Impact of expected value on neural activity in rat substantia nigra pars reticulata. Eur J Neurosci 33:2308-2317.
8. Cardinal RN, Parkinson JA, Hall J, Everitt BJ (2002a) Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci Biobehav Rev 26:321-352.
9. Cardinal RN, Parkinson JA, Lachenal G, Halkerston KM, Rudarakanchana N, Hall J, Morrison CH, Howes SR, Robbins TW, Everitt BJ (2002b) Effects of selective excitotoxic lesions of the nucleus accumbens core, anterior cingulate cortex, and central nucleus of the amygdala on autoshaping performance in rats. Behav Neurosci 116:553-567.
10. Carelli RM (2002) Nucleus accumbens cell firing during goal-directed behaviors for cocaine vs 'natural' reinforcement. Physiol Behav 76:379-387.
11. Carelli RM, Deadwyler SA (1994) Acomparison of nucleus accumbens neuronal firing patterns during cocaine self-administration and water reinforcement in rats. J Neurosci 14:7735-7746.
12. Cromwell HC, SchultzW (2003) Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. J Neurophysiol 89:2823-2838.
13. Cromwell HC, Hassani OK, SchultzW (2005) Relative reward processing in primate striatum. Exp Brain Res 162:520-525.
14. Day JJ, Jones JL, Carelli RM (2011) Nucleus accumbens neurons encode predicted and ongoing reward costs in rats. Eur J Neurosci 33:308-321.
15. Di Chiara G (2002) Nucleus accumbens shell and core dopamine: differential role in behavior and addiction. Behav Brain Res 137:75-114.
16. Di Ciano P, Cardinal RN, Cowell RA, Little SJ, Everitt BJ (2001) Differential involvement of NMDA, AMPA/kainate, and dopamine receptors in the nucleus accumbens core in the acquisition and performance of pavlovian approach behavior. J Neurosci 21:9471-9477.
17. Everitt BJ, Morris KA, O'Brien A, Robbins TW (1991) The basolateral amygdala-ventral striatal system and conditioned place preference: further evidence of limbic-striatal interactions underlying reward-related processes. Neuroscience 42:1-18.
18. Floresco SB, St Onge JR, Ghods-Sharifi S, Winstanley CA (2008) Corticolimbic-striatal circuits subserving different forms of cost-benefit decision making. Cogn Affect Behav Neurosci 8:375-389.
19. German PW, Fields HL (2007) Rat nucleus accumbens neurons persistently encode locations associated with morphine reward. J Neurophysiol 97:2094-2106.
20. Ghods-Sharifi S, Floresco SB (2010) Differential effects on effort discounting induced by inactivations of the nucleus accumbens core or shell. Behav Neurosci 124:179-191.
21. Giertler C, Bohn I, HauberW (2003) The rat nucleus accumbens is involved in guiding of instrumental responses by stimuli predicting reward magnitude. Eur J Neurosci 18:1993-1996.
22. Groenewegen HJ, Russchen FT (1984) Organization of the efferent projections of the nucleus accumbens to pallidal, hypothalamic, and mesencephalic structures: a tracing and immunohistochemical study in the cat. J Comp Neurol 223:347-367.
23. Gruber AJ, O'Donnell P (2009) Bursting activation of prefrontal cortex drives sustained up states in nucleus accumbens spiny neurons in vivo. Synapse 63:173-180.
24. Gruber AJ, Hussain RJ, O'Donnell P (2009) The nucleus accumbens: a switchboard for goal-directed behaviors. PLoS One 4:e5062.
25. Hassani OK, Cromwell HC, Schultz W (2001) Influence of expectation of different rewards on behavior-related neuronal activity in the striatum. J Neurophysiol 85:2477-2489.
26. Hauber W, Bohn I, Giertler C (2000) NMDA, but not dopamine D(2), receptors in the rat nucleus accumbens are involved in guidance of instrumental behavior by stimuli predicting reward magnitude. J Neurosci 20:6282-6288.
27. Heimer L, Zahm DS, Churchill L, Kalivas PW, Wohltmann C (1991) Specificity in the projection patterns of accumbal core and shell in the rat. Neuroscience 41:89-125.
28. Hollander JA, Carelli RM (2007) Cocaine-associated stimuli increase cocaine seeking and activate accumbens core neurons after abstinence. J Neurosci 27:3535-3539.
29. Houk J, Adams JL, Barto AG (1995) A model of how the basal ganglia generate and use neural signals that predict reinforcement. In: Models of information processing in the basal ganglia (Houk J, Davis JL, Beiser DG, eds), pp 249-270. Cambridge, MA: MIT.
30. Ikemoto S, Panksepp J (1999) The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking. Brain Res Brain Res Rev 31:6-41.
31. Ito M, Doya K (2009) Validation of decision-making models and analysis of decision variables in the rat basal ganglia. J Neurosci 29:9861-9874.
32. Janak PH, Chen MT, Caulder T (2004) Dynamics of neural coding in the accumbens during extinction and reinstatement of rewarded behavior. Behav Brain Res 154:125-135.
33. Joel D, Niv Y, Ruppin E (2002) Actor-critic models of the basal ganglia: new anatomical and computational perspectives. Neural Netw 15:535-547.
34. Kennerley SW, Wallis JD (2009a) Encoding of reward and space during a working memory task in the orbitofrontal cortex and anterior cingulate sulcus. J Neurophysiol 102:3352-3364.
35. Kennerley SW, Wallis JD (2009b) Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables. Eur J Neurosci 29:2061-2073.
36. Kim H, Sul JH, Huh N, Lee D, JungMW (2009) Role of striatum in updating values of chosen actions. J Neurosci 29:14701-14712.
37. Kim YB, Huh N, Lee H, Baeg EH, Lee D, Jung MW (2007) Encoding of action history in the rat ventral striatum. J Neurophysiol 98:3548-3556.
38. Kimchi EY, LaubachM (2009) Dynamic encoding of action selection by the medial striatum. J Neurosci 29:3148-3159.
39. Kobayashi S, Pinto de Carvalho O, SchultzW (2010) Adaptation of reward sensitivity in orbitofrontal neurons. J Neurosci 30:534-544.
40. Minamimoto T, La Camera G, Richmond BJ (2009) Measuring and modeling the interaction among reward size, delay to reward, and satiation level on motivation in monkeys. J Neurophysiol 101:437-447.
41. Mogenson GJ, Jones DL, Yim CY (1980) From motivation to action: functional interface between the limbic system and the motor system. Prog Neurobiol 14:69-97.
42. Montague PR, Hyman SE, Cohen JD (2004) Computational roles for dopamine in behavioural control. Nature 431:760-767.
43. Nicola SM, Yun IA, Wakabayashi KT, Fields HL (2004) Cue-evoked firing of nucleus accumbens neurons encodes motivational significance during a discriminative stimulus task. J Neurophysiol 91:1840-1865.
44. Niv Y, Schoenbaum G (2008) Dialogues on prediction errors. Trends Cogn Sci 12:265-272.
45. O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304:452-454.
46. Padoa-SchioppaC (2007) Orbitofrontal cortex and the computation of economic value. Ann N Y Acad Sci 1121:232-253.
47. Padoa-Schioppa C (2009) Range-adapting representation of economic value in the orbitofrontal cortex. J Neurosci 29:14004-14014.
48. Padoa-Schioppa C (2011) Neurobiology of economic choice: a good-based model. Annu Rev Neurosci 34:333-359.
49. Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD (2006) Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442:1042-1045.
50. Redish AD (2004) Addiction as a computational process gone awry. Science 306:1944-1947.
51. Robinson DL, Carelli RM (2008) Distinct subsets of nucleus accumbens neurons encode operant responding for ethanol versus water. Eur J Neurosci 28:1887-1894.
52. Roesch MR, Olson CR (2004) Neuronal activity related to reward value and motivation in primate frontal cortex. Science 304:307-310.
53. Roesch MR, Olson CR (2005) Neuronal activity in primate orbitofrontal cortex reflects the value of time. J Neurophysiol 94:2457-2471.
54. Roesch MR, Olson CR (2007) Neuronal activity related to anticipated reward in frontal cortex: does it represent value or reflect motivation? Ann N Y Acad Sci 1121:431-446.
55. Roesch MR, Singh T, Brown PL, Mullins SE, Schoenbaum G (2009) Ventral striatal neurons encode the value of the chosen action in rats deciding between differently delayed or sized rewards. J Neurosci 29:13365-13376.
56. Salamone JD, CorreaM (2002) Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine. Behav Brain Res 137:3-25.
57. Schultz W, Apicella P, Scarnati E, Ljungberg T (1992) Neuronal activity in monkey ventral striatum related to the expectation of reward. J Neurosci 12:4595-4610.
58. Setlow B, Schoenbaum G, Gallagher M (2003) Neural encoding in ventral striatum during olfactory discrimination learning. Neuron 38:625-636.
59. Shidara M, Richmond BJ (2004) Differential encoding of information about progress through multi-trial reward schedules by three groups of ventral striatal neurons. Neurosci Res 49:307-314.
60. Shidara M, Aigner TG, Richmond BJ (1998) Neuronal signals in the monkey ventral striatum related to progress through a predictable series of trials. J Neurosci 18:2613-2625.
61. Simmons JM, Ravel S, Shidara M, Richmond BJ (2007) A comparison of reward-contingent neuronal activity in monkey orbitofrontal cortex and ventral striatum: guiding actions toward rewards. Ann N Y Acad Sci 1121:376-394.
62. Stopper CM, Floresco SB (2011) Contributions of the nucleus accumbens and its subregions to different aspects of risk-based decision making. Cogn Affect Behav Neurosci 11:97-112.
63. Sutton RS, Barto AG (eds) (1998) Reinforcement learning: an introduction. Cambridge, MA: MIT.
64. Taha SA, Fields HL (2006) Inhibitions of nucleus accumbens neurons encode a gating signal for reward-directed behavior. J Neurosci 26:217-222.
65. Takahashi Y, Schoenbaum G, Niv Y (2008) Silencing the critics: understanding the effects of cocaine sensitization on dorsolateral and ventral striatum in the context of an actor/critic model. Front Neurosci 2:86-99.
66. Takahashi Y, Roesch MR, Stalnaker TA, Schoenbaum G (2007) Cocaine exposure shifts the balance of associative encoding from ventral to dorsolateral striatum. Front Integr Neurosci 1:11.
67. Tremblay L, Schultz W (1999) Relative reward preference in primate orbitofrontal cortex. Nature 398:704-708.
68. van der Meer MA, Redish AD (2009) Covert expectation-of-reward in rat ventral striatum at decision points. Front Integr Neurosci 3:1.
69. van der Meer MA, Redish AD (2011) Ventral striatum: a critical look at models of learning and evaluation. Curr Opin Neurobiol 21:387-392.
70. van der Meer MA, Johnson A, Schmitzer-Torbert NC, Redish AD (2010) Triple dissociation of information processing in dorsal striatum, ventral striatum, and hippocampus on a learned spatial decision task. Neuron 67:25-32.
71. Voorn P, Vanderschuren LJ, Groenewegen HJ, Robbins TW, Pennartz CM (2004) Putting a spin on the dorsal-ventral divide of the striatum. Trends Neurosci 27:468-474.
72. Wadenberg ML, Ericson E, Magnusson O, Ahlenius S (1990) Suppression of conditioned avoidance behavior by the local application of ()sulpiride into the ventral, but not the dorsal, striatum of the rat. Biol Psychiatry 28:297-307.
73. Wakabayashi KT, Fields HL, Nicola SM (2004) Dissociation of the role of nucleus accumbens dopamine in responding to reward-predictive cues and waiting for reward. Behav Brain Res 154:19-30.
74. Wallis JD (2007) Orbitofrontal cortex and its contribution to decisionmaking. Annu Rev Neurosci 30:31-56.
75. Wallis JD, KennerleySW (2010) Heterogeneous reward signals in prefrontal cortex. Curr Opin Neurobiol 20:191-198.
76. Watanabe M, Cromwell HC, Tremblay L, Hollerman JR, Hikosaka K, Schultz W (2001) Behavioral reactions reflecting differential reward expectations in monkeys. Exp Brain Res 140:511-518.
77. Wright CI, Groenewegen HJ (1995) Patterns of convergence and segregation in the medial nucleus accumbens of the rat: relationships of prefrontal cortical, midline thalamic, and basal amygdaloid afferents. J Comp Neurol 361:383-403.
78. Yun IA, Wakabayashi KT, Fields HL, NicolaSM (2004) The ventral tegmental area is required for the behavioral and nucleus accumbens neuronal firing responses to incentive cues. J Neurosci 24:2923-2933.