Impact of appetitive and aversive outcomes on brain responses: Linking the animal and human literatures

Frontiers in Systems Neuroscience

Volumn 8, Issue MAR, 2014, Pages

  Bissonette, Gregory B ^a   Gentry, Ronny N ^a   Padmala, Srikanth ^a   Pessoa, Luiz ^a   Roesch, Matthew R ^a  

^a UNIVERSITY OF MARYLAND   (United States)

Author keywords

Appetitive and aversive outcomes; Neural encoding; Reward processing; Salience; Value encoding

Indexed keywords

DOPAMINE;

ANTERIOR CINGULATE; APPETITE; AVERSIVE BEHAVIOR; BASOLATERAL AMYGDALA; BEHAVIOR; CORPUS STRIATUM; DECISION MAKING; DOPAMINERGIC NERVE CELL; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HAPLORHINI; HUMAN; MENTAL DISEASE; MENTAL TASK; MOTIVATION; NERVE CELL; NONHUMAN; ORBITAL CORTEX; PARIETAL CORTEX; PREDICTION; PUNISHMENT; REVIEW; REWARD;

EID: 84896967269     PISSN: 16625137     EISSN: None     Source Type: Journal    
DOI: 10.3389/fnsys.2014.00024     Document Type: Review

References (146)

1. Adolphs, R., and Tranel, D. (2000). "Emotion recognition and the human amygdala, in TheAmygdala:A FunctionalAnlysis, ed J. P. Aggleton (NewYork: Oxford University Press), 587-630.
2. Amemori, K., and Graybiel, A. M. (2012). Localized microstimulation of primate pregenual cingulate cortex induces negative decision-making. Nat. Neurosci. 15, 776-785. doi: 10.1038/nn.3088
3. Anderson, A. K. (2005). Affective influences on the attentional dynamics supporting awareness. J. Exp. Psychol. Gen. 134, 258-281. doi: 10.1037/0096-3445.134. 2.258
4. Anderson, A. K., Christoff, K., Stappen, I., Panitz, D., Ghahremani, D. G., Glover, G., et al. (2003). Dissociated neural representations of intensity and valence in human olfaction. Nat. Neurosci. 6, 196-202. doi: 10.1038/nn1001
5. Anderson, B. A., Laurent, P. A., and Yantis, S. (2011). Value-driven attentional capture. Proc. Natl. Acad. Sci. USA 108, 10367-10371. doi: 10.1073/pnas. 1104047108
6. Anstrom, K. K., Miczek, K. A., and Budygin, E. A. (2009). Increased phasic dopamine signaling in the mesolimbic pathway during social defeat in rats. Neuroscience 161, 3-12. doi: 10.1016/j.neuroscience.2009.03.023
7. Belova, M. A., Paton, J. J., Morrison, S. E., and Salzman, C. D. (2007). Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala. Neuron 55, 970-984. doi: 10.1016/j.neuron.2007.08.004
8. Berridge, K. C., and Robinson, T. E. (1998). What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res. Brain Res. Rev. 28, 309-369. doi: 10.1016/s0165-0173(98)00019-8
9. Berridge, K. C. (2007). The debate over dopamine's role in reward: the case for incentive salience. Psychopharmacology (Berl) 191, 391-431. doi: 10. 1007/s00213-006-0578-x
10. Bissonette, G. B., Burton, A. C., Gentry, R. N., Goldstein, B. L., Hearn, T. N., Barnett, B. R., et al. (2013). Separate populations of neurons in ventral striatum encode value and motivation. PLoS One 8:e64673. doi: 10.1371/journal.pone. 0064673
11. Blokland, A. (1998). Reaction time responding in rats. Neurosci. Biobehav. Rev. 22, 847-864. doi: 10.1016/s0149-7634(98)00013-x
12. Breiter, H. C., Aharon, I., Kahneman, D., Dale, A., andShizgal, P. (2001). Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 30, 619-639. doi: 10.1016/s0896-6273(01)00303-8
13. Brischoux, F., Chakraborty, S., Brierley, D. I., and Ungless, M. A. (2009). Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc. Natl. Acad. Sci. USA 106,4894-4899. doi: 10.1073/pnas.0811507106
14. Bromberg-Martin, E. S., Matsumoto, M., and Hikosaka, O. (2010). Dopamine in motivational control: rewarding, aversive and alerting. Neuron 68, 815-834. doi: 10.1016/j.neuron.2010.11.022
15. Bryden, D. W., Johnson, E. E., Tobia, S. C., Kashtelyan, V., andRoesch, M. R. (2011). Attention for learning signals in anterior cingulate cortex. J. Neurosci. 31, 18266-18274. doi: 10.1523/JNEUR0SCI.4715-11.2011
16. Burton, A. C., Bissonette, G. B., Lichtenberg, N. T., Kashtelyan, V., and Roesch, M. R. (2013). Ventral striatum lesions enhance stimulus and response encoding in dorsal striatum. Biol. Psychiatry 75, 132-139. doi: 10.1016/j.biopsych.2013. 05.023
17. Cardinal, R. N., Parkinson, J. A., Hall, J., and Everitt, B. J. (2002b). Emotion and motivation: the role of the amygdala, ventral striatum and prefrontal cortex. Neurosci. Biobehav. Rev. 26, 321-352. doi: 10.1016/s0149-7634(02)00 007-6
18. Cardinal, R. N., Parkinson, J. A., Lachenal, G., Halkerston, K. M., Rudarakanchana, N., Hall, J., et al. (2002a). Effects of selective excitotoxic lesions of the nucleus accumbens core, anterior cingulate cortex and central nucleus of the amygdala on autoshaping performance in rats. Behav. Neurosci. 116, 553-567. doi: 10. 1037/0735-7044.116.4.553
19. Cardinal, R. N., Pennicott, D. R., Sugathapala, C. L., Robbins, T. W., and Everitt, B. J. (2001). Impulsive choice induced in rats by lesions of the nucleus accumbens core. Science 292, 2499-2501. doi: 10.1126/science.1060818
20. Carelli, R. M., and Deadwyler, S. A. (1994). A comparison of nucleus accumbens neuronal firing patterns during cocaine self-administration and water reinforcement in rats. J. Neurosci. 14, 7735-7746.
21. Carter, R. M., Macinnes, J. J., Huettel, S. A., and Adcock, R. A. (2009). Activation in the VTA and nucleus accumbens increases in anticipation of both gains and losses. Front. Behav. Neurosci. 3:21. doi: 10.3389/neuro.08.021.2009
22. Choi, J. M., Padmala, S., Spechler, P., and Pessoa, L. (2013). Pervasive competition between threat and reward in the brain. Soc. Cogn. Affect. Neurosci. doi: 10. 1093/scan/nst053. [Epub ahead of print].
23. Cooper, J. C., and Knutson, B. (2008). Valence and salience contribute to nucleus accumbens activation. Neuroimage 39, 538-547. doi: 10.1016/j.neuroimage. 2007.08.009
24. Craig, A. D. (2002). How do you feel? Interoception: the sense of the physiological condition of the body. Nat. Rev. Neurosci. 3, 655-666.
25. Craig, A. D. (2009). How do you feel-now? The anterior insula and human awareness. Nat. Rev. Neurosci. 10, 59-70. doi: 10.1038/nrn2555
26. Cromwell, H. C., Hassani, O. K., and Schultz, W. (2005). Relative reward processing in primate striatum. Exp. Brain Res. 162, 520-525. doi: 10.1007/s00221-005- 2223-z
27. D'Ardenne, K., McClure, S. M., Nystrom, L. E., and Cohen, J. D. (2008). BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science 319, 1264-1267. doi: 10.1126/science.1150605
28. Davis, M., Walker, D. L., Miles, L., and Grillon, C. (2010). Phasic vs sustained fear in rats and humans: role of the extended amygdala in fear vs anxiety. Neuropsychopharmacology 35, 105-135. doi: 10.1038/npp.2009.109
29. Daw, N. D., Kakade, S., and Dayan, P. (2002). Opponent interactions between serotonin and dopamine. Neural Netw. 15, 603-616. doi: 10.1016/s0893- 6080(02)00052-7
30. Day, J. J., Jones, J. L., and Carelli, R. M. (2011). Nucleus accumbens neurons encode predicted and ongoing reward costs in rats. Eur. J. Neurosci. 33, 308-321. doi: 10. 1111/j.1460-9568.2010.07531.x
31. Delgado, M. R. (2007). Reward related responses in the human striatum. Ann. NY Acad. Sci. 1104, 70-88. doi: 10.1196/annals.1390.002
32. Delgado, M. R., Li, J., Schiller, D., and Phelps, E. A. (2008). The role of the striatum in aversive learning and aversive prediction errors. Philos. Trans. R. Soc. Lond. B Biol. Sci. 363, 3787-3800. doi: 10.1098/rstb.2008.0161
33. Delgado, M. R., Nystrom, L. E., Fissell, C., Noll, D. C., and Fiez, J. A. (2000). Tracking the hemodynamic responses to reward and punishment in the striatum. J. Neurophysiol. 84, 3072-3077.
34. Di Chiara, G. (2002). Nucleus accumbens shell and core dopamine: differential role in behavior and addiction. Behav. Brain Res. 137, 75-114. doi: 10.1016/s0166-4328(02)00286-3
35. Di Ciano, P., Cardinal, R. N., Cowell, R. A., Little, S. J., and Everitt, B. J. (2001). Differential involvement of NMDA, AMPA/kainate and dopamine receptors in the nucleus accumbens core in the acquisition and performance of pavlovian approachbehavior. J. Neurosci. 21, 9471-9477.
36. Everitt, B. J., Cardinal, R. N., Parkinson, J. A., and Robbins, T. W. (2003). Appetitive behavior: impact of amygdala-dependent mechanisms of emotional learning. Ann. NYAcad. Sci. 985, 233-250. doi: 10.1111/j.1749-6632.2003.tb0 7085.x
37. Fiorillo, C. D., Song, M. R., and Yun, S. R. (2013). Multiphasic temporal dynamics in responses of midbrain dopamine neurons to appetitive and aversive stimuli. J. Neurosci. 33, 4710-4725. doi: 10.1523/jneurosci.3883-12.2013
38. Floresco, S. B., St Onge, J. R., Ghods-Sharifi, S., and Winstanley, C. A. (2008). Cortico-limbic-striatal circuits subserving different forms of cost-benefit decision making. Cogn. Affect. Behav. Neurosci. 8, 375-389. doi: 10.3758/cabn. 8.4.375
39. Ghods-Sharifi, S., and Floresco, S. B. (2010). Differential effects on effort discounting induced by inactivations of the nucleus accumbens core or shell. Behav. Neurosci. 124, 179-191. doi: 10.1037/a0018932
40. Giertler, C., Bohn, I., and Hauber, W. (2003). The rat nucleus accumbens is involved in guiding of instrumental responses by stimuli predicting reward magnitude. Eur. J. Neurosci. 18, 1993-1996. doi: 10.1046/j.1460-9568.2003. 02904.x
41. Goense, J. B. M., and Logothetis, N. K. (2008). Neurophysiology of the BOLD fMRI signal in awake monkeys. Curr. Biol. 18, 631-640. doi: 10.1016/j.cub.2008.03.054
42. Goldstein, B. L., Barnett, B. R., Vasquez, G., Tobia, S. C., Kashtelyan, V., Burton, A. C., et al. (2012). Ventral striatum encodes past and predicted value independent of motor contingencies. J. Neurosci. 32, 2027-2036. doi: 10.1523/jneurosci. 5349-11.2012
43. Gruber, A. J., Hussain, R. J., and O'Donnell, P. (2009). The nucleus accum- bens: a switchboard for goal-directed behaviors. PLoS One 4:e5062. doi: 10.1371/journal.pone.0005062
44. Haber, S. N., and Knutson, B. (2010). The reward circuit: linking primate anatomy and human imaging. Neuropsychopharmacology 35, 4-26. doi: 10.1038/npp. 2009.129
45. Haber, S. N., Fudge, J. L., and McFarland, N. R. (2000). Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 20, 2369-2382.
46. Hatfield, T., Han, J. S., Conley, M., Gallagher, M., and Holland, P. (1996). Neurotoxic lesions of basolateral, but not central, amygdala interfere with Pavlovian second-order conditioning and reinforcer devaluation effects. J. Neurosci. 16, 5256-5265.
47. Hayden, B. Y., Heilbronner, S. R., Pearson, J. M., and Platt, M. L. (2011). Surprise signals in anterior cingulate cortex: neuronal encoding of unsigned reward prediction errors driving adjustment in behavior. J. Neurosci. 31, 4178-4187. doi: 10.1523/jneurosci.4652-10.2011
48. Hosokawa, T., Kato, K., Inoue, M., and Mikami, A. (2007). Neurons in the macaque orbitofrontal cortex code relative preference of both rewarding and aversive outcomes. Neurosci. Res. 57, 434-445. doi: 10.1016/j.neures.2006.12.003
49. Houk, J. C., Adams, J. L., and Barto, A. G. (1995). "A model of how the basal ganglia generate and use neural signals that predict reinforcement, in Models ofInformation Processing the Basal Ganglia, eds J. C. Houk, J. L. Davis and D. G. Beiser (Cambridge, USA: MIT press), 249-270.
50. Ikemoto, S., and Panksepp, J. (1999). The role of nucleus a ccumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking. BrainRes. BrainRes. Rev. 31, 6-41. doi: 10.1016/s0165-0173(99) 00023-5
51. Ikemoto, S. (2007). Dopamine reward circuitry: two projection systems from the ventral midbrain to the nucleus accumbens-olfactory tubercle complex. Brain Res. Rev. 56, 27-78. doi: 10.1016/j.brainresrev.2007.05.004
52. Ito, M., and Doya, K. (2009). Validation of decision-making models and analysis of decision variables in the rat basal ganglia. J. Neurosci. 29, 9861-9874. doi: 10. 1523/jneurosci.6157-08.2009
53. Janak, P. H., Chen, M. T., and Caulder, T. (2004). Dynamics of neural coding in the accumbens during extinction and reinstatement of rewarded behavior. Behav. BrainRes. 154, 125-135. doi: 10.1016/j.bbr.2004.02.003
54. Jensen, J., McIntosh, A. R., Crawley, A. P., Mikilis, D. J., Remington, G., and Kapur, S. (2003). Direct activation of the ventral striatum in anticipation of aversive stimuli. Neuron 40, 1251-1257. doi: 10.1016/s0896-6273(03)0 0724-4
55. Jensen, J., Smith, A. J., Willeit, M., Crawley, A. P., Mikulis, D. J., Vitcu, I., et al. (2007). Separate brain regions code for salience vs. valence during reward prediction in humans. Hum. Brain Mapp. 28, 294-302. doi: 10.1002/hbm. 20274
56. Joel, D., Niv, Y., and Ruppin, E. (2002). Actor-critic models of the basal ganglia: new anatomical and computational perspectives. Neural Netw. 15, 535-547. doi: 10. 1016/s0893-6080(02)00047-3
57. Kahnt, T., and Tobler, P. N. (2013). Salience signals in the right temporoparietal junction facilitate value-based decisions. J. Neurosci. 33, 863-869. doi: 10. 1523/jneurosci.3531-12.2013
58. Kalenscher, T., Lansink, C. S., Lankelma, J. V., and Pennartz, C. M. (2010). Reward- associated gamma oscillations in ventral striatum are regionally differentiated and modulate local firing activity. J. Neurophysiol. 103, 1658-1672. doi: 10. 1152/jn.00432.2009
59. Kaping, D., Vinck, M., Hutchison, R. M., Everling, S., and Womelsdorf, T. (2011). Specific contributions ofventromedial, anterior cingulate, and lateral prefrontal cortex for attentional selection and stimulus valuation. PLoS Biol. 9:e1001224. doi: 10.1371/journal.pbio.1001224
60. Kennerley, S. W., Behrens, T. E., and Wallis, J. D. (2011). Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nat. Neurosci. 14, 1581-1589. doi: 10.1038/nn.2961
61. Kim, H., Sul, J. H., Huh, N., Lee, D., and Jung, M. W. (2009). Role of striatum in updating values of chosen actions. J. Neurosci. 29, 14701-14712. doi: 10. 1523/jneurosci.2728-09.2009
62. Knutson, B., Adams, C. M., Fong, G. W., and Hommer, D. (2001). Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J. Neurosci. 21:RC159.
63. Koch, M., Schmid, A., and Schnitzler, H. U. (2000). Role of muscles accumbens dopamine D1 and D2 receptors in instrumental and Pavlovian paradigms of conditioned reward. Psychopharmacology (Berl) 152, 67-73. doi: 10. 1007/s002130000505
64. Lammel, S., Ion, D. I., Roeper, J., and Malenka, R. C. (2011). Projection-specific modulation of dopamine neuron synapses by aversive and rewarding stimuli. Neuron 70, 855-862. doi: 10.1016/j.neuron.2011.03.025
65. Lang, P. J., and Davis, M. (2006). Emotion, motivation and the brain: reflex foundations in animal and human research. Prog. Brain Res. 156, 3-29. doi: 10. 1016/s0079-6123(06)56001-7
66. Lansink, C. S., Goltstein, P. M., Lankelma, J. V., and Pennartz, C. M. (2010). Fast-spiking interneurons of the rat ventral striatum: temporal coordination of activity with principal cells and responsiveness to reward. Eur. J. Neurosci. 32, 494-508. doi: 10.1111/j.1460-9568.2010. 07293.x
67. Laxton, A. W., Neimat, J., Davis, K. D., Womelsdorf, T., Hutchinson, W. D., Dostrovsky, J. O., et al. (2013). Neuronal coding of implicit emotion categories in the subcallosal cortex in patients with depression. Biol. Psychiatry 74, 714-719. doi: 10.1016/j.biopsych.2013.03.029
68. Leathers, M. L., and Olson, C. R. (2012). In monkeys makingvalue-baseddecisions, LIP neurons encode cue salience and not action value. Science 338, 132-135. doi: 10.1126/science.1226405
69. Leathers, M. L., and Olson, C. R. (2013). Response to comment on "In monkeys making value-based decisions, LIP neurons encode cue salience and not action value. Science 340, 430. doi: 10.1126/science.123 3367
70. LeDoux, J. E. (2000). Emotion circuits in the brain. Annu. Rev. Neurosci. 23, 155-184. doi: 10.1146/annurev.neuro.23.1.155
71. Lex, B., and Hauber, W. (2010). The role of nucleus accumbens dopamine in outcome encoding in instrumental and pavlovian conditioning. Neurobiol. Learn. Mem. 93, 283-290. doi: 10.1016/j.nlm.2009.11.002
72. Lindquist, K. A., Wager, T. D., Kober, H., Bliss-Moreau, E., and Barrett, L. F. (2012). The brain basis of emotion: a meta-analytic review. Behav. Brain Sci. 35, 121-143. doi: 10.1017/s0140525x11000446
73. Litt, A., Plassmann, H., Shiv, B., and Rangel, A. (2011). Dissociating valuation and saliency signals during decision-making. Cereb. Cortex 21, 95-102. doi: 10. 1093/cercor/bhq065
74. Liu, X., Hairston, J., Schrier, M., and Fan, J. (2011). Common and distinct networks underlying reward valence and processing stages: a meta-analysis of functional neuroimaging studies. Neurosci. Biobehav. Rev. 35, 1219-1236. doi: 10.1016/j. neubiorev.2010.12.012
75. Matsumoto, M., and Hikosaka, O. (2009). Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459, 837-841. doi: 10. 1038/nature08028
76. Matsumoto, M., Matsumoto, K., Abe, H., and Tanaka, K. (2007). Medial prefrontal cell activity signaling prediction errors of action values. Nat. Neurosci. 10, 647-656. doi: 10.1038/nn1890
77. McClure, S. M., Berns, G. S., and Montague, P. R. (2003). Temporal prediction errors in a passive learning task activate human striatum. Neuron 38, 339-346. doi: 10.1016/s0896-6273(03)00154-5
78. McCullough, L. D., and Salamone, J. D. (1992). Involvement of nucleus accumbens dopamine in the motor activity induced by periodic food presentation: a microdialysis and behavioral study. Brain Res. 592, 29-36. doi: 10.1016/0006- 8993(92)91654-w
79. Metereau, E., and Dreher, J. C. (2013). Cerebral correlates of salient prediction error for different rewards and punishments. Cereb. Cortex 23, 477-487. doi: 10. 1093/cercor/bhs037
80. Monosov, I. E., and Hikosaka, O. (2012). Regionally distinct processing of rewards and punishments by the primate ventromedial prefrontal cortex. J. Neurosci. 32, 10318-10330. doi: 10.1523/jneurosci.1801-12.2012
81. Montague, P. R., Dayan, P., and Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 16, 1936-1947.
82. Morrison, S. E., Saez, A., Lau, B., and Salzman, C. D. (2011). Different time courses for learning-related changes in amygdala and orbitofrontal cortex. Neuron 71, 1127-1140. doi: 10.1016/j.neuron.2011.07.016
83. Morrison, S. E., and Salzman, C. D. (2011). Representations of appetitive and aversive information in the primate orbitofrontal cortex. Ann. N Y Acad. Sci. 1239, 59-70. doi: 10.1111/j.1749-6632.2011.06255.x
84. Nakamura, K., Santos, G. S., Matsuzaki, R., and Nakahara, H. (2012). Differential reward coding in the subdivisions of the primate caudate during an oculomotor task. J. Neurosci. 32, 15963-15982. doi: 10.1523/jneurosci.1518-12. 2012
85. Newsome, W. T., Glimcher, P. W., Gottlieb, J., Lee, D., and Platt, M. L. (2013). Comment on "In monkeys making value-based decisions, LIP neurons encode cue salience and not action value. Science 340, 430. doi: 10.1126/science. 1233214
86. Nicola, S. M., Yun, I. A., Wakabayashi, K. T., and Fields, H. L. (2004). Cue-evoked firing of nucleus accumbens neurons encodes motivational significance during a discriminative stimulus task. J. Neurophysiol. 91, 1840-1865. doi: 10.1152/jn. 00657.2003
87. Niv, Y., and Schoenbaum, G. (2008). Dialogues on prediction errors. Trends Cogn. Sci. 12, 265-272. doi: 10.1016/j.tics.2008.03.006
88. Niv, Y. (2009). Reinforcement learning in the brain. J. Math. Psychol. 53, 139-154. doi: 10.1016/j.jmp.2008.12.005
89. Nunes, E. J., Randall, P. A., Podurgiel, S., Correa, M., and Salamone, J. D. (2013). Nucleus accumbens neurotransmission and effort-related choice behavior in food motivation: effects of drugs acting on dopamine, adenosine and muscarinic acetylcholine receptors. Neurosci. Biobehav. Rev. 37, 2015-2025. doi: 10.1016/j. neubiorev.2013.04.002
90. O'Doherty, J. P. (2004). Reward representations and reward-related learning in the human brain: insights from neuroimaging. Curr. Opin. Neurobiol. 14, 769-776. doi: 10.1016/j.conb.2004.10.016
91. Oleson, E. B., Gentry, R. N., Chioma, V. C., and Cheer, J. F. (2012). Subsecond dopamine release in the nucleus accumbens predicts conditioned punishment and its successful avoidance. J. Neurosci. 32, 14804-14808. doi: 10. 1523/jneurosci.3087-12.2012
92. Padoa-Schioppa, C., and Assad, J. A. (2006). Neurons in the orbitofrontal cortex encode economic value. Nature 441, 223-226. doi: 10.1038/nature04676
93. Padoa-Schioppa, C., and Assad, J. A. (2008). The representation of economic value in the orbitofrontal cortex is invariant for changes of menu. Nat. Neurosci. 11, 95-102. doi: 10.1038/nn2020
94. Padoa-Schioppa, C. (2011). Neurobiology of economic choice: a good-based model. Annu. Rev. Neurosci. 34, 333-359. doi: 10.1146/annurev-neuro-061010- 113648
95. Park, S. Q., Kahnt, T., Rieskamp, J., and Heekeren, H. R. (2011). Neurobiology of value integration: when value impacts valuation. J. Neurosci. 31, 9307-9314. doi: 10.1523/JNEUROSCI.4973-10.2011
96. Paton, J. J., Belova, M. A., Morrison, S. E., and Salzman, C. D. (2006). The primate amygdala represents the positive and negative value of visual stimuli during learning. Nature 439, 865-870. doi: 10.1038/nature04490
97. Phelps, E. A., Ling, S., and Carrasco, M. (2006). Emotion facilitates perception and potentiates the perceptual benefits of attention. Psychol. Sci. 17,292-299. doi: 10. 1111/j.1467-9280.2006.01701.x
98. Plassmann, H., O'Doherty, J. P., and Rangel, A. (2010). Appetitive and aversive goal values are encoded in the medial orbitofrontal cortex at the time of decision making. J. Neurosci. 30, 10799-10808. doi: 10.1523/jneurosci.0788-10. 2010
99. Pruessner, J. C., Champagne, F., Meaney, M. J., and Dagher, A. (2004). Dopamine release in response to a psychological stress in humans and its relationship to early life maternal care: a positron emission tomography study using [11C]raclopride. J. Neurosci. 24, 2825-2831. doi: 10.1523/jneurosci.3422-03.2004
100. Redish, A. D. (2004). Addiction as a computational process gone awry. Science 306, 1944-1947. doi: 10.1126/science.1102384
101. Rescorla, R. A., and Wagner, A. R. (1972). Classical ConditioningII: CurrentResearch and Theory, eds A. Black and W. F. Prokasy (Appleton: Century-Crofts), 64-99.
102. Roesch, M. R., Calu, D. J., Esber, G. R., and Schoenbaum, G. (2010a). Neural correlates of variations in event processing during learning in basolat- eral amygdala. J. Neurosci. 30, 2464-2471. doi: 10.1523/jneurosci.5781-09. 2010
103. Roesch, M. R., Calu, D. J., Esber, G. R., and Schoenbaum, G. (2010b). All that glitters ... dissociating attention and outcome expectancy from prediction errors signals. J. Neurophysiol. 104, 587-595. doi: 10.1152/jn.00173.2010
104. Roesch, M. R., and Olson, C. R. (2004). Neuronal activity related to reward value and motivation in primate frontal cortex. Science 304, 307-310. doi: 10. 1126/science.1093223
105. Rolls, E. T., Sienkiewicz, Z. J., and Yaxley, S. (1989). Hunger modulates the responses to gustatory stimuli of single neurons in the caudolateral orbitofrontal cortex of the macaque monkey. Eur. J. Neurosci. 1, 53-60. doi: 10.1111/j.1460- 9568.1989.tb00774.x
106. Saddoris, M. P., Gallagher, M., and Schoenbaum, G. (2005). Rapid associative encoding in basolateral amygdala depends on connections with orbitofrontal cortex. Neuron 46, 321-331. doi: 10.1016/j.neuron.2005.02.018
107. Salamone, J. D. (1986). Different effects of haloperidol and extinction on instrumental behaviours. Psychopharmacology (Berl) 88, 18-23. doi: 10.1007/ bf00310507
108. Salamone, J. D. (1994). The involvement of nucleus accumbens dopamine in appetitive and aversive motivation. Behav. Brain Res. 61, 117-133. doi: 10. 1016/0166-4328(94)90153-8
109. Salamone, J. D., and Correa, M. (2002). Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accum- bens dopamine. Behav. Brain Res. 137, 3-25. doi: 10.1016/s0166-4328(02)00 282-6
110. Salamone, J. D., and Correa, M. (2012). The mysterious motivational functions of mesolimbic dopamine. Neuron 76, 470-485. doi: 10.1016/j.neuron.2012.10.021
111. Salamone, J. D., Correa, M., Nunes, E. J., Randall, P. A., and Pardo, M. (2012). The behavioral pharmacology of effort-related choice behavior: dopamine, adenosine and beyond. J. Exp. Anal. Behav. 97, 125-146. doi: 10.1901/jeab.2012. 97-125
112. Salamone, J. D., Steinpreis, R. E., McCullough, L. D., Smith, P., Grebel, D., and Mahan, K. (1991). Haloperidol and nucleus accumbens dopamine depletion suppress lever pressing for food but increase free food consumption in a novel food choice procedure. Psychopharmacology (Berl) 104, 515-521. doi: 10. 1007/bf02245659
113. Schoenbaum, G., Chiba, A. A., and Gallagher, M. (1998). Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nat. Neurosci. 1, 155-159. doi: 10.1038/407
114. Schoenbaum, G., Chiba, A. A., and Gallagher, M. (1999). Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning. J. Neurosci. 19, 1876-1884.
115. Schoenbaum, G., and Roesch, M. (2005). Orbitofrontal cortex, associative learning, and expectancies. Neuron 47, 633-636. doi: 10.1016/j.neuron.2005.07.018
116. Schoenbaum, G., Setlow, B., Nugent, S. L., Saddoris, M. P., and Gallagher, M. (2003a). Lesions of orbitofrontal cortex and basolateral amygdala complex disrupt acquisition of odor-guided discriminations and reversals. Learn. Mem. 10, 129-140. doi: 10.1101/lm.55203
117. Schoenbaum, G., Setlow, B., Saddoris, M. P., and Gallagher, M. (2003b). Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala. Neuron 39, 855-867. doi: 10.1016/s0896-6273(03)00474-4
118. Schultz, W. (1997). Dopamine neurons and their role in reward mechanisms. Curr. Opin. Neurobiol. 7, 191-197. doi: 10.1016/s0959-4388(97)80007-4
119. Schultz, W. (1998). Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1-27.
120. Schultz, W., Apicella, P., Scarnati, E., and Ljungberg, T. (1992). Neuronal activity in monkey ventral striatum related to the expectation of reward. J. Neurosci. 12, 4595-4610.
121. Schultz, W., Tremblay, L., and Hollerman, J. R. (2000). Reward processing in primate orbitofrontal cortex and basal ganglia. Cereb. Cortex 10, 272-284. doi: 10.1093/cercor/10.3.272
122. Setlow, B., Schoenbaum, G., and Gallagher, M. (2003). Neural encoding in ventral striatum during olfactory discrimination learning. Neuron 38, 625-636. doi: 10. 1016/s0896-6273(03)00264-2
123. Shabel, S. J., and Janak, P. H. (2009). Substantial similarity in amygdala neuronal activity during conditioned appetitive and aversive emotional arousal. Proc. Natl. Acad. Sci. USA 106, 15031-15036. doi: 10.1073/pnas.0905580106
124. Shidara, M., and Richmond, B. J. (2004). Differential encoding of information about progress through multi-trial reward schedules by three groups of ventral striatal neurons. Neurosci. Res. 49, 307-314. doi: 10.1016/j.neures.2004.03.008
125. Singh, T., McDannald, M. A., Haney, R. Z., Cerri, D. H., and Schoenbaum, G. (2010). Nucleus accumbens core and shell are necessary for reinforcer devaluation effects on pavlovian conditioned responding. Front. Integr. Neurosci. 4:126. doi: 10.3389/fnint.2010.00126
126. Small, D. M., Gregory, M. D., Mak, Y. E., Gitelman, D., Mesulam, M. M., and Parrish, T. (2003). Dissociation of neural representation of intensity and affective valuation in human gustation. Neuron 39, 701-711. doi: 10.1016/s0896- 6273(03)00467-7
127. Solomon, R. L., and Corbit, J. D. (1974). An opponent-process theory of motivation. I. Temporal dynamics of affect. Psychol. Rev. 81, 119-145. doi: 10. 1037/h0036128
128. Stalnaker, T. A., Roesch, M. R., Franz, T. M., Calu, D. J., Singh, T., and Schoenbaum, G. (2007). Cocaine-induced decision-making deficits are mediated by miscoding in basolateral amygdala. Nat. Neurosci. 10, 949-951. doi: 10.1038/nn1931
129. Stalnaker, T. A., Takahashi, Y., Roesch, M. R., and Schoenbaum, G. (2009). Neural substrates of cognitive inflexibility after chronic cocaine exposure. Neuropharmacology 56(Suppl. 1), 63-72. doi: 10.1016/j.neuropharm.2008.07.019
130. Stopper, C. M., and Floresco, S. B. (2011). Contributions of the nucleus accumbens and its subregions to different aspects of risk-based decision making. Cogn. Affect. Behav. Neurosci. 11, 97-112. doi: 10.3758/s13415-010-0015-9
131. Sutton RaB, AG. editor (1998). Reinforcement Learning: An Introduction. MA Cambridge: MIT Press.
132. Takahashi, Y. K., Roesch, M. R., Wilson, R. C., Toreson, K., O'Donnell, P., Niv, Y., et al. (2011). Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex. Nat. Neurosci. 14, 1590-1597. doi: 10.1038/nn. 2957
133. Takahashi, Y., Schoenbaum, G., and Niv, Y. (2008). Silencing the critics: understanding the effects of cocaine sensitization on dorsolateral and ventral striatum in the context of an actor/critic model. Front. Neurosci. 2, 86-99. doi: 10. 3389/neuro.01.014.2008
134. Talmi, D., Dayan, P., Kiebel, S. J., Frith, C. D., and Dolan, R. J. (2009). How humans integrate the prospects of pain and reward during choice. J. Neurosci. 29, 14617-14626. doi: 10.1523/jneurosci.2026-09.2009
135. Tricomi, E. M., Delgado, M. R., and Fiez, J. A. (2004). Modulation of caudate activity by action contingency. Neuron 41, 281-292. doi: 10.1016/s0896- 6273(03)00848-1
136. Tye, K. M., Cone, J. J., Schairer, W. W., and Janak, P. H. (2010). Amygdala neural encoding of the absence of reward during extinction. J. Neurosci. 30, 116-125. doi: 10.1523/jneurosci.4240-09.2010
137. Tye, K. M., and Janak, P. H. (2007). Amygdala neurons differentially encode motivation and reinforcement. J. Neurosci. 27, 3937-3945. doi: 10.1523/jneurosci. 5281-06.2007
138. Ungless, M. A., Magill, P. J., and Bolam, J. P. (2004). Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science 303, 2040-2042. doi: 10.1126/science.1093360
139. van der Meer, M. A., Johnson, A., Schmitzer-Torbert, N. C., and Redish, A. D. (2010). Triple dissociation ofinformation processing in dorsal striatum, ventral striatum and hippocampus on a learned spatial decision task. Neuron 67, 25-32. doi: 10.1016/j.neuron.2010.06.023
140. van der Meer, M. A., and Redish, A. D. (2009). Covert expectation-of-reward in rat ventral striatum at decision points. Front. Integr. Neurosci. 3:1. doi: 10. 3389/neuro.07.001.2009
141. van der Meer, M. A., and Redish, A. D. (2011). Ventral striatum: a critical look at models of learning and evaluation. Curr. Opin. Neurobiol. 21, 387-392. doi: 10. 1016/j.conb.2011.02.011
142. Wadenberg, M. L., Ericson, E., Magnusson, O., and Ahlenius, S. (1990). Suppression of conditioned avoidance behavior by the local application of (-)sulpiride into the ventral, but not the dorsal, striatum of the rat. Biol. Psychiatry 28, 297-307. doi: 10.1016/0006-3223(90)90657-n
143. Wakabayashi, K. T., Fields, H. L., and Nicola, S. M. (2004). Dissociation of the role of nucleus accumbens dopamine in responding to reward-predictive cues and waiting for reward. Behav. Brain Res. 154, 19-30. doi: 10.1016/j.bbr.2004. 01.013
144. Wallis, J. D., and Miller, E. K. (2003). Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task. Eur. J. Neurosci. 18, 2069-2081. doi: 10.1046/j.1460-9568.2003.02 922.x
145. Winston, J. S., Gottfried, J. A., Kilner, J. M., and Dolan, R. J. (2005). Integrated neural representations of odor intensity and affective valence in human amygdala. J. Neurosci. 25, 8903-8907. doi: 10.1523/jneurosci.1569-05.2005
146. Yun, I. A., Wakabayashi, K. T., Fields, H. L., and Nicola, S. M. (2004). The ventral tegmental area is required for the behavioral and nucleus accumbens neuronal firing responses to incentive cues. J. Neurosci. 24, 2923-2933. doi: 10. 1523/jneurosci.5282-03.2004