Flexible neural representations of value in the primate brain

Annals of the New York Academy of Sciences

Volumn 1121, Issue , 2007, Pages 336-354

  Salzman, C Daniel ^a   Paton, Joseph J ^a   Belova, Marina A ^a   Morrison, Sara E ^a  

^a NEW YORK STATE PSYCHIATRIC INSTITUTE   (United States)

Author keywords

Amygdala; Aversive; Conditioning; Learning; Monkey; OFC; Orbitofrontal cortex; Reinforcement learning; Reward; Value

Indexed keywords

AMYGDALOID NUCLEUS; APPETITE; ASSOCIATION; AVERSIVE BEHAVIOR; BEHAVIOR; BRAIN NERVE CELL; CELL POPULATION; CONFERENCE PAPER; EMOTION; INTROSPECTION; LEARNING; NONHUMAN; ORBITAL CORTEX; PHYSIOLOGY; PRIMATE; REINFORCEMENT; REWARD; SENSORY STIMULATION; VISUAL STIMULATION;

PRIMATES;

EID: 37549052420     PISSN: 00778923     EISSN: 17496632     Source Type: Book Series    
DOI: 10.1196/annals.1401.034     Document Type: Conference Paper

References (73)

1. OCHSNER, K.N. & J.J. GROSS. 2005. The cognitive control of emotion. Trends Cogn. Sci. 9: 242-249.
2. MURRAY, E.A. 2008. Neuropsychology of primate reward processes. In New Encyclopedia of Neuroscience. L.R. Squire Ed. Elsevier. In press.
3. PENNARTZ, C.M. 1997. Reinforcement learning by Hebbian synapses with adaptive thresholds. Neuroscience 81: 303-319.
4. RESCORLA, R.A. & A.R.WAGNER. 1972. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and non-reinforcement. In Classical Conditioning II: Current Research and Theory. A.H. Black & W.F. Prokasy Eds.: 64-99. Appleton Century Crofts. New York.
5. SUTTON, R. & A. BARTO. 1998. Reinforcement Learning. MIT Press. Cambridge, MA.
6. SCHULTZ, W., P. DAYAN & P.R. MONTAGUE. 1997. A neural substrate of prediction and reward. Science 275: 1593-1599.
7. PEARCE, J. & G. HALL. 1980. A model for Pavlovian conditioning: variations in the effectiveness of conditioned but not unconditioned stimuli. Psychol. Rev. 87: 532-552.
8. LEDOUX, J.E. 2000. Emotion circuits in the brain. Annu. Rev. Neurosci. 23: 155-184.
9. EVERITT, B.J., R.N. CARDINAL, J.A. PARKINSON & T.W. ROBBINS. 2003. Appetitive behavior: impact of amygdala-dependent mechanisms of emotional learning. Ann. N. Y. Acad. Sci. 985: 233-250.
10. BAXTER, M. & MURRAY, EA. 2002. The amygdala and reward. Nat. Rev. Neurosci. 3: 563-573.
11. AMARAL, D., J. PRICE, A. PITKANEN & S. CARMICHAEL. 1992. Anatomical organization of the primate amygdaloid complex. In The Amygdala: Neurobiological Aspects of Emotion, Memory, and Mental Dysfunction. J. Aggleton Ed.: 1-66. Wiley-Liss. New York.
12. SWANSON, L.W. & G.D. PETROVICH. 1998. What is the amygdala? Trends Neurosci. 21: 323-331.
13. GHASHGHAEI, H.T. & H. BARBAS. 2002. Pathways for emotion: interactions of prefrontal and anterior temporal pathways in the amygdala of the rhesus monkey. Neuroscience 115: 1261-1279.
14. STEFANACCI, L. & D.G. AMARAL. 2002. Some observations on cortical inputs to the macaque monkey amygdala: an anterograde tracing study. J. Comp. Neurol. 451: 301-323.
15. REPA, J.C. et al. 2001. Two different lateral amygdala cell populations contribute to the initiation and storage of memory. Nat. Neurosci. 4: 724-731.
16. SCHOENBAUM, G., A.A. CHIBA & M. GALLAGHER. 1999. Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning. J. Neurosci. 19: 1876-1884.
17. PASCOE, J.P. & B.S. KAPP. 1985. Electrophysiological characteristics of amygdaloid central nucleus neurons during Pavlovian fear conditioning in the rabbit. Behav. Brain Res. 16: 117-133.
18. QUIRK, G.J., C. REPA & LEDOUX, J.E. 1995. Fear conditioning enhances shortlatency auditory responses of lateral amygdala neurons: parallel recordings in the freely behaving rat. Neuron 15: 1029-1039.
19. APPLEGATE, C.D., R.C. FRYSINGER, B.S. KAPP & M. GALLAGHER. 1982. Multiple unit activity recorded from amygdala central nucleus during Pavlovian heart rate conditioning in rabbit. Brain Res. 238: 457-462.
20. GOOSENS, K.A., J.A. HOBIN & S. MAREN. 2003. Auditory-evoked spike firing in the lateral amygdala and Pavlovian fear conditioning: mnemonic code or fear bias? Neuron 40: 1013-1022.
21. COLLINS, D.R. & D. PARE. 2000. Differential fear conditioning induces reciprocal changes in the sensory responses of lateral amygdala neurons to the CS(+) and CS(-). Learn. Mem. 7: 97-103.
22. MAREN, S. & G.J. QUIRK. 2004. Neuronal signaling of fear memory. Nat. Rev. Neurosci. 5: 844-852.
23. SCHOENBAUM, G., A.A. CHIBA & M. GALLAGHER. 1998. Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nat. Neurosci. 1: 155-159.
24. MAREN, S. 2000. Auditory fear conditioning increases CS-elicited spike firing in lateral amygdala neurons even after extensive overtraining. Eur. J. Neurosci. 12: 4047-4054.
25. RORICK-KEHN, L.M. & J.E. STEINMETZ. 2005. Amygdalar unit activity during three learning tasks: eyeblink classical conditioning, Pavlovian fear conditioning, and signaled avoidance conditioning. Behav. Neurosci. 119: 1254-1276.
26. SUGASE-MIYAMOTO, Y. & B.J. RICHMOND. 2005. Neuronal signals in the monkey basolateral amygdala during reward schedules. J. Neurosci. 25: 11071-11083.
27. DEHAENE, S. & J.P. CHANGEUX. 2000. Reward-dependent learning in neuronal networks for planning and decision making. Prog. Brain Res. 126: 217-229.
28. SANGHERA, M.K., E.T. ROLLS & A. ROPER-HALL. 1979. Visual responses of neurons in the dorsolateral amygdala of the alert monkey. Exp. Neurol. 63: 610-626.
29. NISHIJO, H., T. ONO & H. NISHINO. 1988. Single neuron responses in amygdala of alert monkey during complex sensory stimulation with affective significance. J. Neurosci. 8: 3570-3583.
30. NAKAMURA, K., A. MIKAMI & K. KUBOTA. 1992. Activity of single neurons in the monkey amygdala during performance of a visual discrimination task. J. Neurophysiol. 67: 1447-1663.
31. WILSON, F.A. & E.T. ROLLS. 2005. The primate amygdala and reinforcement: a dissociation between rule-based and associatively-mediated memory revealed in neuronal activity. Neuroscience 133: 1061-1072.
32. ROLLS, E. 1992. Neurophysiology and functions of the primate amygdala. In The Amygdala: Neurobiological Aspects of Emotion, Memory, and Mental Dysfunction. J. Aggleton, Ed.: 143-166. Wiley-Liss. New York.
33. PATON, J., M. BELOVA, S. MORRISON & C. SALZMAN. 2006. The primate amygdala represents the positive and negative value of visual stimuli during learning. Nature 439: 865-870.
34. HABER, S.N., K. KUNISHIO, M. MIZOBUCHI & E. LYND-BALTA. 1995. The orbital and medial prefrontal circuit through the primate basal ganglia. J. Neurosci. 15: 4851-4867.
35. CAVADA, C., T. COMPANY, J. TEJEDOR, et al. 2000. The anatomical connections of the macaque monkey orbitofrontal cortex: a review. Cereb. Cortex 10: 220-242.
36. CARMICHAEL, S.T. & J.L. PRICE. 1995. Limbic connections of the orbital and medial prefrontal cortex in macaque monkeys. J. Comp. Neurol. 363: 615-641.
37. STEFANACCI, L. & D.G. AMARAL. 2000. Topographic organization of cortical inputs to the lateral nucleus of the macaque monkey amygdala: a retrograde tracing study. J. Comp. Neurol. 421: 52-79.
38. BECHARA, A., H. DAMASIO & A.R. DAMASIO. 2000. Emotion, decision making and the orbitofrontal cortex. Cereb. Cortex 10: 295-307.
39. BECHARA, A., H. DAMASIO & A.R. DAMASIO. 2003. Role of the amygdala in decision-making. Ann. N.Y. Acad. Sci. 985: 356-369.
40. BECHARA, A., D. TRANEL, H. DAMASIO & A.R. DAMASIO. 1996. Failure to respond autonomically to anticipated future outcomes following damage to prefrontal cortex. Cereb. Cortex 6: 215-225.
41. CHOW, T.W. 2000. Personality in frontal lobe disorders. Curr. Psychiatry Rep. 2: 446-451.
42. IZQUIERDO, A., R.K. SUDA & E.A. MURRAY. 2004. Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. J. Neurosci. 24: 7540-7548.
43. BAXTER, M., A. PARKER, C.C. LINDNER, et al. 2000. Control of response selection by reinforcer value requires interaction of amygdala and orbital prefrontal cortex. J. Neurosci. 20: 4311-4319.
44. COX, S.M., A. ANDRADE & I.S. JOHNSRUDE. 2005. Learning to like: a role for human orbitofrontal cortex in conditioned reward. J. Neurosci. 25: 2733-2740.
45. URSU, S. & C.S. CARTER. 2005. Outcome representations, counterfactual comparisons and the human orbitofrontal cortex: implications for neuroimaging studies of decision-making. Cogn. Brain Res. 23: 51-60.
46. O'DOHERTY, J., M.L. KRINGELBACH, E.T. ROLLS, et al. 2001. Abstract reward and punishment representations in the human orbitofrontal cortex. Nature Neurosci. 4: 95-102.
47. THORPE, S.J., E.T. ROLLS & S. MADDISON. 1983. The orbitofrontal cortex: neuronal activity in the behaving monkey. Exp. Brain Res. 49: 93-115.
48. SCHOENBAUM, G., A. CHIBA & M. GALLAGHER. 1998. Orbitalfrontal cortex and basolateral amygdala encode expected outcomes during learning. Nature Neurosci. 1: 155-159.
49. SCHOENBAUM, G., A.A. CHIBA & M. GALLAGHER. 1999. Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning. J. Neurosci. 19: 1876-1884.
50. SCHOENBAUM, G., B. SETLOW, M.P. SADDORIS & M. GALLAGHER. 2003. Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala. Neuron 39: 855-867.
51. ROLLS, E.T., H.D. CRITCHLEY, R. MASON & E.A. WAKEMAN. 1996. Orbitofrontal cortex neurons: role in olfactory and visual association learning. J. Neurophysiol. 75: 1970-1981.
52. TREMBLAY, L. & W. SCHULTZ. 1999. Relative reward preference in primate orbitofrontal cortex. Nature 398: 704-708.
53. WALLIS, J.D. & E.K. MILLER. 2003. Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task. Eur. J. Neurosci. 18: 2069-2081.
54. TREMBLAY, L. & W. SCHULTZ. 2000. Modifications of reward expectation-related neuronal activity during learning in primate orbitofrontal cortex. J. Neurophysiol. 83: 1877-1885.
55. ROESCH, M.R. & C.R. OLSON. 2004. Neuronal activity related to reward value and motivation in primate frontal cortex. Science 304: 307-310.
56. PADOA-SCHIOPPA, C. & J.A. ASSAD. 2006. Neurons in the orbitofrontal cortex encode economic value. Nature 441: 223-226.
57. FEIERSTEIN, C.E., M.C. QUIRK, N. UCHIDA, et al. 2006. Representation of spatial goals in rat orbitofrontal cortex. Neuron 51: 495-507.
58. ROESCH, M.R., A.R. TAYLOR & G. SCHOENBAUM. 2006. Encoding of timediscounted rewards in orbitofrontal cortex is independent of value representation. Neuron 51: 509-520.
59. OSTLUND, S.B. & B.W. BALLEINE. 2007. Orbitofrontal cortex mediates outcome encoding in Pavlovian but not instrumental conditioning. J. Neurosci. 27: 4819-4825.
60. MAZUR, J.E. 2006. Learning and Behavior. Prentice Hall. Upper Saddle River, NJ.
61. PAVLOV, I.P. 1927. Conditioned Reflexes. Oxford University Press. London.
62. LANG, P.J., M.M. BRADLEY & B.N. CUTHBERT. 1990. Emotion, attention, and the startle reflex. Psychol. Rev. 97: 377-395.
63. GALLISTEL, C.R., S. FAIRHURST & P. BALSAM. 2004. The learning curve: implications of a quantitative analysis. Proc. Natl. Acad. Sci. USA 101: 13124-13131.
64. BELOVA, M.A., J.J. PATON, S.E. MORRISON & C.D. SALZMAN. 2007. Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala. Neuron. 55(6): 970-984.
65. SADDORIS, M.P., M. GALLAGHER & G. SCHOENBAUM. 2005. Rapid associative encoding in basolateral amygdala depends on connections with orbitofrontal cortex. Neuron 46: 321-331.
66. GREEN, D.M. & J.A. SWETS. 1966. Signal Detection Theory and Psychophysics. Wiley. New York.
67. MORRISON, S.E. & C.D. SALZMAN. 2007. Primate orbitofrontal cortex andamygdala encode stimulus value with a similar time course during reinforcement learning. Soc. Neurosci. Abstr. 934.4.
68. ROLLS, E. 2000. Neurophysiology and functions of the primate amygdala, and the neural basis of emotion. In The Amygdala: a Functional Analysis. J. Aggleton, Ed.: 447-478. Oxford University Press. New York.
69. FELLOWS, L.K. & M.J. FARAH. 2003. Ventromedial frontal cortex mediates affective shifting in humans: evidence from a reversal learning paradigm. Brain 126: 1830-1837.
70. IZQUIERDO, A. & E.A. MURRAY. 2007. Selective bilateral amygdala lesions in rhesus monkeys fail to disrupt object reversal learning. J. Neurosci. 27: 1054-1062.
71. SCHOENBAUM, G., B. SETLOW, S.L. NUGENT, M.P. SADDORIS & M. GALLAGHER. 2003. Lesions of orbitofrontal cortex and basolateral amygdala complex disrupt acquisition of odor-guided discriminations and reversals. Learn. Mem. 10: 129-140.
72. SWINEHART, C.D. & L.F. ABBOTT. 2005. Supervised learning through neuronal response modulation. Neural Comput. 17: 609-631.
73. SWINEHART, C.D., K. BOUCHARD, P. PARTENSKY & L.F. ABBOTT. 2004. Control of network activity through neuronal response modulation. Neurocomputing 58-60: 327-335.