Surprise! Neural correlates of Pearce-Hall and Rescorla-Wagner coexist within the brain

European Journal of Neuroscience

Volumn 35, Issue 7, 2012, Pages 1190-1200

  Roesch, Matthew R ^a   Esber, Guillem R ^b   Li, Jian ^c   Daw, Nathaniel D ^c   Schoenbaum, Geoffrey ^b,d  

^a UNIVERSITY OF MARYLAND   (United States)

^b UNIVERSITY OF MARYLAND SCHOOL OF MEDICINE   (United States)

^c NEW YORK UNIVERSITY   (United States)

^d NIDA Intramural Research Program   (United States)

Author keywords

Amygdala; Attention; Dopamine; Learning; Prediction error

Indexed keywords

DOPAMINE;

ADDICTION; ANXIETY DISORDER; ARTICLE; ATTENTION; BRAIN FUNCTION; CENTRAL NUCLEUS (AMYGDALA); CORPUS STRIATUM; DOPAMINERGIC NERVE CELL; ERROR; HABENULA; HUMAN; LEARNING; MATHEMATICAL MODEL; MESENCEPHALON; NEUROMODULATION; NONHUMAN; PEARCE HALL MODEL; PREFRONTAL CORTEX; PRIORITY JOURNAL; RESCORLA WAGNER MODEL; SCHIZOPHRENIA;

ACTION POTENTIALS; ANIMALS; BRAIN; DOPAMINERGIC NEURONS; HUMANS; LEARNING; MODELS, NEUROLOGICAL; NERVE NET;

EID: 84859339780     PISSN: 0953816X     EISSN: 14609568     Source Type: Journal    
DOI: 10.1111/j.1460-9568.2011.07986.x     Document Type: Article

References (121)

1. Ambroggi, F., Ishikawa, A., Fields, H.L. & Nicola, S.M. (2008) Basolateral amygdala neurons facilitate reward-seeking behavior by exciting nucleus accumbens neurons. Neuron, 59, 648-661.
2. Amiez, C., Joseph, J.P. & Procyk, E. (2005) Anterior cingulate error-related activity is modulated by predicted reward. Eur. J. Neurosci., 21, 3447-3452.
3. Amiez, C., Joseph, J.P. & Procyk, E. (2006) Reward encoding in the monkey anterior cingulate cortex. Cereb. Cortex, 16, 1040-1055.
4. Balleine, B.W., Daw, N.D. & O'Doherty, J.P. (2008). Multiple forms of value learning and the function of dopamine. In Glimcher, P.W., Camerer, C.F., Fehr, E. & Poldrack, R.A. (Eds), Neuroeconomics: Decision Making and the Brain. Elsevier, Amsterdam, pp. 367-388.
5. Bayer, H.M. & Glimcher, P.W. (2005) Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron, 47, 129-141.
6. Behrens, T.E., Woolrich, M.W., Walton, M.E. & Rushworth, M.F. (2007) Learning the value of information in an uncertain world. Nat. Neurosci., 10, 1214-1221.
7. Belova, M.A., Paton, J.J., Morrison, S.E. & Salzman, C.D. (2007) Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala. Neuron, 55, 970-984.
8. Belova, M.A., Patton, J.J. & Salzman, C.D. (2008) Moment-to-moment tracking of state value in the amygdala. J. Neurosci., 28, 10023-10030.
9. Bossaerts, P. & Preuschoff, K. (2007) Adding prediction risk to the theory of reward learning in the dopaminergic system. Ann. N.Y. Acad. Sci., 1104, 135-146.
10. Brinschwitz, K., Dittgen, A., Madai, V.I., Lommel, R., Geisler, S. & Veh, R.W. (2010) Glutamatergic axons from the lateral habenula mainly terminate on GABAergic neurons of the ventral midbrain. Neuroscience, 168, 463-476.
11. Bromberg-Martin, E.S., Matsumoto, M. & Hikosaka, O. (2010a) Distinct tonic and phasic anticipatory activity in lateral habenula and dopamine neurons. Neuron, 67, 144-155.
12. Bromberg-Martin, E.S., Matsumoto, M. & Hikosaka, O. (2010b) Dopamine in motivational control: rewarding, aversive and alerting. Neuron, 68, 815-834.
13. Bromberg-Martin, E.S., Matsumoto, M., Hong, S. & Hikosaka, O. (2010d) A pallidus-habenula-dopamine pathway signals inferred stimulus values. J. Neurophysiol., 104, 1068-1076.
14. Bryden, D.W., Johnson, E.E., Tobia, S.C., Kashtelyan, V. & Roesch, M.R. (2011) Attention for learning signals in anterior cingulate cortex. J. Neurosci., 31, 18266-18274.
15. Burke, K.A., Franz, T.M., Miller, D.N. & Schoenbaum, G. (2008) The role of the orbitofrontal cortex in the pursuit of happiness and more specific rewards. Nature, 454, 340-344.
16. Calu, D.J., Roesch, M.R., Haney, R.Z., Holland, P.C. & Schoenbaum, G. (2010) Neural correlates of variations in event processing during learning in central nucleus of amygdala. Neuron, 68, 991-1001.
17. Carter, C.S., Braver, T.S., Barch, D.M., Botvinick, M.M., Noll, D. & Cohen, J.D. (1998) Anterior cingulate cortex, error detection, and the online monitoring of performance. Science, 280, 747-749.
18. Cassell, M.D. & Wright, D.J. (1986) Topography of projections from the medial prefrontal cortex to the amygdala in the rat. Brain Res. Bull., 17, 321-333.
19. Chang, S.E., Wheeler, D.S., McDannald, M. & Holland, P.C. (2010) The effects of basolateral amygdala lesions on unblocking. Soc. Neurosci. Abs.
20. Corbit, L.H. & Balleine, B.W. (2005) Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of pavlovian-instrumental transfer. J. Neurosci., 25, 962-970.
21. Courville, A.C., Daw, N.D. & Touretzky, D.S. (2006) Bayesian theories of conditioning in a changing world. Trends Cogn. Sci., 10, 294-300.
22. D'Ardenne, K., McClure, S.M., Nystrom, L.E. & Cohen, J.D. (2008) BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science, 319, 1264-1267.
23. Davis, M. (2000). The role of the amygdala in conditioned and unconditioned fear and anxiety. In Aggleton, J.P. (Ed.), The Amygdala: A Functional Analysis. Oxford University Press, Oxford, pp. 213-287.
24. Davis, M. & Whalen, P.J. (2001) The amygdala: vigilance and emotion. Mol. Psychiatry, 6, 13-34.
25. Daw, N.D., Niv, Y. & Dayan, P. (2005) Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci., 8, 1704-1711.
26. Daw, N.D., Gershman, S.J., Seymour, B., Dayan, P. & Dolan, R.J. (2011) Model-based influences on humans' choices and striatal prediction errors. Neuron, 69, 1204-1215.
27. Day, J.J., Roitman, M.F., Wightman, R.M. & Carelli, R.M. (2007) Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat. Neurosci., 10, 1020-1028.
28. Dayan, P. & Long, T. (1998) Statistical models of conditioning. Neural Information Processing Systems, 10, 117-123.
29. Dickinson, A. & Balleine, B.W. (2002). The role of learning in the operation of motivational systems. In Pashler, H. & Gallistel, R. (Eds), Stevens' Handbook of Experimental Psychology (3rd Edition, Vol 3: Learning, motivation, and emotion). John Wiley & Sons, New York, NY, pp. 497-533.
30. Dziewiatkowski, J., Spodnik, J.H., Biranowska, J., Kowianski, P., Majak, K. & Morys, J. (1998) The projection of the amygdaloid nuclei to various areas of the limbic cortex in the rat. Folia Morphol. (Warsz), 57, 301-308.
31. Ehrlich, I., Humeau, Y., Grenier, F., Ciocchi, S., Herry, C. & Luthi, A. (2009) Amygdala inhibitory circuits and the control of fear memory. Neuron, 62, 757-771.
32. Esber, G.R. & Haselgrove, M. (2011) Reconciling the influence of predictiveness and uncertainty on stimulus salience: a model of attention in associative learning. Proc. Biol. Sci., 278, 2553-2561.
33. Fiorillo, C.D., Tobler, P.N. & Schultz, W. (2003) Discrete coding of reward probability and uncertainty by dopamine neurons. Science, 299, 1856-1902.
34. Fiorillo, C.D., Newsome, W.T. & Schultz, W. (2008) The temporal precision of reward prediction in dopamine neurons. Nat. Neurosci., 11, 966-973.
35. Gallagher, M. (2000). The amygdala and associative learning. In Aggleton, J.P. (Ed.), The Amygdala: A Functional Analysis. Oxford University Press, Oxford, pp. 311-330.
36. Gallagher, M., Graham, P.W. & Holland, P.C. (1990) The amygdala central nucleus and appetitive Pavlovian conditioning: lesions impair one class of conditioned behavior. J. Neurosci., 10, 1906-1911.
37. Hare, T.A., O'Doherty, J., Camerer, C.F., Schultz, W. & Rangel, A. (2008) Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors. J. Neurosci., 28, 5623-5630.
38. Hayden, B.Y., Heilbronner, S.R., Pearson, J.M. & Platt, M.L. (2011) Surprise signals in anterior cingulate cortex: neuronal encoding of unsigned reward prediction errors driving adjustment in behavior. J. Neurosci., 31, 4178-4187.
39. Hikosaka, O. (2010) The habenula: from stress evasion to value-based decision-making. Nat. Rev. Neurosci., 11, 503-513.
40. Hillman, K.L. & Bilkey, D.K. (2010) Neurons in the rat anterior cingulate cortex dynamically encode cost-benefit in a spatial decision-making task. J. Neurosci., 30, 7705-7713.
41. Holland, P.C. & Gallagher, M. (1993a) Amygdala central nucleus lesions disrupt increments, but not decrements, in conditioned stimulus processing. Behav. Neurosci., 107, 246-253.
42. Holland, P.C. & Gallagher, M. (1993b) Effects of amygdala central nucleus lesions on blocking an d unblocking. Behav. Neurosci., 107, 235-245.
43. Holland, P.C. & Gallagher, M. (1999) Amygdala circuitry in attentional and representational processes. Trends Cogn. Sci., 3, 65-73.
44. Holland, P.C. & Kenmuir, C. (2005) Variations in unconditioned stimulus processing in unblocking. J. Exp. Psychol. Anim. Behav. Process., 31, 155-171.
45. Hollerman, J.R. & Schultz, W. (1998) Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci., 1, 304-309.
46. Holroyd, C.B. & Coles, M.G. (2002) The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol. Rev., 109, 679-709.
47. Hong, S. & Hikosaka, O. (2008) The globus pallidus sends reward-related signals to the lateral habenula. Neuron, 60, 720-729.
48. Hong, S., Jhou, T.C., Smith, M., Saleem, K.S. & Hikosaka, O. (2011) Negative reward signals from the lateral habenula to dopamine neurons are mediated by rostromedial tegmental nucleus in primates. J. Neurosci., 31, 11457-11471.
49. Houk, J.C., Adams, J.L. & Barto, A.G. (1995). A model of how the basal ganglia generate and use neural signals that predict reinforcement. In Houk, J.C., Davis, J.L. & Beiser, D.G. (Eds), Models of Information Processing in the Basal Ganglia. MIT Press, Cambridge, pp. 249-270.
50. Ito, S., Stuphorn, V., Brown, J.W. & Schall, J.D. (2003) Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science, 302, 120-122.
51. Izquierdo, A.D., Suda, R.K. & Murray, E.A. (2004) Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. J. Neurosci., 24, 7540-7548.
52. Jhou, T.C., Fields, H.L., Baxter, M.G., Saper, C.B. & Holland, P.C. (2009) The rostromedial tegmental nucleus (RMTg), a GABAergic afferent to midbrain dopamine neurons, encodes aversive stimuli and inhibits motor responses. Neuron, 61, 786-800.
53. Ji, H. & Shepard, P.D. (2007) Lateral habenula stimulation inhibits rat midbrain dopamine neurons through a GABA(A) receptor-mediated mechanism. J. Neurosci., 27, 6923-6930.
54. Johansen, J.P., Tarpley, J.W., LeDoux, J.E. & Blair, H.T. (2010) Neural substrates for expectation-modulated fear learning in the amygdala and periaqueductal gray. Nat. Neurosci., 13, 979-986.
55. Kakade, S. & Dayan, P. (2002) Acquisition and extinction in autoshaping. Psychol. Rev., 109, 533-544.
56. Kamin, L.J. (1969). Predictability, suprise, attention, and conditioning. In Campbell, B.A. & Church, R.M. (Eds), Punishment and Aversive Behavior. Appleton-Century-Crofts, New York, NY, pp. 242-259.
57. Kapur, S. (2003) Psychosis as a state of aberrant salience: a framework linking biology, phenomenology, and pharmacology in schizophrenia. Am. J. Psychiatry, 160, 13-23.
58. Kaufling, J., Veinante, P., Pawlowski, S.A., Freund-Mercier, M.J. & Barrot, M. (2009) Afferents to the GABAergic tail of the ventral tegmental area in the rat. J. Comp. Neurol., 513, 597-621.
59. Kennerley, S.W. & Wallis, J.D. (2009) Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables. Eur. J. Neurosci., 29, 2061-2073.
60. Kennerley, S.W., Walton, M.E., Behrens, T.E., Buckley, M.J. & Rushworth, M.F. (2006) Optimal decision making and the anterior cingulate cortex. Nat. Neurosci., 9, 940-947.
61. Kennerley, S.W., Dahmubed, A.F., Lara, A.H. & Wallis, J.D. (2009) Neurons in the frontal lobe encode the value of multiple decision variables. J. Cogn. Neurosci., 21, 1162-1178.
62. Knutson, B. & Gibbs, S.E.B. (2007) Linking nucleus accumbens dopamine and blood oxygenation. Psychopharmacology, 191, 813-822.
63. Kobayashi, S. & Schultz, W. (2008) Influence of reward delays on responses of dopamine neurons. J. Neurosci., 28, 7837-7846.
64. Lammel, S., Hetzel, A., Hckel, O., Jones, I., Liss, B. & Roeper, J. (2008) Unique properties of mesoprefrontal neurons within a dual mesocorticolimbic dopamine system. Neuron, 57, 760-773.
65. LeDoux, J.E. (2000). The amygdala and emotion: a view through fear. In Aggleton, J.P. (Ed.), The Amygdala: A Functional Analysis. Oxford University Press, New York, NY, pp. 289-310.
66. Lepelley, M.E. (2004) The role of associative history in models of associative learning: a selective review and a hybrid model. Q. J. Exp. Psychol., 57, 193-243.
67. Li, J., Schiller, D., Schoenbaum, G., Phelps, E.A. & Daw, N.D. (2011) Differential roles of human striatum and amygdala in associative learning. Nat. Neurosci., 14, 1250-1252.
68. Mackintosh, N.J. (1975) A theory of attention: variations in the associability of stimuli with reinforcement. Psychol. Rev., 82, 276-298.
69. Magno, E., Foxe, J.J., Molholm, S., Robertson, I.H. & Garavan, H. (2006) The anterior cingulate and error avoidance. J. Neurosci., 26, 4769-4773.
70. Margolis, E.B., Lock, H., Hjelmstad, G.O. & Fields, H.L. (2006) The ventral tegmental area revisited: is there an electrophysiological marker for dopaminergic neurons? J. Physiol., 577, 907-924.
71. Margolis, E.B., Mitchell, J.M., Ishikawa, Y., Hjelmstad, G.O. & Fields, H.L. (2008) Midbrain dopamine neurons: projection target determines action potential duration and dopamine D(2) receptor inhibition. J. Neurosci., 28, 8908-8913.
72. Matsumoto, M. & Hikosaka, O. (2007) Lateral habenula as a source of negative reward signals in dopamine neurons. Nature, 447, 1111-1115.
73. Matsumoto, M. & Hikosaka, O. (2009) Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature, 459, 837-841.
74. Matsumoto, M., Matsumoto, K., Abe, H. & Tanaka, K. (2007) Medial prefrontal cell activity signaling prediction errors of action values. Nat. Neurosci., 10, 647-656.
75. McClure, S.M., Berns, G.S. & Montague, P.R. (2003) Temporal prediction errors in a passive learning task activate human striatum. Neuron, 38, 339-346.
76. McLaughlin, R.J. & Floresco, S.B. (2007) The role of different subregions of the basolateral amygdala in cue-induced reinstatement and extinction of food-seeking behavior. Neuroscience, 146, 1484-1494.
77. Mileykovskiy, B. & Morales, M. (2011) Duration of inhibition of ventral tegmental area dopamine neurons encodes a level of conditioned fear. J. Neurosci., 31, 7471-7476.
78. Mirenowicz, J. & Schultz, W. (1994) Importance of unpredictability for reward responses in primate dopamine neurons. J. Neurophysiol., 72, 1024-1027.
79. Montague, P.R., Dayan, P. & Sejnowski, T.J. (1996) A framework for mesencephalic dopamine systems based on predictive hebbian learning. J. Neurosci., 16, 1936-1947.
80. Morris, G., Nevet, A., Arkadir, D., Vaadia, E. & Bergman, H. (2006) Midbrain dopamine neurons encode decisions for future action. Nat. Neurosci., 9, 1057-1063.
81. Murray, E.A. (2007) The amygdala, reward and emotion. Trends Cogn. Sci., 11, 489-497.
82. O'Doherty, J., Dayan, P., Friston, K.J., Critchley, H. & Dolan, R.J. (2003) Temporal difference learning model accounts for responses in human ventral striatum and orbitofrontal cortex during Pavlovian appetitive learning. Neuron, 38, 329-337.
83. O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K.J. & Dolan, R.J. (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science, 304, 452-454.
84. Oliveira, F.T., McDonald, J.J. & Goodman, D. (2007) Performance monitoring in the anterior cingulate is not all error related: expectancy deviation and the representation of action-outcome associations. J. Cogn. Neurosci., 19, 1994-2004.
85. Omelchenko, N., Bell, R. & Sesack, S.R. (2009) Lateral habenula projections to dopamine and GABA neurons in the rat ventral tegmental area. Eur. J. Neurosci., 30, 1239-1250.
86. Ostlund, S.B. & Balleine, B.W. (2007) Orbitofrontal cortex mediates outcome encoding in Pavlovian but not instrumental learning. J. Neurosci., 27, 4819-4825.
87. Pan, W.-X., Schmidt, R., Wickens, J.R. & Hyland, B.I. (2005) Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J. Neurosci., 25, 6235-6242.
88. Paus, T. (2001) Primate anterior cingulate cortex: where motor control, drive and cognition interface. Nat. Rev. Neurosci., 2, 417-424.
89. Pearce, J.M. & Hall, G. (1980) A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review, 87, 532-552.
90. Pearce, J.M. & Mackintosh, N.J. (2010). Two theories of attention: a review and a possible integration. In Mitchell, C.J. & LePelley, M.E. (Eds), Attention and Associative Learning: From Brain to Behaviour. Oxford University Press, Oxford, UK, pp. 11-39.
91. Pearce, J.M., Kaye, H. & Hall, G. (1982). Predictive accuracy and stimulus associability: development of a model for Pavlovian learning. In Commons, M.L., Herrnstein, R.J. & Wagner, A.R. (Eds), Quantitative Analyses of Behavior. Ballinger, Cambridge, MA, pp. 241-255.
92. Pessiglione, M., Seymour, P., Flandin, G., Dolan, R.J. & Frith, C.D. (2006) Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature, 442, 1042-1045.
93. Pickens, C.L., Setlow, B., Saddoris, M.P., Gallagher, M., Holland, P.C. & Schoenbaum, G. (2003) Different roles for orbitofrontal cortex and basolateral amygdala in a reinforcer devaluation task. J. Neurosci., 23, 11078-11084.
94. Quilodran, R., Rothe, M. & Procyk, E. (2008) Behavioral shifts and action valuation in the anterior cingulate cortex. Neuron, 57, 314-325.
95. Redgrave, P. & Gurney, K. (2006) The short-latency dopamine signal: a role in discovering novel actions? Nat. Rev. Neurosci., 7, 967-975.
96. Rescorla, R.A. & Wagner, A.R. (1972). A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In Black, A.H. & Prokasy, W.F. (Eds), Classical Conditioning II: Current Research and Theory. Appleton-Century-Crofts, New York, NY, pp. 64-99.
97. Roesch, M.R., Calu, D.J. & Schoenbaum, G. (2007) Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat. Neurosci., 10, 1615-1624.
98. Roesch, M.R., Calu, D.J., Esber, G.R. & Schoenbaum, G. (2010) Neural correlates of variations in event processing during learning in basolateral amygdala. J. Neurosci., 30, 2464-2471.
99. Rothe, M., Quilodran, R., Sallet, J. & Procyk, E. (2011) Coordination of high gamma activity in anterior cingulate and lateral prefrontal cortical areas during adaptation. J. Neurosci., 31, 11110-11117.
100. Rudebeck, P.H., Bannerman, D.M. & Rushworth, M.F. (2008) The contribution of distinct subregions of the ventromedial frontal cortex to emotion, social behavior, and decision making. Cogn. Affect. Behav. Neurosci., 8, 485-497.
101. Rushworth, M.F. & Behrens, T.E. (2008) Choice, uncertainty and value in prefrontal and cingulate cortex. Nat. Neurosci., 11, 389-397.
102. Rushworth, M.F., Walton, M.E., Kennerley, S.W. & Bannerman, D.M. (2004) Action sets and decisions in the medial frontal cortex. Trends Cogn. Sci., 8, 410-417.
103. Rushworth, M.F., Behrens, T.E., Rudebeck, P.H. & Walton, M.E. (2007) Contrasting roles for cingulate and orbitofrontal cortex in decisions and social behaviour. Trends Cogn. Sci., 11, 168-176.
104. Sallet, J., Quilodran, R., Rothe, M., Vezoli, J., Joseph, J.P. & Procyk, E. (2007) Expectations, gains, and losses in the anterior cingulate cortex. Cogn. Affect. Behav. Neurosci., 7, 327-336.
105. Scheffers, M.K. & Coles, M.G. (2000) Performance monitoring in a confusing world: error-related brain activity, judgments of response accuracy, and types of errors. J. Exp. Psychol. Hum. Percept. Perform., 26, 141-151.
106. Schoenbaum, G., Setlow, B., Saddoris, M.P. & Gallagher, M. (2003) Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala. Neuron, 39, 855-867.
107. Schultz, W., Dayan, P. & Montague, P.R. (1997) A neural substrate for prediction and reward. Science, 275, 1593-1599.
108. Simon, D.A. & Daw, N.D. (in press) Neural correlates of forward planning in a spatial decision task in humans. J. Neurosci., 31, 5526-5539.
109. Sripanidkulchai, K., Sripanidkulchai, B. & Wyss, J.M. (1984) The cortical projection of the basolateral amygdaloid nucleus in the rat: a retrograde fluorescent dye study. J. Comp. Neurol., 229, 419-431.
110. Sutton, R.S. (1988) Learning to predict by the method of temporal difference. Machine Learning, 3, 9-44.
111. Sutton, R.S. (1992) Adapting bias by gradient descent: an incremental version of delta-bar-delta. Proceedings of the Tenth National Conference on Artificial Intelligence, pp. 171-176.
112. Takahashi, Y.K., Roesch, M.R., Wilson, R.C., Toreson, K., O'Donnell, P., Niv, Y. & Schoenbaum, G. (2011) Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex. Nat. Neurosci., 14, 1590-1597.
113. Taylor, S.F., Phan, K.L., Britton, J.C. & Liberzon, I. (2005) Neural responses to emotional salience in schizophrenia. Neuropsychopharmacology, 30, 984-995.
114. Tobler, P.N., Dickinson, A. & Schultz, W. (2003) Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. J. Neurosci., 23, 10402-10410.
115. Totah, N.K., Kim, Y.B., Homayoun, H. & Moghaddam, B. (2009) Anterior cingulate neurons represent errors and preparatory attention within the same behavioral sequence. J. Neurosci., 29, 6418-6426.
116. Tye, K.M., Stuber, G.D., De Ridder, B., Bonci, A. & Janak, P.H. (2008) Rapid strengthening of thalamo-amygdala synapses mediates cue-reward learning. Nature, 453, 1253-1257.
117. Tye, K.M., Cone, J.J., Schairer, W.W. & Janak, P.H. (2010) Amygdala neural encoding of the absence of reward during extinction. J. Neurosci., 30, 116-125.
118. Ungless, M.A., Magill, P.J. & Bolam, J.P. (2004) Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science, 303, 2040-2042.
119. Waelti, P., Dickinson, A. & Schultz, W. (2001) Dopamine responses comply with basic assumptions of formal learning theory. Nature, 412, 43-48.
120. Wallis, J.D. & Kennerley, S.W. (2010) Heterogeneous reward signals in prefrontal cortex. Curr. Opin. Neurobiol., 20, 191-198.
121. Walton, M.E., Devlin, J.T. & Rushworth, M.F. (2004) Interactions between decision making and performance monitoring within prefrontal cortex. Nat. Neurosci., 7, 1259-1265.