Lateral habenula neurons signal errors in the prediction of reward information

Nature Neuroscience

Volumn 14, Issue 9, 2011, Pages 1209-1218

  Bromberg Martin, Ethan S ^a   Hikosaka, Okihide ^a  

^a NATIONAL EYE INSTITUTE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; DOPAMINERGIC NERVE CELL; HABENULA; INFORMATION PREDICTION ERROR; INFORMATION SEEKING; MENTAL CAPACITY; MENTAL FUNCTION; MESENCEPHALON; NERVE CELL; NERVE CELL EXCITABILITY; NERVE CELL INHIBITION; NONHUMAN; PREDICTION; PRIORITY JOURNAL; REWARD PREDICTION ERROR; TASK PERFORMANCE; VISUAL STIMULATION;

ACTION POTENTIALS; ANIMALS; CHOICE BEHAVIOR; CUES; HABENULA; MACACA MULATTA; MALE; MODELS, NEUROLOGICAL; NEURONS; PHOTIC STIMULATION; PREDICTIVE VALUE OF TESTS; PROBABILITY; REACTION TIME; REWARD; ROC CURVE; SIGNAL DETECTION, PSYCHOLOGICAL; TIME FACTORS;

EID: 80052211211     PISSN: 10976256     EISSN: 15461726     Source Type: Journal    
DOI: 10.1038/nn.2902     Document Type: Article

References (50)

1. Schultz, W., Dayan, P. & Montague, P.R. A neural substrate of prediction and reward. Science 275, 1593-1599 (1997). (Pubitemid 27120526)
2. Rescorla, R.A. & Wagner, A.R. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. in Classical Conditioning II: Current Research and Theory (eds. Black, A.H. & Prokasy, W.F.) 64-99 (Appleton Century Crofts, New York, 1972).
3. Bayer, H.M. & Glimcher, P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129-141 (2005). (Pubitemid 40922855)
4. Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366-8377 (2007). (Pubitemid 47219624)
5. Matsumoto, M., Matsumoto, K., Abe, H. & Tanaka, K. Medial prefrontal cell activity signaling prediction errors of action values. Nat. Neurosci. 10, 647-656 (2007). (Pubitemid 46652446)
6. Kim, H., Sul, J.H., Huh, N., Lee, D. & Jung, M.W. Role of striatum in updating values of chosen actions. J. Neurosci. 29, 14701-14712 (2009).
7. Oyama, K., Hernadi, I., Iijima, T. & Tsutsui, K. Reward prediction error coding in dorsal striatal neurons. J. Neurosci. 30, 11447-11457 (2010).
8. Hong, S. & Hikosaka, O. The globus pallidus sends reward-related signals to the lateral habenula. Neuron 60, 720-729 (2008).
9. Matsumoto, M. & Hikosaka, O. Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 447, 1111-1115 (2007). (Pubitemid 47014424)
10. Matsumoto, M. & Hikosaka, O. Representation of negative motivational value in the primate lateral habenula. Nat. Neurosci. 12, 77-84 (2009).
11. Holroyd, C.B. & Coles, M.G. The neural basis of human error processing: reinforcement learning, dopamine and the error-related negativity. Psychol. Rev. 109, 679-709 (2002).
12. O'Doherty, J.P., Dayan, P., Friston, K., Critchley, H. & Dolan, R.J. Temporal difference models and reward-related learning in the human brain. Neuron 38, 329-337 (2003). (Pubitemid 36555657)
13. McClure, S.M., Berns, G.S. & Montague, P.R. Temporal prediction errors in a passive learning task activate human striatum. Neuron 38, 339-346 (2003). (Pubitemid 36555658)
14. Salas, R., Baldwin, P., de Biasi, M. & Montague, P.R. BOLD responses to negative reward prediction errors in human habenula. Front. Hum. Neurosci. 4, 36 (2010).
15. Chew, S.H. & Ho, J.L. Hope: an empirical study of attitude toward the timing of uncertainty resolution. J. Risk Uncertain. 8, 267-288 (1994).
16. Eliaz, K. & Schotter, A. Experimental testing of intrinsic preferences for noninstrumental information. Am. Econ. Rev. 97, 166-169 (2007).
17. Luhmann, C.C., Chun, M.M., Yi, D.-J., Lee, D. & Wang, X.-J. Neural dissociation of delay and uncertainty in inter-temporal choice. J. Neurosci. 28, 14459-14466 (2008).
18. Prokasy, W.F. Jr. The acquisition of observing responses in the absence of differential external reinforcement. J. Comp. Physiol. Psychol. 49, 131-134 (1956).
19. Fantino, E. Conditioned reinforcement: choice and information. in Handbook of Operant Behavior (eds. Honig, W.K. & Staddon, J.E.R.) (Prentice Hall, Englewood Cliffs, New Jersey, 1977).
20. Dinsmoor, J.A. Observing and conditioned reinforcement. Behav. Brain Sci. 6, 693-728 (1983).
21. Daly, H.B. Preference for unpredictability is reversed when unpredictable nonreward is aversive: procedures, data, and theories of appetitive observing response acquisition. in Learning and Memory: The Behavioral and Biological Substrates (eds. Gormezano, I. & Wasserman, E.A.) 81-104 (L.E. Associates, 1992).
22. Bromberg-Martin, E.S. & Hikosaka, O. Midbrain dopamine neurons signal preference for advance information about upcoming rewards. Neuron 63, 119-126 (2009).
23. Ward, E. Acquisition and extinction of the observing response as a function of stimulus predictive validity. Psychon. Sci. 24, 139-141 (1971).
24. Hayden, B.Y., Heilbronner, S.R. & Platt, M.L. Ambiguity aversion in rhesus macaques. Front. Neurosci. 4, 166 (2010).
25. Kreps, D.M. & Porteus, E.L. Temporal resolution of uncertainty and dynamic choice theory. Econometrica 46, 185-200 (1978).
26. Wyckoff, L.B. Jr. The role of observing responses in discrimination learning. Psychol. Rev. 59, 431-442 (1952).
27. Hikosaka, O. The habenula: from stress evasion to value-based decision-making. Nat. Rev. Neurosci. 11, 503-513 (2010).
28. Christoph, G.R., Leonzio, R.J. & Wilcox, K.S. Stimulation of the lateral habenula inhibits dopamine-containing neurons in the substantia nigra and ventral tegmental area of the rat. J. Neurosci. 6, 613-619 (1986). (Pubitemid 16134900)
29. Ji, H. & Shepard, P.D. Lateral habenula stimulation inhibits rat midbrain dopamine neurons through a GABA(A) receptor-mediated mechanism. J. Neurosci. 27, 6923-6930 (2007). (Pubitemid 47015898)
30. Shumake, J., Ilango, A., Scheich, H., Wetzel, W. & Ohl, F.W. Differential neuromodulation of acquisition and retrieval of avoidance learning by the lateral habenula and ventral tegmental area. J. Neurosci. 30, 5876-5883 (2010).
31. Friedman, A. et al. Electrical stimulation of the lateral habenula produces an inhibitory effect on sucrose self-administration. Neuropharmacology 60, 381-387 (2010).
32. Doya, K. Metalearning and neuromodulation. Neural Netw. 15, 495-506 (2002). (Pubitemid 34947460)
33. Daw, N.D., O'Doherty, J.P., Dayan, P., Seymour, B. & Dolan, R.J. Cortical substrates for exploratory decisions in humans. Nature 441, 876-879 (2006).
34. Sutherland, R.J. & Nakajima, S. Self-stimulation of the habenular complex in the rat. J. Comp. Physiol. Psychol. 95, 781-791 (1981).
35. Lecourtier, L., Defrancesco, A. & Moghaddam, B. Differential tonic influence of lateral habenula on prefrontal cortex and nucleus accumbens dopamine release. Eur. J. Neurosci. 27, 1755-1762 (2008). (Pubitemid 351473573)
36. Ikemoto, S. Brain reward circuitry beyond the mesolimbic dopamine system: a neurobiological theory. Neurosci. Biobehav. Rev. 35, 129-150 (2010).
37. Matsumoto, M. & Hikosaka, O. Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459, 837-841 (2009).
38. Joshua, M., Adler, A. & Bergman, H. The dynamics of dopamine in control of motor behavior. Curr. Opin. Neurobiol. 19, 615-620 (2009).
39. Bromberg-Martin, E.S., Matsumoto, M. & Hikosaka, O. Dopamine in motivational control: rewarding, aversive, and alerting. Neuron 68, 815-834 (2010).
40. Caplin, A. & Leahy, J. Psychological expected utility theory and anticipatory feelings. Q. J. Econ. 116, 55-79 (2001). (Pubitemid 33593850)
41. Butko, N.J. & Movellan, J.R. Infomax control of eye movements. IEEE Trans. Auto. Ment. Dev. 2, 91-107 (2010).
42. Wyckoff, L.B. Jr. Toward a quantitative theory of secondary reinforcement. Psychol. Rev. 66, 68-78 (1959).
43. Beierholm, U.R. & Dayan, P. Pavlovian-instrumental interaction in 'observing behavior'. PLOS Comput. Biol. 6, 9 (2010).
44. Bromberg-Martin, E.S., Matsumoto, M., Nakahara, H. & Hikosaka, O. Multiple timescales of memory in lateral habenula and dopamine neurons. Neuron 67, 499-510 (2010).
45. Nakamura, K., Matsumoto, M. & Hikosaka, O. Reward-dependent modulation of neuronal activity in the primate dorsal raphe nucleus. J. Neurosci. 28, 5331-5343 (2008).
46. Ranade, S.P. & Mainen, Z.F. Transient firing of dorsal raphe neurons encodes diverse and specific sensory, motor, and reward events. J. Neurophysiol. 102, 3026-3037 (2009).
47. Redgrave, P. & Gurney, K. The short-latency dopamine signal: a role in discovering novel actions? Nat. Rev. Neurosci. 7, 967-975 (2006).
48. Morris, G., Nevet, A., Arkadir, D., Vaadia, E. & Bergman, H. Midbrain dopamine neurons encode decisions for future action. Nat. Neurosci. 9, 1057-1063 (2006). (Pubitemid 44264919)
49. Roesch, M.R., Calu, D.J. & Schoenbaum, G. Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat. Neurosci. 10, 1615-1624 (2007). (Pubitemid 350175882)
50. Houk, J.C., Adams, J.L. & Barto, A.G. A model of how the basal ganglia generate and use neural signals that predict reinforcement. in Models of Information Processing in the Basal Ganglia (eds. Houk, J.C., Davis, J.L. & Beiser, D.G.) 249-274 (MIT Press, Cambridge, Massachusetts, 1995).