Arithmetic and local circuitry underlying dopamine prediction errors

Nature

Volumn 525, Issue 7568, 2015, Pages 243-246

  Eshel, Neir ^a   Bukwich, Michael ^a   Rao, Vinod ^a   Hemmelder, Vivian ^a   Tian, Ju ^a   Uchida, Naoshige ^a  

^a HARVARD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

4 AMINOBUTYRIC ACID; DOPAMINE; 4 AMINOBUTYRIC ACID RECEPTOR;

BRAIN; ERROR ANALYSIS; INHIBITION; LEARNING; NEUROLOGY; NUMERICAL METHOD; ORGANIC COMPOUND; PHYSIOLOGICAL RESPONSE; PREDICTION; RODENT;

ADULT; ANIMAL EXPERIMENT; ARITHMETIC; ARTICLE; BRAIN; BRAIN CORTEX; CONNECTOME; CONTROLLED STUDY; DOPAMINERGIC NERVE CELL; GABAERGIC SYSTEM; MALE; MEASUREMENT ERROR; MOUSE; NERVE CELL; NONHUMAN; ODOR; PREDICTION; PREDICTION ERROR CALCULATION; PRIORITY JOURNAL; REINFORCEMENT; REWARD; STATE DEPENDENT LEARNING; TEMPORAL LOBE; VENTRAL TEGMENTUM; ANALYSIS; ANIMAL; BIOLOGICAL MODEL; C57BL MOUSE; CONDITIONED REFLEX; CYTOLOGY; METABOLISM; NERVE TRACT; OPTOGENETICS; PHYSIOLOGY; TIME FACTOR;

MUS;

ANIMALS; CONDITIONING, CLASSICAL; DOPAMINE; DOPAMINERGIC NEURONS; GABAERGIC NEURONS; GAMMA-AMINOBUTYRIC ACID; MALE; MICE; MICE, INBRED C57BL; MODELS, NEUROLOGICAL; NEURAL PATHWAYS; ODORS; OPTOGENETICS; REINFORCEMENT (PSYCHOLOGY); REWARD; TIME FACTORS; VENTRAL TEGMENTAL AREA;

EID: 84941212802     PISSN: 00280836     EISSN: 14764687     Source Type: Journal    
DOI: 10.1038/nature14855     Document Type: Article

References (39)

1. Schultz, W., Dayan, P., Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593-1599 (1997).
2. Bayer, H.M., Glimcher, P.W.Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129-141 (2005).
3. Bush, R. R., Mosteller, F. A mathematical model for simple learning. Psychol. Rev. 58, 313-323 (1951).
4. Rescorla, R. A., Wagner, A. R. in Classical Conditioning II: Current Research and Theory (eds Black, A., Prokasy, W.) 64-99 (Appleton-Century-Crofts, 1972).
5. Carandini, M., Heeger, D. J. Normalization as a canonical neural computation. Nature Rev. Neurosci. 13, 51-62 (2012).
6. Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., Uchida, N. Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482, 85-88 (2012).
7. Tobler, P. N., Fiorillo, C. D., Schultz, W. Adaptive coding of reward value by dopamine neurons. Science 307, 1642-1645 (2005).
8. Silver, R. A. Neuronal arithmetic. Nature Rev. Neurosci. 11, 474-489 (2010).
9. Houk, J.C., Adams, J. L.Barto, A.G. inModels of Information Processing in the Basal Ganglia (eds Houk, J. C., Davis, J. L., Beiser, D. G.) 249-270 (MIT Press, 1995).
10. Kawato, M., Samejima, K. Efficient reinforcement learning: computational theories, neuroscience and robotics. Curr. Opin. Neurobiol. 17, 205-212 (2007).
11. Matsumoto, M., Hikosaka, O. Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 447, 1111-1115 (2007).
12. Hong, S., Jhou, T. C., Smith,M., Saleem,K. S.Hikosaka,O.Negative reward signals from the lateral habenula to dopamine neurons are mediated by rostromedial tegmental nucleus in primates. J. Neurosci. 31, 11457-11471 (2011).
13. Omelchenko, N., Sesack, S. R. Ultrastructural analysis of local collaterals of rat ventral tegmental area neurons: GABA phenotype and synapses onto dopamine and GABA cells. Synapse 63, 895-906 (2009).
14. Van Zessen, R., Phillips, J. L., Budygin, E. A., Stuber, G. D. Activation of VTA GABA neurons disrupts reward consumption. Neuron 73, 1184-1194 (2012).
15. Tan, K. R. et al. GABA neurons of the VTA drive conditioned place aversion. Neuron 73, 1173-1183 (2012).
16. Hazy, T. E., Frank, M. J., O'Reilly, R. C. Neural mechanisms of acquired phasic dopamine responses in learning. Neurosci. Biobehav. Rev. 34, 701-720 (2010).
17. Rivest, F., Kalaska, J. F., Bengio, Y. Conditioning and time representation in long short-term memory networks. Biol. Cybern. 108, 23-48 (2014).
18. Vitay, J.& Hamker, F.H. Timing and expectation of reward: a neuro-computational model of the afferents to the ventral tegmental area. Front. Neurorobot. 8, 4 (2014).
19. Ludvig, E. A., Sutton, R. S., Kehoe, E. J. Stimulus representation and the timing of reward-prediction errors in models of the dopamine system. Neural Comput. 20, 3034-3054 (2008).
20. Tan, C. O., Bullock, D. A local circuit model of learned striatal and dopamine cell responses under probabilistic schedules of reward. J. Neurosci. 28, 10062-10074 (2008).
21. Han, X. et al. A high-light sensitivity optical neural silencer: development and application to optogenetic control of non-human primate cortex. Front. Syst. Neurosci. 5, 18 (2011).
22. Fiorillo, C. D., Song,M. R.& Yun, S. R. Multiphasic temporal dynamics in responses of midbrain dopamine neurons to appetitive and aversive stimuli. J. Neurosci. 33, 4710-4725 (2013).
23. Pi, H.-J. et al. Cortical interneurons that specialize in disinhibitory control. Nature 503, 521-524 (2013).
24. Wilson, N. R.,Runyan, C.A.,Wang, F. L.Sur,M.Divisionandsubtractionby distinct cortical inhibitory networks in vivo. Nature 488, 343-348 (2012).
25. Atallah, B. V., Bruns, W., Carandini, M., Scanziani, M. Parvalbumin-expressing interneurons linearly transform cortical responses to visual stimuli. Neuron 73, 159-170 (2012).
26. Murphy, B. K., Miller, K. D. Multiplicative gain changes are induced by excitation or inhibition alone. J. Neurosci. 23, 10040-10051 (2003).
27. Ayaz, A.& Chance, F. S. Gain Modulation of neuronal responses by subtractive and divisive mechanisms of inhibition. J. Neurophysiol. 101, 958-968 (2009).
28. Holt, G. R., Koch, C. Shunting inhibition does not have a divisive effect on firing rates. Neural Comput. 9, 1001-1013 (1997).
29. Roy, J. E., Cullen, K. E. Dissociating self-generated from passively applied head motion: neural mechanisms in the vestibular nuclei. J. Neurosci. 24, 2102-2111 (2004).
30. Rust, N. C., Schwartz, O.,Movshon, J.A.Simoncelli, E.P.Spatiotemporal elements of macaque V1 receptive fields. Neuron 46, 945-956 (2005).
31. Bäckman, C. M. et al. Characterization of a mouse strain expressing Cre recombinase from the 39 untranslated region of the dopamine transporter locus. Genesis 44, 383-390 (2006).
32. Vong, L. et al. Leptin action on GABAergic neurons prevents obesity and reduces inhibitory tone to POMC neurons. Neuron 71, 142-154 (2011).
33. Boyden, E. S., Zhang, F., Bamberg, E., Nagel, G., Deisseroth, K. Millisecondtimescale, genetically targeted optical control of neural activity. Nature Neurosci. 8, 1263-1268 (2005).
34. Atasoy, D., Aponte, Y., Su, H. H., Sternson, S. M. A FLEX switch targets Channelrhodopsin-2 to multiple cell types for imaging and long-range circuit mapping. J. Neurosci. 28, 7025-7030 (2008).
35. Uchida, N.Mainen, Z. F.Speed andaccuracy of olfactory discrimination inthe rat. Nature Neurosci. 6, 1224-1229 (2003).
36. Schmitzer-Torbert, N., Redish, A. D. Neuronal activity in the rodent dorsal striatum in sequential navigation: separation of spatial and reward responses on the multiple T task. J. Neurophysiol. 91, 2259-2272 (2004).
37. Lima, S. Q., Hromádka, T., Znamenskiy, P., Zador, A. M. PINP: a new method of tagging neuronal populations for identification during in vivo electrophysiological recording. PLoS One 4, e6099 (2009).
38. Kvitsiani, D. et al. Distinct behavioural and network correlates of two interneuron types in prefrontal cortex. Nature 498, 363-366 (2013).
39. Olsen, S. R., Bhandawat, V., Wilson, R. I. Divisive normalization in olfactory population codes. Neuron 66, 287-299 (2010).