Midbrain dopamine neurons encode decisions for future action

Nature Neuroscience

Volumn 9, Issue 8, 2006, Pages 1057-1063

  Morris, Genela ^a   Nevet, Alon ^a   Arkadir, David ^a   Vaadia, Eilon ^a   Bergman, Hagai ^a  

^a HEBREW UNIVERSITY OF JERUSALEM   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

ANIMAL EXPERIMENT; ARTICLE; BASAL GANGLION; BEHAVIOR; CALCULATION; CONTROLLED STUDY; DECISION MAKING; DOPAMINERGIC NERVE CELL; FEMALE; MACACA; MALE; MATHEMATICAL ANALYSIS; MESENCEPHALON; NONHUMAN; PRIORITY JOURNAL; PROBABILITY; REINFORCEMENT; REWARD; SYNAPTIC TRANSMISSION; TASK PERFORMANCE;

ANIMALS; BEHAVIOR, ANIMAL; DECISION MAKING; DOPAMINE; ELECTROPHYSIOLOGY; FEMALE; MACACA; MALE; MESENCEPHALON; NEURONS; PROBABILITY; RANDOM ALLOCATION; REWARD; SACCADES;

EID: 33747585633     PISSN: 10976256     EISSN: None     Source Type: Journal    
DOI: 10.1038/nn1743     Document Type: Article

References (40)

1. Herrnstein, R.J. On the law of effect. J. Exp. Anal. Behav. 13, 243-266 (1970).
2. Tversky, A. & Kahneman, D. The framing of decisions and the psychology of choice. Science 211, 453-458 (1981).
3. Bayer, H.M. & Glimcher, P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129-141 (2005).
4. Sugrue, L.P., Corrado, G.S. & Newsome, W.T. Matching behavior and the representation of value in the parietal cortex. Science 304, 1782-1787 (2004).
5. Romo, R., Hernandez, A. & Zainos, A. Neuronal correlates of a perceptual decision in ventral premotor cortex. Neuron 41, 165-173 (2004).
6. Barraclough, D.J., Conroy, M.L. & Lee, D. Prefrontal cortex and decision making in a mixed-strategy game. Nat. Neurosci. 7, 404-410 (2004).
7. Samejima, K., Ueda, Y., Doya, K. & Kimura, M. Representation of action-specific reward values in the striatum. Science 310, 1337-1340 (2005).
8. Pagnoni, G., Zink, C.F., Montague, P.R. & Berns, G.S. Activity in human ventral striatum locked to errors of reward prediction. Nat. Neurosci. 5, 97-98 (2002).
9. Sutton, R.S. & Barto, A.G. Reinforcement Learning: an Introduction (MIT Press, Cambridge, Massachusetts, 1998).
10. Rummery, G.A. & Niranjan, M. On-line Q-learning using connectionist systems. Technical Report CUED/F-INFENG/TR 166 (Engineering Department, Cambridge University, Cambridge, UK, 1994).
11. Watkins, C.J.C.H. & Dayan, P. Q learning. Mach. Learn. 8, 279-292 (1992).
12. Dayan, P. & Balleine, B.W. Reward, motivation, and reinforcement learning. Neuron 36, 285-298 (2002).
13. Schultz, W., Dayan, P. & Montague, P.R. A neural substrate of prediction and reward. Science 275, 1593-1599 (1997).
14. Fiorillo, C.D., Tobler, P.N. & Schultz, W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299, 1898-1902 (2003).
15. Satoh, T., Nakai, S., Sato, T. & Kimura, M. Correlated coding of motivation and outcome of decision by dopamine neurons. J. Neurosci. 23, 9913-9923 (2003).
16. Morris, G., Arkadir, D., Nevet, A., Vaadia, E. & Bergman, H. Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron 43, 133-143 (2004).
17. Nakahara, H., Itoh, H., Kawagoe, R., Takikawa, Y. & Hikosaka, O. Dopamine neurons can represent context-dependent prediction error. Neuron 41, 269-280 (2004).
18. Pan, W.X., Schmidt, R., Wickens, J.R. & Hyland, B.I. Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J. Neurosci. 25, 6235-6242 (2005).
19. Gurney, K., Prescott, T.J., Wickens, J.R. & Redgrave, P. Computational models of the basal ganglia: from robots to membranes. Trends Neurosci. 27, 453-459 (2004).
20. Mink, J.W. The basal ganglia: focused selection and inhibition of competing motor programs. Prog. Neurobiol. 50, 381-425 (1996).
21. Bar-Gad, I., Morris, G. & Bergman, H. Information processing, dimensionality reduction and reinforcement learning in the basal ganglia. Prog. Neurobiol. 71, 439-473 (2003).
22. Barto, A.G. Adaptive critics and the basal ganglia. in Models of Information Processing in the Basal Ganglia (eds. Houk, J.C., Davis, J.L. & Beiser, D.G.) 215-232 (MIT Press, Cambridge, Massachusetts, 1995).
23. Reynolds, J.N., Hyland, B.I. & Wickens, J.R. A cellular mechanism of reward-related learning. Nature 413, 67-70 (2001).
24. McClure, S.M., Daw, N.D. & Montague, P.R. A computational substrate for incentive salience. Trends Neurosci. 26, 423-428 (2003).
25. Tobler, P.N., Fiorillo, C.D. & Schultz, W. Adaptive coding of reward value by dopamine neurons. Science 307, 1642-1645 (2005).
26. Wolford, G., Miller, M.B. & Gazzaniga, M. The left hemisphere's role in hypothesis formation. J. Neurosci. 20, RC64 (2000).
27. Vulkan, N. An economist's perspective on probability matching. J. Econ. Surv. 14, 101-118 (2000).
28. Lau, B. & Glimcher, P.W. Dynamic response-by-response models of matching behavior in rhesus monkeys. J. Exp. Anal. Behav. 84, 555-579 (2005).
29. Dommett, E. et al. How visual stimuli activate dopaminergic neurons at short latency. Science 307, 1476-1479 (2005).
30. Wise, R.A. Dopamine, learning and motivation. Nat. Rev. Neurosci. 5, 483-494 (2004).
31. Cragg, S.J. & Rice, M.E. DAncing past the DAT at a DA synapse. Trends Neurosci. 27, 270-277 (2004).
32. Wickens, J.R. & Arbuthnot, G.W. Structural and functional interactions in the striatum at the receptor level. in Dopamine (eds. Dunnett, S.B., Bentivoglio, M., Bjorklund, A. & Hokfelt, T.) 199-236 (Elsevier, Amsterdam, 2005).
33. Waelti, P., Dickinson, A. & Schultz, W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 412, 43-48 (2001).
34. Szabo, J. & Cowan, W.M. A stereotaxic atlas of the brain of the cynomolgus monkey (Macaca fascicularis). J. Comp. Neurol. 222, 265-300 (1984).
35. Martin, R.F. & Bowden, D.M. Primate Brain Maps: Structure of the Macaque Brain (Elsevier Science, Amsterdam, 2000).
36. Grace, A.A. & Bunney, B.S. Intracellular and extracellular electrophysiology of nigral dopaminergic neurons-1. Identification and characterization. Neuroscience 10, 301-315 (1983).
37. Ungless, M.A., Magill, P.J. & Bolam, J.P. Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science 303, 2040-2042 (2004).
38. Hollerman, J.R. & Schultz, W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci. 1, 304-309 (1998).
39. Nevet, A., Morris, G., Saban, G., Fainstein, N. & Bergman, H. Discharge rate of substantia nigra pars reticulata neurons is reduced in non-parkinsonian monkeys with apomorphine-induced orofacial dyskinesia. J. Neurophysiol. 92, 1973-1981 (2004).
40. Sokal, R.R. & Rohlf, F.J. Biometry (W.H. Freeman & Co., New York, 1981).