Phasic dopamine signals: From subjective reward value to formal economic utility

Current Opinion in Behavioral Sciences

Volumn 5, Issue , 2015, Pages 147-154

  Schultz, Wolfram ^a   Carelli, Regina M ^b   Wightman, R Mark ^b  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^b University of North Carolina at Chapel Hill   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DOPAMINE;

BEHAVIOR; DELAY DISCOUNTING; DOPAMINE RELEASE; DOPAMINERGIC NERVE CELL; ECONOMICS; NEUROBIOLOGY; NEUROPHYSIOLOGY; NONHUMAN; POTENTIOMETRY; PREDICTION; REVIEW; REWARD;

EID: 84949322784     PISSN: None     EISSN: 23521546     Source Type: Journal    
DOI: 10.1016/j.cobeha.2015.09.006     Document Type: Review

References (56)

1. Kagel J.H., Battalio R.C., Green L. Economic Choice Theory: An Experimental Analysis of Animal Behavior 1995, Cambridge University Press.
2. von Neumann J., Morgenstern O. The Theory of Games and Economic Behavior 1944, Princeton University Press.
3. Padoa-Schioppa C., Assad J.A. Neurons in the orbitofrontal cortex encode economic value. Nature 2006, 441:223-226.
4. Lak A., Stauffer W.R., Schultz W. Dopamine prediction error responses integrate subjective value from different reward dimensions. Proc Natl Acad Sci U S A 2014, 111:2343-2348.
5. Fiorillo C.D., Tobler P.N., Schultz W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 2003, 299:1898-1902.
6. Morris G., Arkadir D., Nevet A., Vaadia E., Bergman H. Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron 2004, 43:133-143.
7. Tobler P.N., Fiorillo C.D., Schultz W. Adaptive coding of reward value by dopamine neurons. Science 2005, 307:1642-1645.
8. Day J.J., Roitman M.F., Wightman R.M., Carelli R.M. Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat Neurosci 2007, 10:1020-1028.
9. Sugam J.A., Day J.J., Wightman R.M., Carelli R.M. Phasic nucleus accumbens dopamine encodes risk-based decision-making behavior. Biol Psychiatry 2012, 71:199-2015.
10. Saddoris M.P., Sugam J.A., Stuber G.D., Witten I.B., Deisseroth K., Carelli R.M. Mesolimbic dopamine dynamically tracks, and is causally linked to, discrete aspects of value-based decision making. Biol Psychiatry 2015, 77:903-911.
11. Mirenowicz J., Schultz W. Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 1996, 379:449-451.
12. Matsumoto M., Hikosaka O. Two types of dopamine neuron distinctively convey positive and negative motivational signals. Nature 2009, 459:837-841.
13. Fiorillo C.D., Song M.R., Yun S.R. Multiphasic temporal dynamics in responses of midbrain dopamine neurons to appetitive and aversive stimuli. J Neurosci 2013, 33:4710-4725.
14. Fiorillo C.D. Two dimensions of value: dopamine neurons represent reward but not aversiveness. Science 2013, 341:546-549.
15. Rothschild M., Stiglitz J.E. Increasing risk: I. A definition. J Econ Theory 1970, 2:225-243.
16. McCoy A.N., Platt M.L. Risk-sensitive neurons in macaque posterior cingulate cortex. Nat Neurosci 2005, 8:1220-1227.
17. O'Neill M., Schultz W. Coding of reward risk distinct from reward value by orbitofrontal neurons. Neuron 2010, 68:789-800.
18. Stauffer W.R., Lak A., Schultz W. Dopamine reward prediction error responses reflect marginal utility. Curr Biol 2014, 24:2491-2500.
19. Weber B.J., Chapman G.B. Playing for peanuts: why is risk seeking more common for low-stakes gambles?. Organ Behav Hum Decis Process 2005, 97:31-46.
20. Fiorillo C.D. Transient activation of midbrain dopamine neurons by reward risk. Neuroscience 2011, 197:162-171.
21. Mas-Colell A., Whinston M., Green J. Microeconomic Theory 1995, Oxford University Press.
22. Ainslie G.W. Impulse control in pigeons. J Exp Anal Behav 1974, 21:485-489.
23. Rodriguez M.L., Logue A.W. Adjusting delay to reinforcement: comparing choice in pigeons and humans. J Exp Psychol Anim Behav Proc 1988, 14:105-117.
24. Richards J.B., Mitchell S.H., de Wit H., Seiden L.S. Determination of discount functions in rats with an adjusting-amount procedure. J Exp Anal Behav 1997, 67:353-366.
25. Kobayashi S., Schultz W. Influence of reward delays on responses of dopamine neurons. J Neurosci 2008, 28:7837-7846.
26. Day J.J., Jones J.L., Wightman R.M., Carelli R.M. Phasic nucleus accumbens dopamine release encodes effort- and delay-related costs. Biol Psychiatry 2010, 68:306-309.
27. Fiorillo C.D., Newsome W.T., Schultz W. The temporal precision of reward prediction in dopamine neurons. Nat Neurosci 2008, 11:966-973.
28. Gan J.O., Walton M.E., Phillips P.E.M. Dissociable cost and benefit encoding of future rewards by mesolimbic dopamine. Nat Neurosci 2010, 13:25-27.
29. Hollon N.G., Arnold M.M., Gan J.O., Walton M.E., Phillips P.E. Dopamine-associated cached values are not sufficient as the basis for action selection. Proc Natl Acad Sci U S A 2014, 111:18357-18362.
30. Corbett D., Wise R.A. Intracranial self-stimulation in relation to the ascending dopaminergic systems of the midbrain: a moveable microelectrode study. Brain Res 1980, 185:1-15.
31. Tsai H.-C., Zhang F., Adamantidis A., Stuber G.D., Bonci A., de Lecea L., Deisseroth K. Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science 2009, 324:1080-1084.
32. Steinberg E.E., Keiflin R., Boivin J.R., Witten I.B., Deisseroth K., Janak P.H. A causal link between prediction errors, dopamine neurons and learning. Nat Neurosci 2013, 16:966-973.
33. Savage L.J. The Foundations of Statistics 1954, Wiley.
34. Debreu G. Cardinal utility for even-chance mixtures of pairs of sure prospects. Rev Econ Stud 1959, 26:174-177.
35. Pasquereau B., Turner R.S. Limited encoding of effort by dopamine neurons in a cost-benefit trade-off task. J Neurosci 2013, 33:8288-8300.
36. Varazzani C., San-Galli A., Gilardeau S., Bouret S. Noradrenaline and dopamine neurons in the reward/effort trade-off: a direct electrophysiological comparison in behaving monkeys. J Neurosci 2015, 35:7866-7877.
37. Wanat M.J., Kuhnen C.M., Phillips P.E.M. Delays conferred by escalating costs modulate dopamine release to rewards but not their predictors. J Neurosci 2010, 30:12020-12027.
38. Clark J.J., Sandberg S.G., Wanat M.J., Gan J.O., Horne E.A., Hart A.S., Akers C.A., Parker J.G., Willuhn I., Martinez V., Evans S.B., Stella N., Phillips P.E.M. Chronic microsensors for longitudinal, subsecond dopamine detection in behaving animals. Nat Methods 2010, 7:126-129.
39. Garris P.A., Christensen J.R., Rebec G.V., Wightman R.M. Real-time measurement of electrically evoked extracellular dopamine in the striatum of freely moving rats. J Neurochem 1997, 68:152-161.
40. Wightman R.M., Heien M.L., Wassum K.M., Sombers L.A., Aragona B.J., Khan A.S., Ariansen J.L., Cheer J.F., Phillips P.E., Carelli R.M. Dopamine release is heterogeneous within microenvironments of the rat nucleus accumbens. Eur J Neurosci 2007, 26:2046-2054.
41. Willuhn I., Burgeno L.M., Everitt B.J., Phillips P.E. Hierarchical recruitment of phasic dopamine signaling in the striatum during the progression of cocaine use. Proc Natl Acad Sci U S A 2012, 109:20703-20708.
42. Swiergiel A.H., Palamarchouk V.S., Dunn A.J. A new design of carbon fiber microelectrode for in vivo voltammetry using fused silica. J Neurosci Methods 1997, 73:29-33.
43. Gerhardt G.A., Ksir C., Pivik C., Dickinson S.D., Sabeti J., Zahniser N.R. Methodology for coupling local application of dopamine and other chemicals with rapid in vivo electrochemical recordings in freely-moving rats. J Neurosci Methods 1999, 87:67-76.
44. Polikov V.S., Tresco P.A., Reichert W.M. Response of brain tissue to chronically implanted neural electrodes. J Neurosci Methods 2005, 148:1-18.
45. Kozai T.D., Jaquins-Gerstl A., Vazquez A., Michael A.C., Cui X. Brain tissue responses to neural implants impact signal sensitivity and intervention strategies. ACS Chem Neurosci 2015, 6:48-67.
46. Winslow B.D., Tresco P.A. Quantitative analysis of the tissue response to chronically implanted microwire electrodes in rat cortex. Biomaterials 2010, 31:1558-1567.
47. Cengiz E., Sherr J.L., Weinzimer S.A., Tamborlane W.V. New-generation diabetes management: glucose sensor-augmented insulin pump therapy. Expert Rev Med Devices 2011, 8:449-458.
48. Giros B., Jaber M., Jones S.R., Wightman R.M., Caron M.G. Hyperlocomotion and indifference to cocaine and amphetamine in mice lacking the dopamine transporter. Nature 1996, 379:606-612.
49. Montague P.R., McClure S.M., Baldwin P.R., Phillips P.E., Budygin E.A., Stuber G.D., Kilpatrick M.R., Wightman R.M. Dynamic gain control of dopamine delivery in freely moving animals. J Neurosci 2004, 24:1754-1759.
50. Robinson D.L., Hermans A., Seipel A.T., Wightman R.M. Monitoring rapid chemical communication in the brain. Chem Rev 2008, 108:2554-2584.
51. Jones S.R., Joseph J.D., Barak L.S., Caron M.G., Wightman R.M. Dopamine neuronal transport kinetics and effects of amphetamine. J Neurochem 1999, 73:2406-2414.
52. Ariansen J.L., Heien M.L., Hermans A., Phillips P.E., Hernadi I., Bermudez M.A., Schultz W., Wightman R.M. Monitoring extracellular pH, oxygen, and dopamine during reward delivery in the striatum of primates. Front Behav Neurosci 2012, 6:1-10.
53. Rodeberg N.T., Johnson J.A., Cameron C.M., Saddoris M.P., Carelli R.M., Wightman R.M. Construction of training sets for valid calibration of in vivo cyclic voltammetric data by principal component analysis. Anal Chem 2015, 87:11484-11491.
54. Heien M.L., Khan A.S., Ariansen J.L., Cheer J.F., Phillips P.E., Wassum K.M., Wightman R.M. Real-time measurement of dopamine fluctuations after cocaine in the brain of behaving rats. Proc Natl Acad Sci U S A 2005, 102:10023-10028.
55. Caraco T., Martindale S., Whitham T.S. An empirical demonstration of risk-sensitive foraging preferences. Anim Behav 1980, 28:820-830.
56. Machina M.J. Choice under uncertainty: problems solved and unsolved. J Econ Perspect 1987, 1:121-154.