The basal ganglia and the encoding of value

Neuroeconomics

Volumn , Issue , 2009, Pages 407-416

  Doya, Kenji ^a   Kimura, Minoru ^b  

^a OKINAWA INSTITUTE OF SCIENCE AND TECHNOLOGY GRADUATE UNIVERSITY   (Japan)

^b KYOTO PREFECTURAL UNIVERSITY OF MEDICINE   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84871782729     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1016/B978-0-12-374176-9.00026-9     Document Type: Chapter

References (58)

1. Bayer H.M., Glimcher P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 2005, 47:129-141.
2. Calabresi P., Pisani A., Mercuri N.B., Bernardi G. The corticostriatal projection: from synaptic plasticity to dysfunctions of the basal ganglia. Trends Neurosci. 1996, 19:19-24.
3. Daw N.D., Doya K. The computational neurobiology of learning and reward. Curr. Opin. Neurobiol. 2006, 16:199-204.
4. Dayan P., Balleine B.W. Reward, motivation, and reinforcement learning. Neuron 2002, 36:285-298.
5. Dickinson A. Contemporary Animal Learning Theory 1980, Cambridge University Press, Cambridge.
6. Dorris M.C., Glimcher P.W. Activity in posterior parietal cortex is correlated with the relative subjective desirability of action. Neuron 2004, 44:365-378.
7. Doya K. What are the computations of the cerebellum, the basal ganglia and the cerebral cortex?. Neural Networks 1999, 12:961-974.
8. Doya K. Complementary roles of basal ganglia and cerebellum in learning and motor control. Curr. Opin. Neurobiol. 2000, 10:732-739.
9. Doya K. Metalearning and neuromodulation. Neural Networks 2002, 15:495-506.
10. Doya K. Reinforcement learning: computational theory and biological mechanisms. HFSP J. 2007, 1:30-40.
11. Fiorillo C.D., Tobler P.N., Schultz W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 2003, 299:1898-1902.
12. Groenewegen H.J., Berendse H.W. The specificity of the "nonspecific" midline and intralaminar thalamic nuclei. Trends Neurosci. 1994, 17:52-57.
13. Haruno M., Kawato M. Heterarchical reinforcement-learning model for integration of multiple cortico-striatal loops: fMRI examination in stimulus-action-reward association learning. Neural Networks 2006, 19:1242-1254.
14. Horvitz J.C., Stewart T., Jacobs B.L. Burst activity of ventral tegmental dopamine neurons is elicited by sensory stimuli in the awake cat. Brain Res. 1997, 759:251-258.
15. Houk J.C., Adams J.L., Barto A.G. A model of how the basal ganglia generate and use neural signals that predict reinforcement. Models of Information Processing in the Basal Ganglia 1995, 249-270. MIT Press, Cambridge, MA. J.C. Houk, J.L. Davis, D.G. Beiser (Eds.).
16. Isoda M., Hikosaka O. Switching from automatic to controlled action by monkey medial frontal cortex. Nat. Neurosci. 2007, 10:240-248.
17. Jacobs B.L. Single unit activity of brain monoamine-containing neurons in freely moving animals. Ann. NY Acad. Sci. 1986, 473:70-77.
18. Kawagoe R., Takikawa Y., Hikosaka O. Expectation of reward modulates cognitive signals in the basal ganglia. Nat. Neurosci. 1998, 1:411-416.
19. Kobayashi Y., Inoue Y., Yamamoto M., et al. Contribution of pedunculopontine tegmental nucleus neurons to performance of visually guided saccade tasks in monkeys. J. Neurophysiol. 2002, 88:715-731.
20. Koepp M.J., Gunn R.N., Lawrence A.D., et al. Evidence for striatal dopamine release during a video game. Nature 1998, 393:266-268.
21. Ljungberg T., Apicella P., Schultz W. Responses of monkey dopamine neurons during learning of behavioral reactions. J. Neurophysiol. 1992, 67:145-163.
22. Matsumoto N., Minamimoto T., Graybiel A.M., Kimura M. Neurons in the thalamic CM-Pf complex supply striatal neurons with information about behaviorally significant sensory events. J. Neurophysiol. 2001, 85:960-976.
23. McClure S.M., Laibson D.I., Loewenstein G., Cohen J.D. Separate neural systems value immediate and delayed monetary rewards. Science 2004, 306:503-507.
24. Minamimoto T., Kimura M. Participation of the thalamic CM-Pf complex in attentional orienting. J. Neurophysiol. 2002, 87:3090-3101.
25. Minamimoto T., Hori Y., Kimura M. Complementary process to response bias in the centromedian nucleus of the thalamus. Science 2005, 308:1798-1801.
26. Morris G., Arkadir D., Nevet A., et al. Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron 2004, 43:133-143.
27. Morris G., Nevet A., Arkadir D., et al. Midbrain dopamine neurons encode decisions for future action. Nat. Neurosci. 2006, 9:1057-1063.
28. Nakahara H., Itoh H., Kawagoe R., et al. Dopamine neurons can represent context-dependent prediction error. Neuron 2004, 41:269-280.
29. Niv Y., Daw N.D., Joel D., Dayan P. Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacol. (Berl) 2007, 191:507-520.
30. O'Doherty J., Dayan P., Schultz J., et al. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 2004, 304:452-454.
31. Pasquereau B., Nadjar A., Arkadir D., et al. Shaping of motor responses by incentive values through the basal ganglia. J. Neurosci. 2007, 27:1176-1183.
32. Redgrave P., Prescott T.J., Gurney K. Is the short-latency dopamine response too short to signal reward error?. Trends Neurosci. 1999, 22:146-151.
33. Rescorla R.A., Wagner A.R. Current research and theory. Classical Conditioning 1972, Vol. II:64-99. Appleton Century Crofts, New York, NY. A.H. Black, W.F. Prokasy (Eds.).
34. Reynolds J.N., Wickens J.R. Dopamine-dependent plasticity of corticostriatal synapses. Neural Networks 2002, 15:507-521.
35. Reynolds J.N., Hyland B.I., Wickens J.R. A cellular mechanism of reward-related learning. Nature 2001, 413:67-70.
36. Robbins T.W., Everitt B.J. Neurobehavioural mechanisms of reward and motivation. Curr. Opin. Neurobiol. 1996, 6:228-236.
37. Salamone J.D., Correa M. Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine. Behav. Brain Res. 2002, 137:3-25.
38. Samejima K., Ueda Y., Doya K., Kimura M. Representation of action-specific reward values in the striatum. Science 2005, 310:1337-1340.
39. Satoh T., Nakai S., Sato T., Kimura M. Correlated coding of motivation and outcome of decision by dopamine neurons. J. Neurosci. 2003, 23:9913-9923.
40. Schultz W. Predictive reward signal of dopamine neurons. J. Neurophysiol. 1998, 80:1-27.
41. Schultz W., Dickinson A. Neuronal coding of prediction errors. Annu. Rev. Neurosci. 2000, 23:473-500.
42. Schultz W., Romo R. Responses of nigrostriatal dopamine neurons to high-intensity somatosensory stimulation in the anesthetized monkey. J. Neurophysiol. 1987, 57:201-217.
43. Schultz W., Dayan P., Montague P.R. A neural substrate of prediction and reward. Science 1997, 275:1593-1599.
44. Shidara M., Aigner T.G., Richmond B.J. Neuronal signals in the monkey ventral striatum related to progress through a predictable series of trials. J. Neurosci. 1998, 18:2613-2625.
45. Shima K., Tanji J. Role for cingulate motor area cells in voluntary movement selection based on reward. Science 1998, 282:1335-1338.
46. Smith Y., Raju D.V., Pare J.F., Sidibe M. The thalamostriatal system: a highly specific network of the basal ganglia circuitry. Trends Neurosci. 2004, 27:520-527.
47. Spanagel R., Weiss F. The dopamine hypothesis of reward: past and current status. Trends Neurosci. 1999, 22:521-527.
48. Steriade M., Jones E.G., McCormick C.D. Organisation and Function 1997, Elsevier Science, Oxford.
49. Sugrue L.P., Corrado G.S., Newsome W.T. Matching behavior and the representation of value in the parietal cortex. Science 2004, 304:1782-1787.
50. Sutton R.S., Barto A.G. Reinforcement Learning 1998, MIT Press, Cambridge, MA.
51. Takada M., Tokuno H., Hamada I., et al. Organization of inputs from cingulate motor areas to basal ganglia in macaque monkey. Eur. J. Neurosci. 2001, 14:1633-1650.
52. Takikawa Y., Kawagoe R., Itoh H., et al. Modulation of saccadic eye movements by predicted reward outcome. Exp. Brain Res. 2002, 142:284-291.
53. Tanaka S.C., Doya K., Okada G., et al. Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat. Neurosci. 2004, 7:887-893.
54. Watanabe K., Lauwereyns J., Hikosaka O. Neural correlates of rewarded and unrewarded eye movements in the primate caudate nucleus. J. Neurosci. 2003, 23:10052-10057.
55. Watanabe M. Reward expectancy in primate prefrontal neurons. Nature 1996, 382:629-632.
56. Watanabe M., Cromwell H.C., Tremblay L., et al. Behavioral reactions reflecting differential reward expectations in monkeys. Exp. Brain Res. 2001, 140:511-518.
57. Wickens J.R., Begg A.J., Arbuthnott G.W. Dopamine reverses the depression of rat corticostriatal synapses which normally follows high-frequency stimulation of cortex in vitro. Neuroscience 1996, 70:1-5.
58. Wise R.A. Brain reward circuitry: insights from unsensed incentives. Neuron 2002, 36:229-240.