Basal ganglia circuits for reward value-guided behavior

Annual Review of Neuroscience

Volumn 37, Issue , 2014, Pages 289-306

  Hikosaka, Okihide ^a   Kim, Hyoung F ^a   Yasuda, Masaharu ^a   Yamamoto, Shinya ^a,b  

^a NATIONAL EYE INSTITUTE   (United States)

^b NATIONAL INSTITUTE OF ADVANCED INDUSTRIAL SCIENCE AND TECHNOLOGY AIST   (Japan)

Author keywords

Caudate nucleus; Flexible value; Stable value; Substantia nigra; Superior colliculus; Visual object

Indexed keywords

ARTICLE; BASAL GANGLION; CAUDATE NUCLEUS; DECISION MAKING; EXPERIENCE; GAZE; HUMAN; PRIORITY JOURNAL; REWARD; SHORT TERM MEMORY; SUBSTANTIA NIGRA; SUPERIOR COLLICULUS; ANIMAL; BRAIN MAPPING; EXTRAPYRAMIDAL SYNDROME; MEMORY; NERVE TRACT; PATHOPHYSIOLOGY; PHYSIOLOGY; SACCADIC EYE MOVEMENT; VISION;

ANIMALS; BASAL GANGLIA; BASAL GANGLIA DISEASES; BRAIN MAPPING; DECISION MAKING; HUMANS; MEMORY; NEURAL PATHWAYS; REWARD; SACCADES; SUPERIOR COLLICULI; VISUAL PERCEPTION;

EID: 84904700222     PISSN: 0147006X     EISSN: 15454126     Source Type: Book Series    
DOI: 10.1146/annurev-neuro-071013-013924     Document Type: Article

References (111)

1. Ahissar M, Hochstein S. 1996. Learning pop-out detection: specificities to stimulus characteristics. Vis. Res. 36:3487-500
2. Alexander GE, Crutcher MD. 1990. Functional architecture of basal ganglia circuits: neural substrates of parallel processing. Trends Neurosci. 13:266-71
3. Alexander GE, DeLong MR, Strick PL. 1986. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 9:357-81
4. Anderson BA, Laurent PA, Yantis S. 2011. Value-driven attentional capture. Proc. Natl. Acad. Sci. USA 108:10367-71
5. Anderson BA, Yantis S. 2012. Value-driven attentional and oculomotor capture during goal-directed, unconstrained viewing. Atten. Percept. Psychophys. 74:1644-53
6. Ashby FG, Maddox WT. 2005. Human category learning. Annu. Rev. Psychol. 56:149-78
7. Awh E, Belopolsky AV, Theeuwes J. 2012. Top-down versus bottom-up attentional control: a failed theoretical dichotomy. Trends Cogn. Sci. 16:437-43
8. Bechara A, Damasio H, Tranel D, Damasio AR. 1997. Deciding advantageously before knowing the advantageous strategy. Science 275:1293-95
9. Beckstead RM, Frankfurter A. 1982. The distribution and some morphological features of substantia nigra neurons that project to the thalamus, superior colliculus and pedunculopontine nucleus in the monkey. Neuroscience 7:2377-88
10. Benecke R, Rothwell JC, Dick JP, Day BL, Marsden CD. 1986. Performance of simultaneous movements in patients with Parkinsons disease. Brain 109:739-57
11. Bhatia KP, Marsden CD. 1994. The behavioural and motor consequences of focal lesions of the basal ganglia in man. Brain 117:859-76
12. Bichot NP, Schall JD. 1999. Effects of similarity and history on neural mechanisms of visual selection. Nat. Neurosci. 2:549-54
13. Bromberg-Martin ES, Matsumoto M, Hikosaka O. 2010. Dopamine in motivational control: rewarding, aversive, and alerting. Neuron 68:815-34
14. Brown LL, Schneider JS, Lidsky TI. 1997. Sensory and cognitive functions of the basal ganglia. Curr. Opin. Neurobiol. 7:157-63
15. Brown RG, Marsden CD. 1990. Cognitive function in Parkinsons disease: from description to theory. Trends Neurosci. 13:21-29
16. Brown VJ, Desimone R, Mishkin M. 1995. Responses of cells in the tail of the caudate nucleus during visual discrimination learning. J. Neurophysiol. 74:1083-94
17. Caan W, Perrett DI, Rolls ET. 1984. Responses of striatal neurons in the behaving monkey. 2. Visual processing in the caudal neostriatum. Brain Res. 290:53-65
18. Caplan LR, Schmahmann JD, Kase CS, Feldmann E, Baquis G, et al. 1990. Caudate infarcts. Arch. Neurol. 47:133-43
19. Chelazzi L, Perlato A, Santandrea E, Della Libera C. 2013. Rewards teach visual selective attention. Vis. Res. 85:58-72
20. Chun MM, Jiang Y. 1998. Contextual cueing: implicit learning and memory of visual context guides spatial attention. Cogn. Psychol. 36:28-71
21. Chun MM, Nakayama K. 2000. On the functional role of implicit visual memory for the adaptive deployment of attention across scenes. Vis. Cogn. 7:65-81
22. Cohen JD, McClure SM, Yu AJ. 2007. Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philos. Trans. R. Soc. B 362:933-42
23. Cools AR, van den Bercken JH, Horstink MW, van Spaendonck KP, Berger HJ. 1984. Cognitive and motor shifting aptitude disorder in Parkinsons disease. J. Neurol. Neurosurg. Psychiatry 47:443-53
24. Crossman ERFW. 1959. A theory of the acquisition of speed-skill. Ergonomics 2:153-66
25. Daw ND, ODoherty JP, Dayan P, Seymour B, Dolan RJ. 2006. Cortical substrates for exploratory decisions in humans. Nature 441:876-79
26. Evans JSBT. 2008. Dual-processing accounts of reasoning, judgment, and social cognition. Annu. Rev. Psychol. 59:255-78
27. Fernandez-Ruiz J, Wang J, Aigner TG, Mishkin M. 2001. Visual habit formation in monkeys with neurotoxic lesions of the ventrocaudal neostriatum. Proc. Natl. Acad. Sci. USA 98:4196-201
28. Francois C, Percheron G, Yelnik J. 1984. Localization of nigrostriatal, nigrothalamic and nigrotectal neurons in ventricular coordinates in macaques. Neuroscience 13:61-76
29. Goldman-Rakic PS. 1988. Topography of cognition: parallel distributed networks in primate association cortex. Annu. Rev. Neurosci. 11:137-56
30. Henderson JM. 2003. Human gaze control during real-world scene perception. Trends Cogn. Sci. 7:498-504
31. Hikosaka O, Nakahara H, Rand MK, Sakai K, Lu X, et al. 1999. Parallel neural networks for learning sequential procedures. Trends Neurosci. 22:464-71
32. Hikosaka O, Nakamura K, Nakahara H. 2006. Basal ganglia orient eyes to reward. J. Neurophysiol. 95:567-84
33. Hikosaka O, Sakamoto M, Miyashita N. 1993. Effects of caudate nucleus stimulation on substantia nigra cell activity in monkey. Exp. Brain Res. 95:457-72
34. Hikosaka O, Sakamoto M, Usui S. 1989. Functional properties of monkey caudate neurons. II. Visual and auditory responses. J. Neurophysiol. 61:799-813
35. Hikosaka O, Takikawa Y, Kawagoe R. 2000. Role of the basal ganglia in the control of purposive saccadic eye movements. Physiol. Rev. 80:953-78
36. Hikosaka O, Wurtz RH. 1983a. Visual and oculomotor functions of monkey substantia nigra pars reticulata. I. Relation of visual and auditory responses to saccades. J. Neurophysiol. 49:1230-53
37. Hikosaka O, Wurtz RH. 1983b. Visual and oculomotor functions of monkey substantia nigra pars reticulata. IV. Relation of substantia nigra to superior colliculus. J. Neurophysiol. 49:1285-301
38. Hikosaka O, Yamamoto S, Yasuda M, Kim HF. 2013. Why skill matters. Trends Cogn. Sci. 17:434-41
39. Johansson RS, Westling G, Backstrom A, Flanagan JR. 2001. Eye-hand coordination in object manipulation. J. Neurosci. 21:6917-32
40. Joseph JP, Boussaoud D. 1985. Role of the cat substantia nigra pars reticulata in eye and head movements. I. Neural activity. Exp. Brain Res. 57:286-96
41. Karni A, Sagi D. 1993. The time course of learning a visual skill. Nature 365:250-52
42. Kato M, MiyashitaN,Hikosaka O, Matsumura M, Usui S, Kori A. 1995. Eye movements in monkeyswith local dopamine depletion in the caudate nucleus. I. Deficits in spontaneous saccades. J. Neurosci. 15:912-27
43. Kawagoe R, Takikawa Y, Hikosaka O. 1998. Expectation of reward modulates cognitive signals in the basal ganglia. Nat. Neurosci. 1:411-16
44. Kehagia AA, Murray GK, Robbins TW. 2010. Learning and cognitive flexibility: frontostriatal function and monoaminergic modulation. Curr. Opin. Neurobiol. 20:199-204
45. Kim HF, Hikosaka O. 2013. Distinct basal ganglia circuits controlling behaviors guided by flexible and stable values. Neuron 79:1001-10
46. Kitama T, Ohno T, Tanaka M, Tsubokawa H, Yoshida K. 1991. Stimulation of the caudate nucleus induces contraversive saccadic eye movements as well as head turning in the cat. Neurosci. Res. 12:287-92
47. Knowlton BJ, Mangels JA, Squire LR. 1996. A neostriatal habit learning system in humans. Science 273:1399-402
48. Kording KP, Tenenbaum JB, Shadmehr R. 2007. The dynamics of memory as a consequence of optimal adaptation to a changing body. Nat. Neurosci. 10:779-86
49. Kording KP, Wolpert DM. 2006. Bayesian decision theory in sensorimotor control. Trends Cogn. Sci. 10:319-26
50. Kristjansson A, Sigurjonsdottir O, Driver J. 2010. Fortune and reversals of fortune in visual search: reward contingencies for pop-out targets affect search efficiency and target repetition effects. Atten. Percept. Psychophys. 72:1229-36
51. Kyllingsbaek S, Schneider WX, Bundesen C. 2001. Automatic attraction of attention to former targets in visual displays of letters. Percept. Psychophys. 63:85-98
52. Land MF. 2006. Eye movements and the control of actions in everyday life. Prog. Retin. Eye Res. 25:296-324
53. Lauwereyns J, Watanabe K, Coe B, Hikosaka O. 2002. A neural correlate of response bias in monkey caudate nucleus. Nature 418:413-17
54. Lawrence AD, Sahakian BJ,Hodges JR, Rosser AE, Lange KW, Robbins TW. 1996. Executive and mnemonic functions in early Huntingtons disease. Brain 119:1633-45
55. Lees AJ, Smith E. 1983. Cognitive deficits in the early stages of Parkinsons disease. Brain 106:257-70
56. Lieberman MD. 2000. Intuition: a social cognitive neuroscience approach. Psychol. Bull. 126:109-37
57. Logan GD. 1985. Skill and automaticity: relations, implications, and future directions. Can. J. Psychol. 39:367-86
58. Lynch JC, Hoover JE, Strick PL. 1994. Input to the primate frontal eye field from the substantia nigra, superior colliculus, and dentate nucleus demonstrated by transneuronal transport. Exp. Brain Res. 100:181-86
59. Marewski JN, Gaissmaier W, Gigerenzer G. 2010. Good judgments do not require complex cognition. Cogn. Process. 11:103-21
60. Marin O, Smeets WJ, Gonzalez A. 1998. Evolution of the basal ganglia in tetrapods: a new perspective based on recent studies in amphibians. Trends Neurosci. 21:487-94
61. Marsden CD. 1982. The mysterious motor function of the basal ganglia: the Robert Wartenberg lecture. Neurology 32:514-39
62. Matsumura M, Kojima J, Gardiner TW, Hikosaka O. 1992. Visual and oculomotor functions of monkey subthalamic nucleus. J. Neurophysiol. 67:1615-32
63. McNamara JM, Green RF, Olsson O. 2006. Bayes theorem and its applications in animal behaviour. OIKOS 112:243-51
64. Middleton FA, Strick PL. 1996. The temporal lobe is a target of output from the basal ganglia. Proc. Natl. Acad. Sci. USA 93:8683-87
65. Middleton FA, Strick PL. 2000. Basal ganglia and cerebellar loops: motor and cognitive circuits. Brain Res. Brain Res. Rev. 31:236-50
66. Mink JW. 1996. The basal ganglia: focused selection and inhibition of competing motor programs. Prog. Neurobiol. 50:381-425
67. Mishkin M, Malamut B, Bachevalier J. 1984. Memories and habits: two neural systems. In Neurobiology of Learning and Memory, ed. G Lynch, JL McGaugh, NM Weinberger, pp. 65-77. New York: Guilford
68. Mishkin M, Ungerleider LG, Macko KA. 1983. Object vision and spatial vision: two cortical pathways. Trends Neurosci. 6:414-17
69. Miyachi S, Hikosaka O, Lu X. 2002. Differential activation of monkey striatal neurons in the early and late stages of procedural learning. Exp. Brain Res. 146:122-26
70. Miyachi S, Hikosaka O, Miyashita K, Karadi Z, Rand MK. 1997. Differential roles of monkey striatum in learning of sequential hand movement. Exp. Brain Res. 115:1-5
71. Miyashita K, Rand MK, Miyachi S,Hikosaka O. 1996. Anticipatory saccades in sequential procedural learning in monkeys. J. Neurophysiol. 76:1361-66
72. Miyashita N, Hikosaka O, Kato M. 1995. Visual hemineglect induced by unilateral striatal dopamine deficiency in monkeys. NeuroReport 6:1257-60
73. Nakamura K, Hikosaka O. 2006. Role of dopamine in the primate caudate nucleus in reward modulation of saccades. J. Neurosci. 26:5360-69
74. Nambu A, Tokuno H, Takada M. 2002. Functional significance of the cortico-subthalamo-pallidal hyperdirect pathway. Neurosci. Res. 43:111-17
75. Newell A, Rosenbloom PS. 1981. Mechanisms of skill acquisition and the law of practice. In Cognitive Skills and Their Acquisition, ed. JR Anderson, pp. 1-55. Hillsdale, NJ: Erlbaum
76. Parent A, Mackey A, Smith Y, Boucher R. 1983. The output organization of the substantia nigra in primate as revealed by a retrograde double labeling method. Brain Res. Bull. 10:529-37
77. Peck CJ, Jangraw DC, Suzuki M, Efem R, Gottlieb J. 2009. Reward modulates attention independently of action value in posterior parietal cortex. J. Neurosci. 29:11182-91
78. Rangel A, Camerer C, Montague PR. 2008.Aframework for studying the neurobiology of value-based decision making. Nat. Rev. Neurosci. 9:545-56
79. Redgrave P, Rodriguez M, Smith Y, Rodriguez-Oroz MC, Lehericy S, et al. 2010. Goal-directed and habitual control in the basal ganglia: implications for Parkinsons disease. Nat. Rev. Neurosci. 11:760-72
80. Rothkopf CA, Ballard DH, Hayhoe MM. 2007. Task and context determine where you look. J. Vis. 7:1-20
81. Rudebeck PH, Behrens TE, Kennerley SW, Baxter MG, Buckley MJ, et al. 2008. Frontal cortex subregions play distinct roles in choices between actions and stimuli. J. Neurosci. 28:13775-85
82. Saint-Cyr JA, Ungerleider LG, Desimone R. 1990. Organization of visual cortical inputs to the striatum and subsequent outputs to the pallido-nigral complex in the monkey. J. Comp. Neurol. 298:129-56
83. Sakai K, Kitaguchi K, Hikosaka O. 2003. Chunking during human visuomotor sequence learning. Exp. Brain Res. 152:229-42
84. Salmon DP, Butters N. 1995. Neurobiology of skill and habit learning. Curr. Opin. Neurobiol. 5:184-90
85. Sato M, Hikosaka O. 2002. Role of primate substantia nigra pars reticulata in reward-oriented saccadic eye movement. J. Neurosci. 22:2363-73
86. Schultz W. 1998. Predictive reward signal of dopamine neurons. J. Neurophysiol. 80:1-27
87. Schultz W, Dayan P, Montague PR. 1997. A neural substrate of prediction and reward. Science 275:1593-99
88. Schwab RS, Chafetz ME, Walker S. 1954. Control of two simultaneous voluntary motor acts in normals and in parkinsonism. AMA Arch. Neurol. Psychiatry 72:591-98
89. Seitz AR, Kim D, Watanabe T. 2009. Rewards evoke learning of unconsciously processed visual stimuli in adult humans. Neuron 61:700-7
90. Sheinberg DL, Logothetis NK. 2001. Noticing familiar objects in real world scenes: the role of temporal cortical neurons in natural vision. J. Neurosci. 21:1340-50
91. Shiffrin RM, Schneider W. 1977. Controlled and automatic human information processing: II. Perceptual learning, automatic attending, and a general theory. Psychol. Rev. 84:127-90
92. Sigman M, Gilbert CD. 2000. Learning to find a shape. Nat. Neurosci. 3:264-69
93. Smith Y, Parent A. 1986. Differential connections of caudate nucleus and putamen in the squirrel monkey (Saimiri sciureus). Neuroscience 18:347-71
94. Sternberg S. 1969. Memory-scanning: mental processes revealed by reaction-time experiments. Am. Sci. 57:421-57
95. Takikawa Y, Kawagoe R, Itoh H, Nakahara H, Hikosaka O. 2002. Modulation of saccadic eye movements by predicted reward outcome. Exp. Brain Res. 142:284-91
96. Tanibuchi I, Kitano H, Jinnai K. 2009. Substantia nigra output to prefrontal cortex via thalamus in monkeys. I. Electrophysiological identification of thalamic relay neurons. J. Neurophysiol. 102:2933-45
97. Tatler BW,HayhoeMM,Land MF, Ballard DH. 2011. Eye guidance in natural vision: reinterpreting salience. J. Vis. 11:5
98. Theeuwes J, Belopolsky AV. 2012. Reward grabs the eye: oculomotor capture by rewarding stimuli. Vis. Res. 74:80-85
99. Treisman AM, Gelade G. 1980. A feature-integration theory of attention. Cogn. Psychol. 12:97-136
100. Watanabe M, Munoz DP. 2010. Saccade suppression by electrical microstimulation in monkey caudate nucleus. J. Neurosci. 30:2700-9
101. Watanabe M, Munoz DP. 2011. Saccade reaction times are influenced by caudate microstimulation following and prior to visual stimulus appearance. J. Cogn. Neurosci. 23:1794-807
102. Wichmann T, DeLong MR. 1996. Functional and pathophysiological models of the basal ganglia. Curr. Opin. Neurobiol. 6:751-58
103. Wickens JR, Reynolds JN, Hyland BI. 2003. Neural mechanisms of reward-related motor learning. Curr. Opin. Neurobiol. 13:685-90
104. Xu-Wilson M, Zee DS, Shadmehr R. 2009. The intrinsic value of visual information affects saccade velocities. Exp. Brain Res. 196:475-81
105. Yamamoto S, KimHF, Hikosaka O. 2013. Reward value-contingent changes of visual responses in the primate caudate tail associated with a visuomotor skill. J. Neurosci. 33:11227-38
106. Yamamoto S, Monosov IE, Yasuda M, Hikosaka O. 2012. What and where information in the caudate tail guides saccades to visual objects. J. Neurosci. 32:11005-16
107. Yarbus AL. 1967. Eye Movements and Vision. New York: Plenum
108. Yasuda M, Hikosaka O. 2013. Functional territories in the primate substantia nigra separately signaling flexible and stable values. Soc. Neuroscience Meet. Plann. Abstr. 291.05
109. Yasuda M, Yamamoto S, Hikosaka O. 2012. Robust representation of stable object values in the oculomotor basal ganglia. J. Neurosci. 32:16917-32
110. Yeterian EH, PandyaDN.1991. Prefrontostriatal connections in relation to cortical architectonic organization in rhesus monkeys. J. Comp. Neurol. 312:43-67
111. Yin HH, Knowlton BJ. 2006. The role of the basal ganglia in habit formation. Nat. Rev. Neurosci. 7:464-76