Distinct neural representation in the dorsolateral, dorsomedial, and ventral parts of the striatum during fixed- and free-choice tasks

Journal of Neuroscience

Volumn 35, Issue 8, 2015, Pages 3499-3514

  Ito, Makoto ^a   Doya, Kenji ^a  

^a OKINAWA INSTITUTE OF SCIENCE AND TECHNOLOGY GRADUATE UNIVERSITY   (Japan)

Author keywords

Action value; Basal ganglia; Decision making; Reinforcement learning; State value; Striatum

Indexed keywords

ANIMAL TISSUE; ANTIDROMIC STIMULATION; ARTICLE; AUDITORY STIMULATION; BASAL GANGLION; BEHAVIOR; BRAIN ELECTROPHYSIOLOGY; BRAIN NERVE CELL; CORPUS STRIATUM; DECISION MAKING; DECISION TREE; DORSOLATERAL STRIATUM; DORSOMEDIAL STRIATUM; GOAL ATTAINMENT; HABIT; HISTOLOGY; INFORMATION PROCESSING; LEARNING; MALE; MATHEMATICAL PARAMETERS; MOTIVATION; NERVE CONDUCTION; NONHUMAN; PRIORITY JOURNAL; RAT; STIMULUS RESPONSE; TASK PERFORMANCE; ANIMAL; ASSOCIATION; CYTOLOGY; LONG EVANS RAT; NERVE CELL; PHYSIOLOGY; REWARD;

ANIMALS; CHOICE BEHAVIOR; CORPUS STRIATUM; CUES; MALE; NEURONS; RATS; RATS, LONG-EVANS; REWARD;

EID: 84923330932     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.1962-14.2015     Document Type: Article

References (48)

1. Atallah HE, Lopez-Paniagua D, Rudy JW, O’Reilly RC (2007) Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nat Neurosci 10:126-131. CrossRef Medline.
2. Balleine BW, O’Doherty JP (2010) Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology 35:48–69. CrossRef Medline.
3. Balleine BW, Delgado MR, Hikosaka O (2007) The role of the dorsal striatum in reward and decision-making. J Neurosci 27:8161-8165. CrossRef Medline.
4. Barnes TD, Kubota Y, Hu D, Jin DZ, Graybiel AM (2005) Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature 437:1158-1161. CrossRef Medline.
5. Bayley PJ, Frascino JC, Squire LR (2005) Robust habit learning in the absence of awareness and independent of the medial temporal lobe. Nature 436:550-553. CrossRef Medline.
6. Berridge KC, Robinson TE (1998) What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res Rev 28:309-369. CrossRef Medline.
7. Broadbent NJ, Squire LR, Clark RE (2007) Rats depend on habit memory for discrimination learning and retention. Learn Mem 14:145–151. CrossRef Medline.
8. Cardinal RN, Parkinson JA, Hall J, Everitt BJ (2002) Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci Biobehav Rev 26:321–352. CrossRef Medline.
9. Corrado G, Doya K (2007) Understanding neural coding through the model-based analysis of decision making. J Neurosci 27:8178–8180. CrossRef Medline.
10. Daw ND, Niv Y, Dayan P (2005) Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci 8:1704-1711. CrossRef Medline.
11. Daw ND, Gershman SJ, Seymour B, Dayan P, Dolan RJ (2011) Model-based influences on humans’ choices and striatal prediction errors. Neuron 69:1204-1215. CrossRef Medline.
12. Doya K (1999) What are the computations of the cerebellum, the basal ganglia and the cerebral cortex? Neural Netw 12:961–974. CrossRef Medline.
13. Doya K (2000) Complementary roles of basal ganglia and cerebellum in learning and motor control. Curr Opin Neurobiol 10:732–739. CrossRef Medline.
14. Gremel CM, Costa RM (2013) Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions. Nat Commun 4:2264. CrossRef Medline.
15. Hori Y, Minamimoto T, Kimura M (2009) Neuronal encoding of reward value and direction of actions in the primate putamen. J Neurophysiol 102:3530-3543. CrossRef Medline.
16. Houk JC, Adams JL, Barto AG (1995) A model of how the basal ganglia generate and use neural signals that predict reinforcement. In: Models of information processing in the basal ganglia (Houk JC, Davis JL, Beiser DG, eds). Cambridge, MA: Massachusetts Institute of Technology.
17. Ito M, Doya K (2009) Validation of decision-making models and analysis of decision variables in the rat basal ganglia. J Neurosci 29:9861–9874. CrossRef Medline.
18. Ito M, Doya K (2011) Multiple representations and algorithms for reinforcement learning in the cortico-basal ganglia circuit. Curr Opin Neurobiol 21:368-373. CrossRef Medline.
19. Joel D, Niv Y, Ruppin E (2002) Actor-critic models of the basal ganglia: new anatomical and computational perspectives. Neural Netw 15:535–547. CrossRef Medline.
20. Kim H, Sul JH, Huh N, Lee D, Jung MW (2009) Role of striatum in updating values of chosen actions. J Neurosci 29:14701–14712. CrossRef Medline.
21. Kim H, Lee D, Jung MW (2013) Signals for previous goal choice persist in the dorsomedial, but not dorsolateral striatum of rats. J Neurosci 33:52–63. CrossRef Medline.
22. Kimchi EY, Laubach M (2009) The dorsomedial striatum reflects response bias during learning. J Neurosci 29:14891–14902. CrossRef Medline.
23. Kimchi EY, Torregrossa MM, Taylor JR, Laubach M (2009) Neuronal correlates of instrumental learning in the dorsal striatum. J Neurophysiol 102:475-489. CrossRef Medline.
24. Lau B, Glimcher PW (2008) Value representations in the primate striatum during matching behavior. Neuron 58:451-463. CrossRef Medline.
25. Nicola SM (2010) The flexible approach hypothesis: unification of effort and cue-responding hypotheses for the role of nucleus accumbens dopamine in the activation of reward-seeking behavior. J Neurosci 30:16585-16600. CrossRef Medline.
26. O’Doherty JP, Hampton A, Kim H (2007) Model-based fMRI and its application to reward learning and decision making. Ann N Y Acad Sci 1104: 35–53. CrossRef Medline.
27. O’Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304:452-454. CrossRef Medline.
28. Panzeri S, Treves A (1996) Analytical estimates of limited sampling biases in different information measures. Network 7:87–107.CrossRef.
29. Pasquereau B, Nadjar A, Arkadir D, Bezard E, Goillandeau M, Bioulac B, Gross CE, Boraud T (2007) Shaping of motor responses by incentive values through the basal ganglia. J Neurosci 27:1176-1183. CrossRef Medline.
30. Pasupathy A, Miller EK (2005) Different time courses of learning-related activity in the prefrontal cortex and striatum. Nature 433:873-876. CrossRef Medline.
31. Paxinos G, Watson C (1998) The rat brain stereotaxic coordinates, Ed 4. San Diego: Academic.
32. Roesch MR, Singh T, Brown PL, Mullins SE, Schoenbaum G (2009) Ventral striatal neurons encode the value of the chosen action in rats deciding between differently delayed or sized rewards. J Neurosci 29:13365–13376. CrossRef Medline.
33. Samejima K, Doya K (2007) Multiple representations of belief states and action values in corticobasal ganglia loops. Ann N Y Acad Sci 1104:213-228. CrossRef Medline.
34. Samejima K, Ueda Y, Doya K, Kimura M (2005) Representation of actionspecific reward values in the striatum. Science 310:1337–1340. CrossRef Medline.
35. Schmitzer-Torbert N, Redish AD (2004) Neuronal activity in the rodent dorsal striatum in sequential navigation: separation of spatial and reward responses on the multiple T task. J Neurophysiol 91:2259-2272. CrossRef Medline.
36. Stalnaker TA, Calhoon GG, Ogawa M, Roesch MR, Schoenbaum G (2010) Neural correlates of stimulus–response and response-outcome associations in dorsolateral versus dorsomedial striatum. Front Integr Neurosci 4:12. CrossRef Medline.
37. Sutton RS, Barto AG (1998) Reinforcement learning. Cambridge, MA: Massachusetts Institute of Technology. Takahashi Y, Roesch MR, Stalnaker TA, Schoenbaum G (2007) Cocaine exposure shifts the balance of associative encoding from ventral to dorsolateral striatum. Front Integr Neurosci 1:11. CrossRef Medline.
38. Takahashi Y, Schoenbaum G, Niv Y (2008) Silencing the critics: understanding the effects of cocaine sensitization on dorsolateral and ventral striatum in the context of an actor/critic model. Front Neurosci 2:86-99. CrossRef Medline.
39. Thorn CA, Atallah H, Howe M, Graybiel AM (2010) Differential dynamics of activity changes in dorsolateral and dorsomedial striatal loops during learning. Neuron 66:781–795. CrossRef Medline.
40. van der Meer MA, Johnson A, Schmitzer-Torbert NC, Redish AD (2010) Triple dissociation of information processing in dorsal striatum, ventral striatum, and hippocampus on a learned spatial decision task. Neuron 67:25–32. CrossRef Medline.
41. Voorn P, Vanderschuren LJ, Groenewegen HJ, Robbins TW, Pennartz CM (2004) Putting a spin on the dorsal-ventral divide of the striatum. Trends Neurosci 27:468–474. CrossRef Medline.
42. Watkins CJCH, Dayan P (1992) Q-learning. Machine Learning 8:279-292.CrossRef.
43. Wickens JR, Budd CS, Hyland BI, Arbuthnott GW (2007) Striatal contributions to reward and decision making: making sense of regional variations in a reiterated processing matrix. Ann N Y Acad Sci 1104:192–212. CrossRef Medline.
44. Wunderlich K, Rangel A, O’Doherty JP (2009) Neural computations underlying action-based decision making in the human brain. Proc Natl Acad Sci U S A 106:17199-17204. CrossRef Medline.
45. Yin HH, Knowlton BJ, Balleine BW (2004) Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur J Neurosci 19:181–189. CrossRef Medline.
46. Yin HH, Knowlton BJ, Balleine BW (2005a) Blockade of NMDA receptors in the dorsomedial striatum prevents action-outcome learning in instrumental conditioning. Eur J Neurosci 22:505–512. CrossRef Medline.
47. Yin HH, Ostlund SB, Knowlton BJ, Balleine BW (2005b) The role of the dorsomedial striatum in instrumental conditioning. Eur J Neurosci 22: 513–523. CrossRef Medline.
48. Yin HH, Knowlton BJ, Balleine BW (2006) Inactivation of dorsolateral striatum enhances sensitivity to changes in the action-outcome contingency in instrumental conditioning. Behav Brain Res 166:189-196. CrossRef Medline.