Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies

Frontiers in Behavioral Neuroscience

Volumn , Issue OCTOBER 2012, 2012, Pages

  Khamassi, Mehdi ^a,b   Humphries, Mark D ^c,d  

^a UNIVERSITÉ PIERRE ET MARIE CURIE   (France)

^b CNRS   (France)

^c Group for Neural Theory   (France)

^d UNIVERSITY OF MANCHESTER   (United Kingdom)

Author keywords

Action outcome; Basal ganglia; Nucleus accumbens; Reinforcement learning; Stimulus response

Indexed keywords

ALPHA AMINO 3 HYDROXY 5 METHYL 4 ISOXAZOLEPROPIONIC ACID; N METHYL DEXTRO ASPARTIC ACID;

ASSOCIATION; BASAL GANGLION; BOLD SIGNAL; CORPUS STRIATUM; DECISION MAKING; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HABIT; HABITUATION; HIPPOCAMPUS; INSTRUMENTAL CONDITIONING; LEARNING; LEARNING ALGORITHM; LIMBIC CORTEX; LOCOMOTION; MOTIVATION; NEURONAVIGATION; NUCLEUS ACCUMBENS; REINFORCEMENT; REVIEW; SENSITIVITY ANALYSIS; STIMULUS RESPONSE; TASK PERFORMANCE; TAXONOMY;

EID: 84869006148     PISSN: 16625153     EISSN: None     Source Type: Journal    
DOI: 10.3389/fnbeh.2012.00079     Document Type: Review

References (154)

1. Adams, S., Kesner, R. P., Ragozzino, M. E., 2001. Role of the medial and lateral caudate-putamen in mediating an auditory conditional response association. Neurobiology of Learning and Memory 76 (1), 106-116.
2. Albertin, S. V., Mulder, A. B., Tabuchi, E., Zugaro, M. B., Wiener, S. I., 2000. Lesions of the medial shell of the nucleus accumbens impair rats in finding larger rewards, but spare reward-seeking behavior. Behavioral Brain Research 117 (1-2), 173-183.
3. Alexander, G. E., Crutcher, M. D., DeLong, M. R., 1990. Basal ganglia-thalamocortical circuits: parallel substrates for motor, oculomotor, prefrontal and limbic functions. Prog Brain Res 85, 119-146.
4. Arleo, A., Gerstner, W., 2000. Spatial cognition and neuro-mimetic navigation: a model of hippocampal place cell activity. Biol Cybern 83, 287-299.
5. Arleo, A., Rondi-Reig, L., 2007. Multimodal sensory integration and concurrent navigation strategies for spatial cognition in real and artificial organisms. Journal of Integrative Neuroscience 6 (3), 327-366.
6. Atallah, H. E., Lopez-Paniagua, D., Rudy, J. W., O'Reilly, R. C., 2007. Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nat Neurosci 10 (1), 126-131.
7. Balleine, B., 2005. Neural bases of foodseeking: affect, arousal and reward in corticostriatolimbic circuits. Physiol Behav 86 (5), 717-730.
8. Balleine, B., Dickinson, A., 1998. Goaldirected instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 37, 407-419.
9. Balleine, B., Killcross, S., 1994. Effects of ibotenic acid lesions of the nucleus accumbens on instrumental action. Behav Brain Res 65, 181-193.
10. Banquet, J. P., Gaussier, P., Quoy, M., Revel, A., Burnod, Y., 2005. A hierarchy of associations in hippocampo-cortical systems: cognitive maps and navigation strategies. Neural Comput 17 (6), 1339-1384.
11. Barnes, T. D., Kubota, Y., Hu, D., Jin, D. Z., Graybiel, A. M., 2005. Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature 437, 1158-1161.
12. Bauter, M. R., Brockel, B. J., Pankevich, D. E., Virgolini, M. B., Cory-Slechta, D. A., 2003. Glutamate and dopamine in nucleus accumbens core and shell: sequence learning versus performance. Neurotoxicology 24, 227-243.
13. Bayer, H. M., Glimcher, P. W., 2005. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129-141.
14. Berke, J. D., Breck, J. T., Eichenbaum, H., 2009. Striatal versus hippocampal representations during win-stay maze performance. J Neurophysiol 101, 1575-1587.
15. Bolam, J. P., Bergman, H., Graybiel, A. M., Kimura, M., Plenz, D., Seung, H. S., Surmeier, D. J., Wickens, J. R., 2006. Microcircuits in the striatum. In: Grillner, S., Graybiel, A. M. (Eds.), Microcircuits: the interface between neurons and global brain function. MIT Press, Cambridge, MA, pp. 165-190.
16. Bornstein, A. M., Daw, N. D., 2011. Multiplicity of control in the basal ganglia: computational roles of striatal subregions. Curr Opin Neurobiol 21, 374-380.
17. Bornstein, A. M., Daw, N. D., 2012. Dissociating hippocampal and striatal contributions to sequential prediction learning. European Journal of Neuroscience 35, 1011-1023.
18. Botreau, F., Gisquet-Verrier, P., 2010. Re-thinking the role of the dorsal striatum in egocentric/response strategy. Front Behav Neurosci 4, 7.
19. Brown, L., Sharp, F., 1995. Metabolic mapping of rat striatum: Somatotopic organization of sensorimotor activity. Brain Research 686, 207-222.
20. Burgess, N., Recce, M., O'Keefe, J., 1994. A model of hippocampal function. Neural Networks 7 (6/7), 1065-1081.
21. Chang, Q., Gold, P. E., 2003. Switching memory systems during learning: changes in patterns of brain acetylcholine release in the hippocampus and striatum in rats. J Neurosci 23, 3001-3005.
22. Chang, Q., Gold, P. E., 2004. Inactivation of dorsolateral striatum impairs acquisition of response learning in cue-deficient, but not cue-available, conditions. Behav Neurosci 118, 383-388.
23. Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., Uchida, N., 2012. Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482, 85-88.
24. Corbit, L. H., Balleine, B. W., 2011. The general and outcome-specific forms of pavlovian-instrumental transfer are differentially mediated by the nucleus accumbens core and shell. J Neurosci 31 (33), 11786-11794.
25. Corbit, L. H., Muir, J. L., Balleine, B. W., 2001. The role of the nucleus accumbens in instrumental conditioning: Evidence of a functional dissociation between accumbens core and shell. J Neurosci 21, 3251-3260.
26. Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., Dolan, R. J., 2011. Model-based influences on humans' choices and striatal prediction errors. Neuron 69, 1204-1215.
27. Daw, N. D., Niv, Y., Dayan, P., 2005. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci 8, 1704-1711.
28. Dayan, P., Balleine, B., 2002. Reward, motivation, and reinforcement learning. Neuron 36 (2), 285-298.
29. Dayan, P., Niv, Y., 2008. Reinforcement learning: The good, the bad and the ugly. Curr Opin Neurobiol 18, 185-196.
30. De Leonibus, E., Costantini, V. J. A., Massaro, A., Mandolesi, G., Vanni, V., Luvisetto, S., Pavone, F., Oliverio, A., Mele, A., 2011. Cognitive and neural determinants of response strategy in the dualsolution plus-maze task. Learn Mem 18, 241-244.
31. De Leonibus, E., Oliverio, A., Mele, A., 2005. A study on the role of the dorsal striatum and the nucleus accumbens in allocentric and egocentric spatial memory consolidation. Learn Mem 12 (5), 491-503.
32. Devan, B. D., White, N. M., 1999. Parallel information processing in the dorsal striatum: Relation to hippocampal function. J Neurosci 19 (7), 2789-2798.
33. Dickinson, A., 1980. Contemporary animal learning theory. Cambridge: Cambridge University Press.
34. Dickinson, A., 1985. Actions and habits: The development of behavioural autonomy. Phil Trans Roy Soc B: Biol Sci 308 (1135), 67-78.
35. Dollé, L., Sheynikhovich, D., Girard, B., Chavarriaga, R., Guillot, A., 2010. Path planning versus cue responding: a bio-inspired model of switching between navigation strategies. Biol Cybern 103, 299-317.
36. Euston, D. R., Tatsuno, M., McNaughton, B. L., 2007. Fast-forward playback of recent memory sequences in prefrontal cortex during sleep. Science 318 (5853), 1147-1150.
37. Faure, A., Haberland, U., Cond, F., Massioui, N. E., 2005. Lesion to the nigrostriatal dopamine system disrupts stimulus-response habit formation. J Neurosci 25, 2771-2780.
38. Foster, D., Morris, R., Dayan, P., 2000. Models of hippocampally dependent navigation using the temporal difference learning rule. Hippocampus 10, 1-16.
39. Foster, D. J., Wilson, M. A., 2006. Reverse replay of behavioural sequences in hippocampal place cells during the awake state. Nature 440, 680-683.
40. Franz, M. O., Mallot, H. A., 2000. Biomimetic robot navigation. Robotics and Autonomous Systems 30, 133-153.
41. Gallistel, C. R., 1990. The organisation of learning. MIT Press, Cambridge.
42. Gaussier, P., Revel, A., Banquet, J. P., Babeau, V., 2002. From view cells and place cells to cognitive map learning: processing stages of the hippocampal system. Biol Cybern 86, 15-28.
43. Girard, B., Tabareau, N., Pham, Q. C., Berthoz, A., Slotine, J. J., 2008. Where neuroscience and dynamic system theory meet autonomous robotics: a contracting basal ganglia model for action selection. Neural Networks 21 (4), 628-641.
44. Glascher, J., Daw, N., Dayan, P., O'Doherty, J. P., 2010. States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66, 585-595.
45. Gorny, J. H., Gorny, B., Wallace, D. G., Whishaw, I. Q., 2002. Fimbria-fornix lesions disrupt the dead reckoning (homing) component of exploratory behavior in mice. Learn Mem 9, 387-394.
46. Graybiel, A. M., 1998. The basal ganglia and chunking of action repertoires. Neurobiol Learn Mem 70 (1-2), 119-36.
47. Groenewegen, H. J., Wright, C. I., Beijer, A. V., 1996. The nucleus accumbens: gateway for limbic structures to reach the motor system? Prog Brain Res 107, 485-511.
48. Gruber, A. J., Hussain, R. J., O'Donnell, P., 2009. The nucleus accumbens: a switchboard for goaldirected behaviors. PLoS One 4, e5062.
49. Gupta, A. S., van der Meer, M. A., Touretzky, D. S., Redish, A. D., 2010. Hippocampal replay is not a simple function of experience. Neuron 65, 695-705.
50. Haber, S. N., 2003. The primate basal ganglia: parallel and integrative networks. J Chem Neuroanat 26, 317-330.
51. Haber, S. N., Fudge, J. L., McFarland, N. R., 2000. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J Neurosci 20, 2369-2382.
52. Hannesson, D. K., Skelton, R. W., 1998. Recovery of spatial performance in the morris water maze following bilateral transection of the fimbria/fornix in rats. Behavioural Brain Research 90 (1), 35-56.
53. Hartley, T., Burgess, N., 2005. Complementary memory systems: competition, cooperation and compensation. Trends in Neurosciences 28 (4), 169-170.
54. Hasselmo, M., 2005. A model of prefrontal cortical mechanisms for goal-directed behavior. J Cogn Neurosci 17 (7), 1115-1129.
55. Heimer, L., Alheid, G. F., de Olmos, J. S., Groenewegen, H., Haber, S., E., H. R., Zahm, D., 1997. The accumbens: beyond the coreshell dichotomy. Journal of Neuropsychiatry 9, 354-381.
56. Hok, V., Save, E., LenckSantini, P. P., Poucet, B., 2005. Coding for spatial goals in the prelimbic/infralimbic area of the rat frontal cortex. PNAS 102 (12), 4602-4607.
57. Honzik, C. H., 1936. The sensory basis of maze learning in rats. Comp Psychol Monog 13, 113.
58. Houk, J. C., Wise, S. P., 1995. Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: their role in planning and controlling action. Cereb Cortex 5, 95-110.
59. Humphries, M. D., Khamassi, M., Gurney, K., 2012. Dopaminergic control of the exploration-exploitation trade-off via the basal ganglia. Frontiers in Neuroscience 6, 9.
60. Humphries, M. D., Prescott, T. J., 2010. The ventral basal ganglia, a selection mechanism at the crossroads of space, strategy, and reward. Prog Neurobiol 90, 385-417.
61. Humphries, M. D., Stewart, R. D., Gurney, K. N., 2006. A physiologically plausible model of action selection and oscillatory activity in the basal ganglia. J Neurosci 26, 12921-12942.
62. Ikemoto, S., 2002. Ventral striatal anatomy of locomotor activity induced by cocaine, (d)amphetamine, dopamine and d1/d2 agonists. Neuroscience 113, 939-955.
63. Ito, M., Doya, K., 2011. Multiple representations and algorithms for reinforcement learning in the cortico-basal ganglia circuit. Curr Opin Neurobiol 21, 368-373.
64. Jacobson, T. K., Gruenbaum, B. F., Markus, E. J., 2012. Extensive training and hippocampus or striatum lesions: effect on place and response strategies. Physiol Behav 105, 645-652.
65. Jadhav, S. P., Kemere, C., German, P. W., Frank, L. M., 2012. Awake hippocampal sharp-wave ripples support spatial memory. Science 336, 1454-1458.
66. Joel, D., Niv, Y., Ruppin, E., 2002. Actor-critic models of the basal ganglia: new anatomical and computational perspectives. Neural Netw 15, 535-547.
67. Joel, D., Weiner, I., 1994. The organization of the basal ganglia-thalamocortical circuits: open interconnected rather than closed segregated. Neuroscience 63, 363-379.
68. Joel, D., Weiner, I., 2000. The connections of the dopaminergic system with the striatum in rats and primates: an analysis with respect to the functional and compartmental organization of the striatum. Neuroscience 96, 451-474.
69. Jog, M. S., Kubota, Y., Connolly, C. I., Hillegaart, V., Graybiel, A. M., 1999. Building neural representations of habits. Science 286, 1745-1749.
70. Johnson, A., Redish, A. D., 2005. Hippocampal replay contributes to within session learning in a temporal difference reinforcement learning model. Neural Netw 18, 1163-1171.
71. Johnson, A., Redish, A. D., 2007. Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. Journal of Neuroscience 27 (45), 12176-12189.
72. Kelley, A. E., 1999. Neural integrative activities of nucleus accumbens subregions in relation to learning and motivation. Psychobiology 27, 198-213.
73. Khamassi, M., 2007. Complementary roles of the rat prefrontal cortex and striatum in reward-based learning and shifting navigation strategies. Ph.D. thesis, Université Pierre et Marie Curie.
74. Khamassi, M., Lacheze, L., Girard, B., Berthoz, A., Guillot, A., 2005. Actor-critic models of reinforcement learning in the basal ganglia: from natural to arificial rats. Adaptive Behavior 13, 131-148.
75. Khamassi, M., Mulder, A. B., Tabuchi, E., Douchamps, V., Wiener, S. I., 2008. Anticipatory reward signals in ventral striatal neurons of behaving rats. Eur J Neurosci 28, 1849-1866.
76. Kim, S. M., Frank, L. M., 2009. Hippocampal lesions impair rapid learning of a continuous spatial alternation task. PLoS ONE 4 (5), e5494.
77. Kimchi, E. Y., Laubach, M., 2009. Dynamic encoding of action selection by the medial striatum. J Neurosci 29, 3148-3159.
78. Kimchi, E. Y., Torregrossa, M. M., Taylor, J. R., Laubach, M., 2009. Neuronal correlates of instrumental learning in the dorsal striatum. J Neurophysiol 102, 475-489.
79. Krech, D., 1932. The genesis of hypotheses in rats. Publications in Psychology 6, 45-64.
80. Lansink, C. S., Goltstein, P. M., Lankelma, J. V., McNaughton, B. L., Pennartz, C. M. A., 2009. Hippocampus leads ventral striatum in replay of place-reward information. PLoS Biol 7, e1000173.
81. Leblois, A., Boraud, T., Meissner, W., Bergman, H., Hansel, D., 2006. Competition between feedback loops underlies normal and pathological dynamics in the basal ganglia. J Neurosci 26, 3567-3583.
82. Lex, B., Sommer, S., Hauber, W., 2011. The role of dopamine in the dorsomedial striatum in place and response learning. Neuroscience 172, 212-218.
83. Martel, G., Blanchard, J., Mons, N., Gastambide, F., Micheau, J., Guillou, J., 2007. Dynamic interplays between memory systems depend on practice: The hippocampus is not always the first to provide solution. Neuroscience 150, 743-753.
84. Martinet, L.-E., Sheynikhovich, D., Benchenane, K., Arleo, A., 2011. Spatial learning and action planning in a prefrontal cortical network model. PLoS Comput Biol 7, e1002045.
85. Maurin, Y., Banrezes, B., Menetrey, A., Mailly, P., Deniau, J. M., 1999. Three-dimensional distribution of nigrostriatal neurons in the rat: relation to the topography of striatonigral projections. Neuroscience 91, 891-909.
86. McDannald, M. A., Lucantonio, F., Burke, K. A., Niv, Y., Schoenbaum, G., 2011. Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. J Neurosci 31, 2700-2705.
87. Middleton, F. A., Strick, P. L., 2000. Basal ganglia and cerebellar loops: motor and cognitive circuits. Brain Res Brain Res Rev 31 (2-3), 236-250.
88. Mink, J. W., 1996. The basal ganglia: Focused selection and inhibition of competing motor programs. Prog Neurobiol 50, 381-425.
89. Mogenson, G. J., Jones, D. L., Yim, C. Y., 1980. From motivation to action: functional interface between the limbic system and the motor system. Prog Neurobiol 14 (2-3), 69-97.
90. Morris, R. G. M., 5 1981. Spatial localization does not require the presence of local cues. Learning and Motivation 12 (2), 239-260.
91. Moussa, R., Poucet, B., Amalric, M., Sargolini, F., 2011. Contributions of dorsal striatal subregions to spatial alternation behavior. Learning and Memory 18, 444-451.
92. Mulder, A. B., Tabuchi, E., Wiener, S. I., 2004. Neurons in hippocampal afferent zones of rat striatum parse routes into multi-pace segments during maze navigation. Eur J Neurosci 19, 1923-1932.
93. Nicola, S. M., 2007. The nucleus accumbens as part of a basal ganglia action selection circuit. Psychopharmacology (Berl) 191, 521-550.
94. O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., Dolan, R. J., 2004. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452-454.
95. O'Keefe, J., Nadel, L., 1978. The hippocampus as a cognitive map. Clarendon Press; Oxford University Press, Oxford: New York:.
96. Packard, M., 1999. Glutamate infused posttraining into the hippocampus or caudateputamen differentially strengthens place and response learning. PNAS 96 (22), 12881-12886.
97. Packard, M., McGaugh, J., 1992. Double dissociation of fornix and caudate nucleus lesions on acquisition of two water maze tasks: Further evidence for multiple memory systems. Behavioural Neuroscience 106, 439-446.
98. Packard, M., McGaugh, J., 1996. Inactivation of hippocampus or caudate nucleus with lidocaine differentially affects the expression of place and response learning. Neurobiology of Learning and Memory 65 (1), 65-72.
99. Packard, M. G., Hirsh, R., White, N. M., 1989. Differential effects of fornix and caudate nucleus lesions on two radial maze tasks: evidence for multiple memory systems. Journal of Neuroscience 9 (5), 1465-1472.
100. Packard, M. G., Knowlton, B. J., 2002. Learning and memory functions of the basal ganglia. Annual Review of Neuroscience 25, 563-593.
101. Pearce, J. M., Roberts, A. D., Good, M., 1998. Hippocampal lesions disrupt navigation based on cognitive maps but not heading vectors. Nature 396 (6706), 75-77.
102. Pennartz, C. M., Groenewegen, H. J., da Silva, F. H. L., 1994. The nucleus accumbens as a complex of functionally distinct neuronal ensembles: an integration of behavioural, electrophysiological and anatomical data. Prog Neurobiol 42, 719-761.
103. Penner, M. R., Mizumori, S. J. Y., 2012. Neural systems analysis of decision making during goaldirected navigation. Prog Neurobiol 96, 96-135.
104. Peoples, L. L., Gee, F., Bibi, R., West, M. O., 1998. Phasic firing time locked to cocaine self-infusion and locomotion: dissociable firing patterns of single nucleus accumbens neurons in the rat. J Neurosci 18, 7588-7598.
105. Ploeger, G. E., Spruijt, B. M., Cools, A. R., 1994. Spatial localization in the morris water maze in rats: acquisition is affected by intraaccumbens injections of the dopaminergic antagonist haloperidol. Behavioural Neuroscience 108 (5), 927-934.
106. Potegal, M., 1972. The caudate nucleus egocentric localization system. Acta Neurobiol Exp 32, 479-494.
107. Poucet, B., Lenck-Santini, P. P., Hok, V., Save, E., Banquet, J. P., Gaussier, P., Muller, R. U., 2004. Spatial navigation and hippocampal place cell firing: the problem of goal encoding. Rev Neurosci 15, 89-107.
108. Pych, J. C., Chang, Q., Colon-Rivera, C., Gold, P. E., 2005. Acetylcholine release in hippocampus and striatum during testing on a rewarded spontaneous alternation task. Neurobiology of Learning and Memory 84, 93-101.
109. Ragozzino, M. E., Choi, D., 2004. Dynamic changes in acetylcholine output in the medial striatum during place reversal learning. Learn. Mem. 11, 70-77.
110. Redgrave, P., Prescott, T. J., Gurney, K., 1999. The basal ganglia: A vertebrate solution to the selection problem? Neuroscience 89, 1009-1023.
111. Redish, A. D., 1999. Beyond the Cognitive Map: From Place Cells to Episodic Memory. MIT Press, Cambridge.
112. Redish, A. D., Touretzky, D. S., 1997. Cognitive maps beyond the hippocampus. Hippocampus 7, 15-35.
113. Redish, A. D., Touretzky, D. S., 1998. The role of the hippocampus in solving the morris water maze. Neural Computation 10, 73-111.
114. Reynolds, J. N., Hyland, B. I., Wickens, J. R., 1957. Discrimination of cues in mazes: A resolution of the placevs.response question. Psychological Review 64, 217-228.
115. Reynolds, J. N., Hyland, B. I., Wickens, J. R., 2001. A cellular mechanism of reward-related learning. Nature 413, 67-70.
116. Reynolds, S. M., Berridge, K. C., 2003. Glutamate motivational ensembles in nucleus accumbens: rostrocaudal shell gradients of fear and feeding. Eur J Neurosci 17, 2187-2200.
117. Rudy, J. W., 2009. Context representations, context functions, and the parahippocampal-hippocampal system. Learn Mem 16, 573-585.
118. Sargolini, F., Florian, C., Oliverio, A., Mele, A., Roullet, P., 2003. Differential involvement of NMDA and AMPA receptors within the nucleus accumbens in consolidation of information necessary for place navigation and guidance strategy of mice. Learn Mem 10, 285-292.
119. Schmitzer-Torbert, N. C., Redish, A. D., 2008. Task-dependent encoding of space and events by striatal neurons is dependent on neural subtype. Neuroscience 153, 349-360.
120. Schultz, W., Dayan, P., Montague, P. R., 1997. A neural substrate of prediction and reward. Science 275, 1593-1599.
121. Setlow, B., McGaugh, J., 1998. Sulpiride infused into the nucleus accumbens posttraining impairs memory of spatial water maze training. Behavioural Neuroscience 112 (3), 603-610.
122. Setlow, B., Schoenbaum, G., Gallagher, M., 2003. Neural encoding in ventral striatum during olfactory discrimination learning. Neuron 38, 625-636.
123. Shen, W., Flajolet, M., Greengard, P., Surmeier, D. J., 2008. Dichotomous dopaminergic control of striatal synaptic plasticity. Science 321, 848-851.
124. Shibata, R., Mulder, A. B., Trullier, O., Wiener, S. I., 2001. Position sensitivity in phasically discharging nucleus accumbens neurons of rats alternating between tasks requiring complementary types of spatial cues. Neuroscience 108, 391-411.
125. Smith, D. M., Mizumori, S. J. Y., 2006. Hippocampal place cells, context, and episodic memory. Hippocampus 16, 716-729.
126. Sutherland, R. J., Hamilton, D. A., 2004. Rodent spatial navigation: at the crossroads of cognition and movement. Neuroscience Biobehavioural Reviews 28 (7), 687-697.
127. Sutherland, R. J., Rodriguez, A. J., 1989. The role of the fornix/fimbria and some related subcortical structures in place learning and memory. Behavioural Brain Research 32 (3), 265-277.
128. Sutton, R. S., Barto, A. G., 1998. Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press.
129. Taha, S. A., Nicola, S. M., Fields, H. L., 2007. Cue-evoked encoding of movement planning and execution in the rat nucleus accumbens. J Physiol 584, 801-818.
130. Tang, C., Pawlak, A. P., Prokopenko, V., West, M. O., 2007. Changes in activity of the striatum during formation of a motor habit. Eur J Neurosci 25, 1212-1227.
131. Tepper, J. M., Koos, T., Wilson, C. J., 2004. GABAergic microcircuits in the neostriatum. Trends Neurosci 27, 662-669.
132. Thorn, C. A., Atallah, H., Howe, M., Graybiel, A. M., 2010. Differential dynamics of activity changes in dorsolateral and dorsomedial striatal loops during learning. Neuron 66, 781-795.
133. Tolman, E. C., 1948. Cognitive maps in rats and men. Psychological Reviews 55, 189-208.
134. Trullier, O., Wiener, S., Berthoz, A., Meyer, J.-A., 1997. Biologically-based artificial navigation systems: Review and prospects. Progress in Neurobiology 51, 483-544.
135. Uylings, H. B. M., Groenewegen, H. J., Kolb, B., 2003. Do rats have a prefrontal cortex? Behav Brain Res 146, 3-17.
136. van der Meer, M. A. A., Johnson, A., Schmitzer-Torbert, N. C., Redish, A. D., 2010. Triple dissociation of information processing in dorsal striatum, ventral striatum, and hippocampus on a learned spatial decision task. Neuron 67, 25-32.
137. van der Meer, M. A. A., Kurth-Nelson, Z., Redish, A. D., 2012. Information processing in decisionmaking systems. The Neuroscientist XX (X), 1-18.
138. van der Meer, M. A. A., Redish, A. D., 2009. Covert expectation-of-reward in rat ventral striatum at decision points. Front Integr Neurosci 3, 1.
139. van der Meer, M. A. A., Redish, A. D., 2010. Theta phase precession in rat ventral striatum links place and reward information. J Neurosci 31 (8), 2843-2854.
140. van der Meer, M. A. A., Redish, A. D., 2011. Ventral striatum: a critical look at models of learning and evaluation. Current Opinion in Neurobiology 21 (3), 387-392.
141. Voorn, P., Vanderschuren, L. J., Groenewegen, H. J., Robbins, T. W., Pennartz, C. M., 2004. Putting a spin on the dorsal-ventral divide of the striatum. Trends Neurosci 27, 468-474.
142. Watabe-Uchida, M., Zhu, L., Ogawa, S. K., Vamanrao, A., Uchida, N., 2012. Whole-brain mapping of direct inputs to midbrain dopamine neurons. Neuron 74, 858-873.
143. Whishaw, I. Q., Cassel, J. C., Jarrad, L. E., 1995. Rats with fimbria-fornix lesions display a place response in a swimming pool: a dissociation between getting there and knowing where. J Neurosci 15, 5779-5788.
144. Whishaw, I. Q., Mittleman, G., Bunch, S. T., Dunnett, S. B., 1987. Impairments in the acquisition, retention and selection of spatial navigation strategies after medial caudate-putamen lesions in rats. Behav Brain Res 24, 125-138.
145. White, N. M., McDonald, R. J., 2002. Multiple parallel memory systems in the brain of the rat. Neurobiol Learn Mem 77, 125-184.
146. Wiener, S. I., 1993. Spatial and behavioral correlates of striatal neurons in rats performing a self-initiated navigation task. J Neurosci 13, 3802-3817.
147. Wiener, S. I., Paul, C. A., Eichenbaum, H., 1989. Spatial and behavioral correlates of hippocampal neuronal activity. J Neurosci 9, 2737-2763.
148. Willingham, D. B., 1998. What differentiates declarative and procedural memories: Reply to cohen, poldrack, and eichenbaum (1997). Memory 6 (6), 689-699.
149. Yin, H. H., Knowlton, B. J., 2004. Contributions of striatal subregions to place and response learning. Learn Mem 11, 459-463.
150. Yin, H. H., Knowlton, B. J., 2006. The role of the basal ganglia in habit formation. Nat Rev Neurosci 7, 464-476.
151. Yin, H. H., Knowlton, B. J., Balleine, B. W., 2004. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur J Neurosci 19, 181-189.
152. Yin, H. H., Knowlton, B. J., Balleine, B. W., 2005a. Blockade of NMDA receptors in the dorsomedial striatum prevents action-outcome learning in instrumental conditioning. Eur J Neurosci 22, 505-512.
153. Yin, H. H., Ostlund, S. B., Balleine, B. W., 2008. Reward-guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico-basal ganglia networks. Eur J Neurosci 28, 1437-1448.
154. Yin, H. H., Ostlund, S. B., Knowlton, B. J., Balleine, B. W., 2005b. The role of the dorsomedial striatum in instrumental conditioning. Eur J Neurosci 22, 513-523.