Integrating hippocampus and striatum in decision-making

Current Opinion in Neurobiology

Volumn 17, Issue 6, 2007, Pages 692-697

  Johnson, Adam ^a   van der Meer, Matthijs AA ^a   Redish, A David ^a  

^a UNIVERSITY OF MINNESOTA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BRAIN FUNCTION; COGNITION; CORPUS STRIATUM; DECISION MAKING; HIPPOCAMPUS; NETWORK LEARNING; NEUROPHYSIOLOGY; NONHUMAN; ORBITAL CORTEX; PREFRONTAL CORTEX; PRIORITY JOURNAL; REVIEW; STIMULUS RESPONSE;

ANIMALS; CORPUS STRIATUM; DECISION MAKING; HIPPOCAMPUS; HUMANS; NEURAL PATHWAYS;

EID: 40849087850     PISSN: 09594388     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.conb.2008.01.003     Document Type: Review

References (67)

1. Sutton R.S., and Barto A.G. Reinforcement Learning: An Introduction (1998), MIT Press, Cambridge, MA
2. In: Schacter D.L., and Tulving E. (Eds). Memory Systems 1994 (1994), MIT Press
3. Squire L.R. Memory systems of the brain: a brief history and current perspective. Neurobiol Learn Mem 82 (2004) 171-177
4. Poldrack R.A., and Packard M.G. Competition among multiple memory systems: converging evidence from animal and human studies. Neuropsychologia 41 (2003) 245-251
5. Buckner R.L., and Carroll D.C. Self-projection and the brain. Trends Cognitive Sci 11 (2007) 49-57. This article brings together several areas of research including prospection, episodic memory, theory of mind and navigation in terms of self-projection. The authors highlight the functional use of a core frontal-temporal network within self-projection tasks and suggest that such function requires a similar set of processes.
6. Hikosaka O., Nakahara H., Rand M.K., Sakai K., Lu X., Nakamura K., Miyachi S., and Doya K. Parallel neural networks for learning sequential procedures. Trends Neurosci 22 (1999) 464-471
7. O'Keefe J., and Nadel L. The Hippocampus as a Cognitive Map (1978), Clarendon Press, Oxford
8. Redish A.D. Beyond the Cognitive Map: From Place Cells to Episodic Memory (1999), MIT Press, Cambridge, MA
9. O'Reilly R.C., and Munakata Y. Computational Explorations in Cognitive Neuroscience (2000), MIT Press, Cambridge, MA
10. Packard M.G., and McGaugh J.L. Inactivation of hippocampus or caudate nucleus with lidocaine differentially affects expression of place and response learning. Neurobiol Learn Mem 65 (1996) 65-72
11. Tolman E.C. Cognitive maps in rats and men. Psychol Rev 55 (1948) 189-208
12. Tulving E. Episodic memory: from mind to brain. Annu Rev Psychol 53 (2002) 1-25
13. Dayan P., and Balleine B.W. Reward, motivation, and reinforcement learning. Neuron 36 (2002) 285-298
14. Koene R.A., Gorchetchnikov A., Cannon R.C., and Hasselmo M.E. Modeling goal-directed spatial navigation in the rat based on physiological data from the hippocampal formation. Neural Netw 16 (2003) 577-584
15. Daw N.D., Niv Y., and Dayan P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci 8 (2005) 1704-1711. The authors develop a computational framework for examining the differential contributions of multiple memory systems to decision-making across learning. The model provides a basis for understanding flexible behaviors such as devaluation and planning in terms of multiple memory systems.
16. Zilli EA, Hasselmo ME: Modeling the role of working memory and episodic memory in behavioral tasks. Hippocampus 2008, 18:193-209. This paper provides an early, integrated model of flexible decision-making by incorporating active working memory and episodic memory models into reinforcement learning.
17. Newell A: Unified Theories of Cognition. Harvard University Press, Cambridge MA, 1990.
18. Schacter D.L., Addis D.R., and Buckner R.L. Remembering the past to imagine the future: the prospective brain. Nat Rev Neurosci 8 (2007) 657-661
19. Tolman E.C. The determiners of behavior at a choice point. Psychol Rev 45 (1938) 1-41
20. Gallistel C.R., Fairhurst S., and Balsam P. Inaugural article: the learning curve: implications of a quantitative analysis. PNAS USA 101 (2004) 13124-13131
21. Hu D., and Amsel A. A simple test of the vicarious trial-and-error hypothesis of hippocampal function. PNAS 92 (1995) 5506-5509
22. Hu D., Xu X., and Gonzalez-Lima F. Vicarious trial-and-error behavior and hippocampal cytochrome oxidase activity during Y-maze discrimination learning in the rat. Int J Neurosci 116 (2006) 265-280
23. Hassabis D., Kumaran D., Vann S.D., and Maguire E.A. Patients with hippocampal amnesia cannot imagine new experiences. PNAS 104 (2007) 1726-1731. In this study patients with bilateral damage to the hippocampus displayed deficits constructing newly imagined experiences. The authors suggest that constructing imagined experiences, like episodic memory, are dependent hippocampal function.
24. Nadel L., and Eichenbaum H. Introduction to the special issue on place cells. Hippocampus 9 (1999) 341-345
25. Mizumori S.J.Y. Hippocampal place fields: a neural code for episodic memory?. Hippocampus 16 (2006) 685-690
26. O'Keefe J., and Speakman A. Single unit activity in the rat hippocampus during a spatial memory task. Exp Brain Res 68 (1987) 1-27
27. Lenck-Santini P.P., Muller R.U., Save E., and Poucet B. Relationships between place cell firing fields and navigational decisions by rats. J Neurosci 22 (2002) 9035-9047
28. Rosenzweig E.S., and Barnes C.A. Impact of aging on hippocampal function: plasticity, network dynamics, and cognition. Prog Neurobiol 69 (2003) 143-179
29. Johnson A., and Redish A.D. Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J Neurosci (2007). Spatial representations were decoded from the activity of simultaneously recorded neural ensembles within the CA3 region of the hippocampus. At decision-points, decoded representations encoded non-local positions forward of the animal into each arm. Forward-shifted neural representations provide a potential mechanism for consideration of future possibilities.
30. Schoenbaum G., Roesch M., and Stalnaker T.A. Orbitofrontal cortex, decision making, and drug addiction. Trends Neurosci (2006)
31. Padoa-Schioppa C., and Assad J.A. Neurons in the orbitofrontal cortex encode economic value. Nature 441 (2006) 223-226
32. Kalivas P.W., and Volkow N.D. The neural basis of addiction: a pathology of motivation and choice. Am J Psychiatry 162 (2005) 1403-1413
33. Lavoie A.M., and Mizumori S.J.Y. Spatial-, movement- and reward-sensitive discharge by medial ventral striatum neurons in rats. Brain Res 638 (1994) 157-168
34. O'Doherty J., Dayan P., Schultz J., Deichmann R., Friston K., and Dolan R.J. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304 (2004) 452-454
35. Yin H.H., and Knowlton B.J. The role of the basal ganglia in habit formation. Nat Rev Neurosci 7 (2006) 464-476. This review highlights the differential contributions of dorsolateral and dorsomedial striatal subregions on the classic place versus response plus-maze task.
36. DeCoteau W.E., Thorn C., Gibson D.J., Courtemanche R., Mitra P., Kubota Y., and Graybiel A.M. Learning-related coordination of striatal and hippocampal theta rhythms during acquisition of a procedural maze task. PNAS 104 (2007) 5644-5649
37. Martin P.D., and Ono T. Effects of reward anticipation, reward presentation, and spatial parameters on the firing of single neurons recorded in the subiculum and nucleus accumbens of freely moving rats. Behav Brain Res 116 (2000) 23-38
38. Pennartz CMA, Lee E., Verheul J., Lipa P., Barnes C.A., and McNaughton B.L. The ventral striatum in off-line processing: ensemble reactivation during sleep and modulation by hippocampal ripples. J Neurosci 24 (2004) 6446-6456
39. Ramus SJ, Davis JB, Donahue RJ, Discenza CB, Waite AA: Interactions between the orbitofrontal cortex and hippocampal memory system during the storage of long-term memory. Ann NY Acad Sci 2007, 1121:216-231.
40. Schultz W., Apicella P., Scarnati E., and Ljungberg T. Neuronal activity in monkey ventral striatum related to the expectation of reward. J Neurosci 12 (1992) 4595-4610
41. Yun I.A., Wakabayashi K.T., Fields H.L., and Nicola S.M. The ventral tegmental area is required for the behavioral and nucleus accumbens neuronal firing responses to incentive cues. J Neurosci 24 (2004) 2923-2933
42. van Duuren E., Escámez F.A., Joosten R.N., Visser R., Mulder A.B., and Pennartz C.M. Neural coding of reward magnitude in the orbitofrontal cortex of the rat during a five-odor olfactory discrimination task. Learn Mem 14 (2007) 446-456
43. Schoenbaum G., and Roesch M. Orbitofrontal cortex, associative learning, and expectancies. Neuron 47 (2005) 633-636
44. Graybiel A.M. The basal ganglia and chunking of action repertoires. Neurobiol Learn Mem 70 (1998) 119-136
45. Kesner R.P., Bolland B.L., and Dakis M. Memory for spatial locations, motor responses, and objects: triple dissociation among the hippocampus, caudate nucleus, and extrastriate visual cortex. Exp Brain Res 93 (1993) 462-470
46. McDonald R.J., and White N.M. Parallel information processing in the water maze: evidence for independent memory systems involving dorsal striatum and hippocampus. Behav Neural Biol 61 (1994) 260-270
47. Gold P. Coordination of multiple memory systems. Neurobiol Learn Mem 82 (2004) 230-242
48. Jog M.S., Kubota Y., Connolly C.I., Hillegaart V., and Graybiel A.M. Building neural representations of habits. Science 286 (1999) 1746-1749
49. Schmitzer-Torbert N.C., and Redish A.D. Neuronal activity in the rodent dorsal striatum in sequential navigation: separation of spatial and reward responses on the multiple-T task. J Neurophysiol 91 (2004) 2259-2272
50. Barnes T.D., Kubota Y., Hu D., Jin D.Z., and Graybiel A.M. Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature 437 (2005) 1158-1161. A clear illustration of slowly-developing dorsolateral striatal firing patterns thought to support habit learning.
51. Samejima K., Ueda Y., Doya K., and Kimura M. Representation of action-specific reward values in the striatum. Science 310 (2005) 1337-1340. This study shows neurons in the primate caudate and putamen encode the value of actions, consistent with cache-based reinforcement learning models.
52. Redish A.D., Jensen S., Johnson A., and Kurth-Nelson Z. Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. Psychol Rev 114 (2007) 784-805
53. Devan B.D., and White N.M. Parallel information processing in the dorsal striatum: relation to hippocampal function. J Neurosci 19 (1999) 2789-2798
54. Yin H.H., and Knowlton B.J. Contributions of striatal subregions to place and response learning. Learn Mem 11 (2004) 459-463
55. Ragozzino M.E., Jih J., and Tzavos A. Involvement of the dorsomedial striatum in behavioral flexibility: role of muscarinic cholinergic receptors. Brain Res 953 (2002) 205-214
56. Yin H.H., Knowlton B.J., and Balleine B.W. Blockade of NMDA receptors in the dorsomedial striatum prevents action-outcome learning in instrumental conditioning. Eur J Neurosci 22 (2005) 505-512
57. Yin H.H., Ostlund S.B., Knowlton B.J., and Balleine B.W. The role of the dorsomedial striatum in instrumental conditioning. Eur J Neurosci 22 (2005) 513-523
58. Balleine B.W., and Dickinson A. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 37 (1998) 407-419
59. Adams S., Kesner R.P., and Ragozzino M.E. Role of the medial and lateral caudate-putamen in mediating an auditory conditional response association. Neurobiol Learn Mem 76 (2001) 106-116
60. Zahm D.S., and Brog J.S. On the significance of subterritories in the "accumbens" part of the rat ventral striatum. Neuroscience 50 (1992) 751-767
61. Pennartz C.M.A., Groenewegen H.J., and Lopes da Silva F.H. The nucleus accumbens as a complex of functionally distinct neuronal ensembles: an integration of behavioural, electrophysiological, and anatomical data. Prog Neurobiol 42 (1994) 719-761
62. Baldo B.A., and Kelley A.E. Discrete neurochemical coding of distinguishable motivational processes: insights from nucleus accumbens control of feeding. Psychopharmacology 191 (2007) 439-459
63. Corbit L.H., Muir J.L., and Balleine B.W. The role of the nucleus accumbens in instrumental conditioning: evidence of a functional dissociation between accumbens core and shell. J Neurosci 21 (2001) 3251-3260
64. Kalivas P.W., Volkow N., and Seamans J. Unmanageable motivation in addiction: a pathology in prefrontal-accumbens glutamate transmission. Neuron 45 (2005) 647-650
65. Miyazaki K., Mogi E., Araki N., and Matsumoto G. Reward-quality dependent anticipation in rat nucleus accumbens. NeuroReport 9 (1998) 3943-3948
66. German P.W., and Fields H.L. Rat nucleus accumbens neurons persistently encode locations associated with morphine reward. J Neurophysiol 97 (2007) 2094-2106
67. Mogenson G.J., Jones D.L., and Yim C.Y. From motivation to action: functional interface between the limbic system and the motor system. Prog Neurobiol 14 (1980) 69-97