Sequence learning and the role of the hippocampus in rodent navigation

Current Opinion in Neurobiology

Volumn 22, Issue 2, 2012, Pages 294-300

  Foster, David J ^a   Knierim, James J ^a,b  

^a JOHNS HOPKINS UNIVERSITY SCHOOL OF MEDICINE   (United States)

^b JOHNS HOPKINS UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ANIMAL BEHAVIOR; EXPERIENCE; HIPPOCAMPUS; LEARNING; MOVEMENT (PHYSIOLOGY); NERVE CELL PLASTICITY; NEUROPHYSIOLOGY; NONHUMAN; PREDICTION; PRIORITY JOURNAL; REVIEW; RODENT; SEQUENCE LEARNING; TASK PERFORMANCE;

ANIMALS; BEHAVIOR, ANIMAL; HIPPOCAMPUS; LEARNING; RODENTIA; SPATIAL BEHAVIOR;

EID: 84860353059     PISSN: 09594388     EISSN: 18736882     Source Type: Journal    
DOI: 10.1016/j.conb.2011.12.005     Document Type: Review

References (86)

1. O'Keefe J., Nadel L. The Hippocampus as a Cognitive Map 1978, Clarendon Press, Oxford.
2. Pastalkova E., Itskov V., Amarasingham A., Buzsaki G. Internally generated cell assembly sequences in the rat hippocampus. Science 2008, 321:1322-1327.
3. Macdonald C.J., Lepage K.Q., Eden U.T., Eichenbaum H. Hippocampal " time cells" bridge the gap in memory for discontiguous events. Neuron 2011, 71:737-749.
4. Boccara C.N., Sargolini F., Thoresen V.H., Solstad T., Witter M.P., Moser E.I., Moser M.B. Grid cells in pre- and parasubiculum. Nat Neurosci 2010, 13:987-994.
5. Hafting T., Fyhn M., Molden S., Moser M.B., Moser E.I. Microstructure of a spatial map in the entorhinal cortex. Nature 2005, 436:801-806.
6. Taube J.S. The head direction signal: origins and sensory-motor integration. Annu Rev Neurosci 2007, 30:181-207.
7. Fyhn M., Hafting T., Treves A., Moser M.B., Moser E.I. Hippocampal remapping and grid realignment in entorhinal cortex. Nature 2007, 446:190-194.
8. Yoganarasimha D., Yu X., Knierim J.J. Head direction cell representations maintain internal coherence during conflicting proximal and distal cue rotations: comparison with hippocampal place cells. J Neurosci 2006, 26:622-631.
9. Markus E.J., Qin Y.L., Leonard B., Skaggs W.E., McNaughton B.L., Barnes C.A. Interactions between location and task affect the spatial and directional firing of hippocampal neurons. J Neurosci 1995, 15:7079-7094.
10. Wood E.R., Dudchenko P.A., Robitsek R.J., Eichenbaum H. Hippocampal neurons encode information about different types of memory episodes occurring in the same location. Neuron 2000, 27:623-633.
11. Griffin A.L., Eichenbaum H., Hasselmo M.E. Spatial representations of hippocampal CA1 neurons are modulated by behavioral context in a hippocampus-dependent memory task. J Neurosci 2007, 27:2416-2423.
12. Ferbinteanu J., Shapiro M.L. Prospective and retrospective memory coding in the hippocampus. Neuron 2003, 40:1227-1239.
13. Ross S. Introduction to Stochastic Dynamic Programming 1983, Academic Press, New York.
14. Wilson M.A., McNaughton B.L. Dynamics of the hippocampal ensemble code for space. Science 1993, 261:1055-1058.
15. Muller R.U., Kubie J.L. The effects of changes in the environment on the spatial firing of hippocampal complex-spike cells. J Neurosci 1987, 7:1951-1968.
16. Frank L.M., Brown E.N., Wilson M. Trajectory encoding in the hippocampus and entorhinal cortex. Neuron 2000, 27:169-178.
17. Ferbinteanu J., Shirvalkar P., Shapiro M.L. Memory modulates journey-dependent coding in the rat hippocampus. J Neurosci 2011, 31:9135-9146.
18. Kennedy P.J., Shapiro M.L. Motivational states activate distinct hippocampal representations to guide goal-directed behaviors. Proc Natl Acad Sci USA 2009, 106:10805-10810.
19. Singer A.C., Karlsson M.P., Nathe A.R., Carr M.F., Frank L.M. Experience-dependent development of coordinated hippocampal spatial activity representing the similarity of related locations. J Neurosci 2010, 30:11586-11604.
20. Ainge JA, Tamosiunaite M, Worgotter F, Dudchenko PA: Hippocampal place cells encode intended destination, and not a discriminative stimulus, in a conditional T-maze task. Hippocampus 2011 (published online DOI 10.1002/hipo.20919).
21. Foster D.J., Morris R.G., Dayan P. A model of hippocampally dependent navigation, using the temporal difference learning rule. Hippocampus 2000, 10:1-16.
22. Lansink C.S., Goltstein P.M., Lankelma J.V., McNaughton B.L., Pennartz C.M. Hippocampus leads ventral striatum in replay of place-reward information. PLoS Biol 2009, 7:e1000173.
23. van der Meer M.A., Redish A.D. Covert expectation-of-reward in rat ventral striatum at decision points. Front Integr Neurosci 2009, 3:1.
24. van der Meer M.A., Johnson A., Schmitzer-Torbert N.C., Redish A.D. Triple dissociation of information processing in dorsal striatum, ventral striatum, and hippocampus on a learned spatial decision task. Neuron 2010, 67:25-32.
25. van der Meer M.A., Redish A.D. Theta phase precession in rat ventral striatum links place and reward information. J Neurosci 2011, 31:2843-2854.
26. van der Meer M.A., Redish A.D. Ventral striatum: a critical look at models of learning and evaluation. Curr Opin Neurobiol 2011, 21:387-392.
27. Pennartz C.M., Lee E., Verheul J., Lipa P., Barnes C.A., McNaughton B.L. The ventral striatum in off-line processing: ensemble reactivation during sleep and modulation by hippocampal ripples. J Neurosci 2004, 24:6446-6456.
28. Lansink C.S., Goltstein P.M., Lankelma J.V., Joosten R.N., McNaughton B.L., Pennartz C.M. Preferential reactivation of motivationally relevant information in the ventral striatum. J Neurosci 2008, 28:6372-6382.
29. Levy W.B. A sequence predicting CA3 is a flexible associator that learns and uses context to solve hippocampal-like tasks. Hippocampus 1996, 6:579-590.
30. Blum K.I., Abbott L.F. A model of spatial map formation in the hippocampus of the rat. Neural Comput 1996, 8:85-93.
31. Yu X., Knierim J., Lee I., Shouval H. Simulating place field dynamics using spike timing-dependent plasticity. Neurocomputing 2006, 69:1253-1259.
32. Levy W.B., Steward O. Temporal contiguity requirements for long-term associative potentiation/depression in the hippocampus. Neuroscience 1983, 8:791-797.
33. Mehta M.R., Barnes C.A., McNaughton B.L. Experience-dependent, asymmetric expansion of hippocampal place fields. Proc Natl Acad Sci USA 1997, 94:8918-8921.
34. Ekstrom A.D., Meltzer J., McNaughton B.L., Barnes C.A. NMDA receptor antagonism blocks experience-dependent expansion of hippocampal " place fields" Neuron 2001, 31:631-638.
35. Mehta M.R., Quirk M.C., Wilson M.A. Experience-dependent asymmetric shape of hippocampal receptive fields. Neuron 2000, 25:707-715.
36. Lee I., Rao G., Knierim J.J. A double dissociation between hippocampal subfields: differential time course of CA3 and CA1 place cells for processing changed environments. Neuron 2004, 42:803-815.
37. Yu X., Yoganarasimha D., Knierim J.J. Backward shift of head direction tuning curves of the anterior thalamus: comparison with CA1 place fields. Neuron 2006, 52:717-729.
38. Yao H., Dan Y. Stimulus timing-dependent plasticity in cortical processing of orientation. Neuron 2001, 32:315-323.
39. O'Keefe J., Recce M.L. Phase relationship between hippocampal place units and the EEG theta rhythm. Hippocampus 1993, 3:317-330.
40. Fenton A.A., Lytton W.W., Barry J.M., Lenck-Santini P.P., Zinyuk L.E., Kubik S., Bures J., Poucet B., Muller R.U., Olypher A.V. Attention-like modulation of hippocampus place cell discharge. J Neurosci 2010, 30:4613-4625.
41. Huxter J., Burgess N., O'Keefe J. Independent rate and temporal coding in hippocampal pyramidal cells. Nature 2003, 425:828-832.
42. Dragoi G., Buzsaki G. Temporal encoding of place sequences by hippocampal cell assemblies. Neuron 2006, 50:145-157.
43. Foster D.J., Wilson M.A. Hippocampal theta sequences. Hippocampus 2007, 17:1093-1099.
44. Schmidt R., Diba K., Leibold C., Schmitz D., Buzsaki G., Kempter R. Single-trial phase precession in the hippocampus. J Neurosci 2009, 29:13232-13241.
45. Skaggs W.E., McNaughton B.L., Wilson M.A., Barnes C.A. Theta phase precession in hippocampal neuronal populations and the compression of temporal sequences. Hippocampus 1996, 6:149-172.
46. Sutton R.S., Barto A.G. Reinforcement Learning: An Introduction 1998, MIT Press, Cambridge, MA.
47. Johnson A., Redish A.D. Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J Neurosci 2007, 27:12176-12189.
48. Lee A.K., Wilson M.A. Memory of sequential experience in the hippocampus during slow wave sleep. Neuron 2002, 36:1183-1194.
49. Louie K., Wilson M.A. Temporally structured replay of awake hippocampal ensemble activity during rapid eye movement sleep. Neuron 2001, 29:145-156.
50. Dragoi G., Tonegawa S. Preplay of future place cell sequences by hippocampal cellular assemblies. Nature 2011, 469:397-401.
51. O'Neill J., Pleydell-Bouverie B., Dupret D., Csicsvari J. Play it again: reactivation of waking experience and memory. Trends Neurosci 2010, 33:220-229.
52. Foster D.J., Wilson M.A. Reverse replay of behavioural sequences in hippocampal place cells during the awake state. Nature 2006, 440:680-683.
53. Csicsvari J., O'Neill J., Allen K., Senior T. Place-selective firing contributes to the reverse-order reactivation of CA1 pyramidal cells during sharp waves in open-field exploration. Eur J Neurosci 2007, 26:704-716.
54. Diba K., Buzsaki G. Forward and reverse hippocampal place-cell sequences during ripples. Nat Neurosci 2007, 10:1241-1242.
55. Davidson T.J., Kloosterman F., Wilson M.A. Hippocampal replay of extended experience. Neuron 2009, 63:497-507.
56. Karlsson M.P., Frank L.M. Awake replay of remote experiences in the hippocampus. Nat Neurosci 2009, 12:913-918.
57. Gupta A.S., van der Meer M.A., Touretzky D.S., Redish A.D. Hippocampal replay is not a simple function of experience. Neuron 2010, 65:695-705.
58. Singer A.C., Frank L.M. Rewarded outcomes enhance reactivation of experience in the hippocampus. Neuron 2009, 64:910-921.
59. Lin L.J. Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine Learn 1992, 8:293-321.
60. Sutton R.S., Singh S.P. On step-size and bias in temporal difference learning. Proceedings of the Eighth Yale Workshop on Adaptive and Learning Systems 1994, 91-96.
61. Cichosz P. An analysis of experience replay in temporal difference learning. Cybern Syst 1999, 30:341-363.
62. Russell J.C., Towns D.R., Anderson S.H., Clout M.N. Intercepting the first rat ashore. Nature 2005, 437:1107.
63. Ribas-Fernandes J.J., Solway A., Diuk C., McGuire J.T., Barto A.G., Niv Y., Botvinick M.M. A neural signature of hierarchical reinforcement learning. Neuron 2011, 71:370-379.
64. Foster D., Dayan P. Structure in the space of value functions. Machine Learn 2002, 49:325-346.
65. Jung M.W., Wiener S.I., McNaughton B.L. Comparison of spatial firing characteristics of units in dorsal and ventral hippocampus of the rat. J Neurosci 1994, 14:7347-7356.
66. Maurer A.P., Vanrhoads S.R., Sutherland G.R., Lipa P., McNaughton B.L. Self-motion and the origin of differential spatial scaling along the septo-temporal axis of the hippocampus. Hippocampus 2005, 15:841-852.
67. Kjelstrup K.B., Solstad T., Brun V.H., Hafting T., Leutgeb S., Witter M.P., Moser E.I., Moser M.B. Finite scale of spatial representation in the hippocampus. Science 2008, 321:140-143.
68. Brun V.H., Solstad T., Kjelstrup K.B., Fyhn M., Witter M.P., Moser E.I., Moser M.B. Progressive increase in grid scale from dorsal to ventral medial entorhinal cortex. Hippocampus 2008, 18:1200-1212.
69. Dave A.S., Margoliash D. Song replay during sleep and computational rules for sensorimotor vocal learning. Science 2000, 290:812-816.
70. Rotenberg A., Abel T., Hawkins R.D., Kandel E.R., Muller R.U. Parallel instabilities of long-term potentiation, place cells, and learning caused by decreased protein kinase A activity. J Neurosci 2000, 20:8096-8102.
71. McHugh T.J., Blum K.I., Tsien J.Z., Tonegawa S., Wilson M.A. Impaired hippocampal representation of space in CA1-specific NMDAR1 knockout mice. Cell 1996, 87:1339-1349.
72. O'Keefe J. A review of the hippocampal place cells. Prog Neurobiol 1979, 13:419-439.
73. Ulanovsky N., Moss C.F. Hippocampal cellular and network activity in freely moving echolocating bats. Nat Neurosci 2007, 10:224-233.
74. Nishijo H., Ono T., Eifuku S., Tamura R. The relationship between monkey hippocampus place-related neural activity and action in space. Neurosci Lett 1997, 226:57-60.
75. Ludvig N., Tang H.M., Gohil B.C., Botero J.M. Detecting location-specific neuronal firing rate increases in the hippocampus of freely-moving monkeys. Brain Res 2004, 1014:97-109.
76. Ekstrom A.D., Kahana M.J., Caplan J.B., Fields T.A., Isham E.A., Newman E.L., Fried I. Cellular networks underlying human spatial navigation. Nature 2003, 425:184-188.
77. Siegel J.J., Nitz D., Bingman V.P. Lateralized functional components of spatial cognition in the avian hippocampal formation: evidence from single-unit recordings in freely moving homing pigeons. Hippocampus 2006, 16:125-140.
78. Siegel J.J., Nitz D., Bingman V.P. Spatial-specificity of single-units in the hippocampal formation of freely moving homing pigeons. Hippocampus 2005, 15:26-40.
79. Siegel J.J., Nitz D., Bingman V.P. Hippocampal theta rhythm in awake, freely moving homing pigeons. Hippocampus 2000, 10:627-631.
80. Kahana M.J., Sekuler R., Caplan J.B., Kirschen M., Madsen J.R. Human theta oscillations exhibit task dependence during virtual maze navigation. Nature 1999, 399:781-784.
81. Seager M.A., Johnson L.D., Chabot E.S., Asaka Y., Berry S.D. Oscillatory brain states and learning: Impact of hippocampal theta-contingent training. Proc Natl Acad Sci USA 2002, 99:1616-1620.
82. Harvey C.D., Collman F., Dombeck D.A., Tank D.W. Intracellular dynamics of hippocampal place cells during virtual navigation. Nature 2009, 461:941-946.
83. Yartsev M.M., Witter M.P., Ulanovsky N. Grid cells without theta oscillations in the entorhinal cortex of bats. Nature 2011, 479:103-107.
84. Dombeck D.A., Harvey C.D., Tian L., Looger L.L., Tank D.W. Functional imaging of hippocampal place cells at cellular resolution during virtual navigation. Nat Neurosci 2010, 13:1433-1440.
85. Deisseroth K. Optogenetics. Nat Methods 2011, 8:26-29.
86. O'Doherty J., Dayan P., Schultz J., Deichmann R., Friston K., Dolan R.J. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 2004, 304:452-454.