Design principles of the hippocampal cognitive map

Advances in Neural Information Processing Systems

Volumn 3, Issue January, 2014, Pages 2528-2536

  Stachenfeld, Kimberly L ^a   Botvinick, Matthew M ^a   Gershman, Samuel J ^b  

^a PRINCETON UNIVERSITY   (United States)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COGNITIVE SYSTEMS; INFORMATION SCIENCE; LOCATION; STRUCTURAL DESIGN;

COGNITIVE MAPS; CURRENT TRANSITIONS; DESIGN PRINCIPLES; EIGEN DECOMPOSITION; GEOMETRIC STRUCTURE; HIERARCHICAL DECOMPOSITIONS; HIERARCHICAL REINFORCEMENT LEARNING; NATURAL BOUNDARY;

REINFORCEMENT LEARNING;

EID: 84937876815     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (48)

1. A. Alvernhe, E. Save, and B. Poucet. Local remapping of place cell firing in the tolman detour task. European Journal of Neuroscience, 33:1696-1705, 2011.
2. H. T. Blair, A. C. Welday, and K. Zhang. Scale-invariant memory representations emerge from moire interference between grid fields that produce theta oscillations: a computational model. The Journal of Neuroscience, 27:3211-3229, 2007.
3. M. M. Botvinick, Y. Niv, and A. C. Barto. Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective. Cognition, 113:262-280, 2009.
4. M. P. Brandon, J. Koenig, J. K. Leutgeb, and S. Leutgeb. New and distinct hippocampal place codes are generated in a new environment during septal inactivation. Neuron, 82:789-796, 2014.
5. N. D. Daw, Y. Niv, and P. Dayan. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neuroscience, 8:1704-1711, 2005.
6. P. Dayan. Improving generalization for temporal difference learning: The successor representation. Neural Computation, 5:613-624, 1993.
7. D. Derdikman, J. R. Whitlock, A. Tsao, M. Fyhn, T. Hafting, M.-B. Moser, and E. I. Moser. Fragmentation of grid cell maps in a multicompartment environment. Nature Neuroscience, 12:1325-1332, 2009.
8. C. Diuk, K. Tsai, J. Wallis, M. Botvinick, and Y. Niv. Hierarchical learning induces two simultaneous, but separable, prediction errors in human basal ganglia. The Journal of Neuroscience, 33:5797-5805, 2013.
9. M. S. Fanselow. From contextual fear to a dynamic view of memory systems. Trends in Cognitive Sciences, 14:7-15, 2010.
10. A. Fenton, L. Zinyuk, and J. Bures. Place cell discharge along search and goal-directed trajectories. European Journal of Neuroscience, 12:3450 2001.
11. D. Foster, R. Morris, and P. Dayan. A model of hippocampally dependent navigation, using the temporal difference learning rule. Hippocampus, 10:1-16, 2000.
12. M. Franzius, H. Sprekeler, and L. Wiskott. Slowness and sparseness lead to place, head-direction, and spatial-view cells. PLoS Computational Biology, 3:3287-3302, 2007.
13. C. R. Gallistel. The Organization of Learning. The MIT Press, 1990.
14. S. J. Gershman, C. D. Moore, M. T. Todd, K. A. Norman, and P. B. Sederberg. The successor representation and temporal context. Neural Computation, 24:1553-1568, 2012.
15. N. J. Gustafson and N. D. Daw. Grid cells, place cells, and geodesic generalization for spatial reinforcement learning. PLoS Computational Biology, 7:e1002235, 2011.
16. T. Hafting, M. Fyhn, S. Molden, M.-B. Moser, and E. I. Moser. Microstructure of a spatial map in the entorhinal cortex. Nature, 436:801-806, 2005.
17. S. C. Hirtle and J. Jonides. Evidence of hierarchies in cognitive maps. Memory & Cognition, 13:208-217, 1985.
18. S. A. Hollup, S. Molden, J. G. Donnett, M. B. Moser, and E. I. Moser. Accumulation of hippocampal place fields at the goal location in an annular watermaze task. Journal of Neuroscience, 21:1635-1644, 2001.
19. M. W. Howard, M. S. Fotedar, A. V. Datey, and M. E. Hasselmo. The temporal context model in spatial navigation and relational learning: toward a common explanation of medial temporal lobe function across domains. Psychological Review, 112:75-116, 2005.
20. M. W. Howard and M. J. Kahana. A distributed representation of temporal context. Journal of Mathematical Psychology, 46:269-299, 2002.
21. T. Kobayashi, A. Tran, H. Nishijo, T. Ono, and G. Matsumoto. Contribution of hippocampal place cell activity to learning and formation of goal-directed navigation in rats. Neuroscience, 117:1025-35, 2003.
22. G. Konidaris, S. Osentoski, and P. S. Thomas. Value function approximation in reinforcement learning using the Fourier basis. In AAAI, 2011.
23. J. Krupic, N. Burgess, and J. O'oeefe. Neural representations of location composed of spatially periodic bands. Science, 337:853-857, 2012.
24. P. Lenck-Santini, R. Muller, E. Save, and B. Poucet. Relationships between place cell firing fields and navigational decisions by rats. The Journal of Neuroscience, 22:9035-47, 2002.
25. P. Lenck-Santini, E. Save, and B. Poucet. Place-cell firing does not depend on the direction of turn in a y-maze alternation task. European Journal of Neuroscience, 13(5):1055-8, 2001.
26. S. Mahadevan. Learning representation and control in markov decision processes: New frontiers. Foundations and Trends in Machine Learning, 1:403-565, 2009.
27. B. L. McNaughton, F. P. Battaglia, O. Jensen, E. I. Moser, and M.-B. Moser. Path integration and the neural basis of the 'cognitive map'. Nature Reviews Neuroscience, 7:663-678, 2006.
28. M. R. Mehta, C. A. Barnes, and B. L. McNaughton. Experience-dependent, asymmetric expansion of hippocampal place fields. Proceedings of the National Academy of Sciences, 94:8918-8921, 1997.
29. M. R. Mehta, M. C. Quirk, and M. A. Wilson. Experience-dependent asymmetric shape of hippocampal receptive fields. Neuron, 25:707-715, 2000.
30. I. Menache, S. Mannor, and N. Shimkin. Q-cut-dynamic discovery of sub-goals in reinforcement learning. In European Conference on Machine Learning, pages 295-306. Springer, 2002.
31. L. K. Morgan, S. P. MacEvoy, G. K. Aguirre, and R. A. Epstein. Distances between real-world locations are represented in the human hippocampus. The Journal of Neuroscience, 31:1238-1245, 2011.
32. R. U. Muller and J. L. Kubie. The effects of changes in the environment on the spatial firing of hippocampal complex-spike cells. The Journal of Neuroscience, 7:1951-1968, 1987.
33. R. U. Muller, M. Stead, and J. Pach. The hippocampus as a cognitive graph. The Journal of General Physiology, 107:663-694, 1996.
34. J. O'Keefe and L. Nadel. The Hippocampus as a Cognitive Map. Clarendon Press Oxford, 1978.
35. A. K. Reid and J. R. Staddon. A dynamic route finder for the cognitive map. Psychological Review, 105:585-601, 1998.
36. J. J. Ribas-Fernandes, A. Solway, C. Diuk, J. T. McGuire, A. G. Barto, Y. Niv, and M. M. Botvinick. A neural signature of hierarchical reinforcement learning. Neuron, 71:370-379, 2011.
37. A. C. Schapiro, T. T. Rogers, N. I. Cordova, N. B. Turk-Browne, and M. M. Botvinick. Neural representations of events arise from temporal community structure. Nature Neuroscience, 16:486-492 2013.
38. J. Shi and J. Malik. Normalized cuts and image segmentation. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 22:888-905, 2000.
39. Ö. Şimşek, A. P. Wolfe, and A. G. Barto. Identifying useful subgoals in reinforcement learning by local graph partitioning. In Proceedings of the 22nd International Conference on Machine Learning, pages 816-823. ACM, 2005.
40. W. E. Skaggs and B. L. McNaughton. Spatial firing properties of hippocampal ca1 populations in an environment containing two visually identical regions. The Journal of Neuroscience, 18:8455-8466, 1998.
41. A. Speakman and J. O'Keefe. Hippocampal complex spike cells do not change their place fields if the goal is moved within a cue controlled environment. European Journal of Neuroscience, 2:544-5, 1990.
42. H. Sprekeler. On the relation of slow feature analysis and laplacian eigenmaps. Neural computation, 23:3287-3302, 2011.
43. A. Stevens and P. Coupe. Distortions in judged spatial relations. Cognitive Psychology, 10:422-437, 1978.
44. R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. MIT press, 1998.
45. R. S. Sutton, D. Precup, and S. Singh. Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112:181-211, 1999.
46. E. C. Tolman. Cognitive maps in rats and men. Psychological Review, 55:189-208, 1948.
47. T. J. Wills, F. Cacucci, N. Burgess, and J. O'Keefe. Development of the hippocampal cognitive map in preweanling rats. Science, 328:1573-1576, 2010.
48. B. J. Wiltgen, M. J. Sanders, S. G. Anagnostaras, J. R. Sage, and M. S. Fanselow. Context fear learning in the absence of the hippocampus. The Journal of Neuroscience, 26:5484-5491, 2006.