Forward and Backward Inference in Spatial Cognition

PLoS Computational Biology

Volumn 9, Issue 12, 2013, Pages

  Penny, Will D ^a   Zeidman, Peter ^a   Burgess, Neil ^b  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

MAMMALS;

BACKWARD INFERENCES; FORWARD-AND-BACKWARD; KNOWN ENVIRONMENTS; LOCATION ESTIMATES; PATH INTEGRATION; PROBABILISTIC MODELS; SENSORY INPUT; SPATIAL COGNITION; STATE TRAJECTORY; STATISTICAL INFERENCE;

DECISION MAKING;

ALGORITHM; ARTICLE; BRAIN MAPPING; COMPUTER SIMULATION; DECISION MAKING; DEPTH PERCEPTION; ENTORHINAL CORTEX; HIPPOCAMPUS; IMAGERY; INFERENTIAL STATISTICS; MATHEMATICAL MODEL; MOTIVATION; MOTOR PERFORMANCE; SENSORY STIMULATION; SPATIAL DISCRIMINATION; THETA RHYTHM; VALIDATION PROCESS; VISUAL STIMULATION;

ALGORITHMS; ANIMALS; COGNITION; HIPPOCAMPUS; HUMANS; MODELS, THEORETICAL; PROBABILITY; SPACE PERCEPTION;

EID: 84892767742     PISSN: 1553734X     EISSN: 15537358     Source Type: Journal    
DOI: 10.1371/journal.pcbi.1003383     Document Type: Article

References (120)

1. O'Keefe J, Nadel L (1978) The Hippocampus as a Cognitive Map. Oxford University Press.
2. Gallistel CR (1990) The organization of learning. Cambridge: MIT Press.
3. Redish AD (1999) Beyond the Cognitive Map. Cambridge: MIT Press.
4. Gupta A, van der Meer M, Touretzky D, Redish A, (2012) Segmentation of spatial experience by hippocampal sequences. Nature Neuroscience 15: 1032-1039.
5. Buhry L, Azizi A, Cheng S, (2011) Reactivation, replay, and preplay: how it might all fit together. Neural Plast 2011: 203462.
6. Mumford D, (1992) On the computational architecture of the neocortex II: the role of corticocortical loops. Biological Cybernetics 66: 241-251.
7. Tolman E, (1948) Cognitive maps in rats and men. Psychological Review 55: 189-208.
8. Keith J, McVety K, (1988) Latent place learning in a novel environment and the influence of prior training in rats. Psycho 16: 146-151.
9. Gaussier P, Revel A, Banquet J, Babeau V, (2002) From view cells and place cells to cognitive map learning: processing stages of the hippocampal system. Biological Cybernetics 86: 15-28.
10. Hasselmo M, (2005) A model of prefrontal cortical mechanisms for goal-directed behavior. Journal of Cognitive Neuroscience 17: 1115-1129.
11. van-der Meer M, Kurth-Nelson Z, Redish AD, (2012) Information processing in decision making systems. The Neuroscientist 18: 342-359.
12. Sutton R, Barto A (1998) Reinforcement Learning: An Introduction. Cambridge: MIT Press.
13. Arleo A, Rondi-Reig L, (2007) Multimodal sensory integration and concurrent navigation strategies for spatial cognition in real and artificial organisms. Journal of Integrative Neuroscience 6: 327-366.
14. White N, McDonald R, (2002) Multiple parallel memory systems in the brain of the rat. Neurobiology of Learning and Memory 77: 125-184.
15. Daw N, Niv Y, Dayan P, (2005) Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci 8: 1704-1711.
16. Lengyel M, Dayan P (2007) Hippocampal contributions to control: The third way. In: J Platt YS D Koller, Roweis S, editors, Advances in Neural Information Processing Systems.
17. Tse D, Langston R, Kakeyama M, Bethus I, Spooner P, et al. (2007) Schemas and memory consolidation. Science 316: 76-82.
18. Khamassi M, Humphries M, (2012) Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies. Frontiers Behavioural Neuroscience 6: 79.
19. Pezzulo G, Rigoli F, Chersi F, (2013) The Mixed Instrumental Controller: using Values of Information to combine habitual choice and mental simulation. Frontiers in Psychology 4: 92.
20. Bousquet O, Balakrishnan K, Honavar V (1998) Is the hippocampus a Kalman filter? In: In Proceedings of the Pacific Symposium on Biocomputing. pp. 655-666.
21. Fuhs M, Touretzky D, (2007) Context Learning in the Rodent Hippocampus. Neural Computation 19: 3173-3215.
22. Johnson A, Varberg Z, Benhardus J, Maahs A, Schrater P, (2012) The hippocampus and exploration: dynamically evolving behavior and neural representations. Frontiers in Human Neuroscience 6: 216.
23. Lorincz A, Buzsaki G, (2000) Two-Phase Computational Model Training Long-Term Memories in the Entorhinal-Hippocampal Region. Annals New York Academy of Sciences 911: 83-111.
24. Buzsaki G, (1989) Two-stage model of memory trace formation: a role for noisy brain states. Neuroscience 31: 551-70.
25. Andersen P, Morris R, Amaral D, Bliss T, O'Keefe J (2007) The Hippocampus Book. Oxford: Oxford University Press.
26. Jacobs LF, (2012) From chemotaxis to the cognitive map: the function of olfaction. Proc Natl Acad Sci U S A 109 Suppl 1: 10693-10700.
27. McNaughton B, (2010) Cortical hierarchies, sleep and the extraction of knowledge from memory. Artificial Intelligence 174: 205-214.
28. Schaal S (1997) Learning from demonstration. In: Mozer M, Jordan M, Petsche T, editors, Advances in Neural Information Processing Systems 9. Cambridge, MA: MIT Press.
29. Foster DJ, Morris RG, Dayan P, (2000) A model of hippocampally dependent navigation, using the temporal difference learning rule. Hippocampus 10: 1-16.
30. Pouget A, Sejnowski T, (1997) Spatial transformations in the parietal cortex using basis functions. Journal of Cognitive Neuroscience 9: 222-237.
31. Becker S, Burgess N (2001) Modelling spatial recall, mental imagery and neglect. In: Leen T, Dietterich T, Tresp V, editors, Advances in Neural Information Processing Systems 13. Cambridge: MIT Press.
32. Byrne P, Becker S, Burgess N, (2007) Remembering the past and imagining the future: a neural model of spatial memory and imagery. Psychol Rev 114: 340-375.
33. Bostock E, Muller R, Kubie J, (1991) Experience-dependent modifications of hippocampal place cell firing. Hippocampus 1: 193-206.
34. Leutgeb S, Leutgeb JK, Barnes CA, Moser EI, McNaughton BL, et al. (2005) Independent codes for spatial and episodic memory in hippocampal neuronal ensembles. Science 309: 619-623.
35. Kali S, Dayan P, (2000) The involvement of recurrent connections in area ca3 in establishing the properties of place fields: a model. J Neurosci 20: 7463-7477.
36. Oore S, Hinton G, Dudek G, (1997) A mobile robot that learns its place. Neural Computation 9: 683-699.
37. Durrant-Whyte H, Bailey T, (2006) Simultaneous localisation and mapping (SLAM): Part i the essential algorithms. Robotics and Automation Magazine 13: 99-110.
38. Bishop C (2006) Pattern Recognition and Machine Learning. New York: Springer.
39. Doucet A, Johansen A (2011) Handbook of nonlinear filtering. Oxford University Press, chapter A tutorial on Particle Filtering and Smoothing: Fifteen years Later.
40. Einicke G (2012) Smoothing, Filtering and Prediction- Estimating the past, present and future. New York: InTech.
41. Briers M, Doucet A, Maskell S (2004) Smoothing algorithms for state-space models. Technical report, Engineering Department, Cambridge University.
42. Green D, Swets J (1966) Signal detection theory and psychophysics. Chichester: John Wiley.
43. Knierim J, (2006) Neural representations of location outside the hippocampus. Learn Mem 13: 405-415.
44. van Strien N, Cappaert N, Witter M, (2009) The anatomy of memory: an interactive overview of the parahippocampal-hippocampal network. Nature Reviews Neuroscience 10: 272-282.
45. Mhatre H, Gorchetchnikov A, Grossberg S, (2012) Grid cell hexagonal patterns formed by fast self-organized learning within entorhinal cortex. Hippocampus 22: 320-334.
46. Witter M, Moser E, (2006) Spatial representation and the architecture of the entorhinal cortex. Trends Neuroscience 29: 671-678.
47. Jones M, McHugh T, (2011) Updating hippocampal representations: CA2 joins the circuit. Trends Neurosci 34: 526-535.
48. Manns J, Eichenbaum H, (2006) Evolution of declarative memory. Hippocampus 16: 795-808.
49. Tsao A, Moser M, Moser E, (2013) Traces of experience in the lateral entorhinal cortex. Current Biology 23: 1-7.
50. Conklin J, Eliasmith C, (2005) An attractor network model of path integration in the rat. Journal of Computational Neuroscience 18: 183-203.
51. McNaughton B, Battaglia F, Jensen O, Moser E, Moser M, (2006) Path integration and the neural basis of the 'cognitive map'. Nat Rev Neurosci 7: 663-678.
52. Hafting T, Fyhn M, Molden S, Moser M, Moser E, (2005) Microstructure of a spatial map in the entorhinal cortex. Nature 436: 801-806.
53. Doeller C, Barry C, Burgess N, (2010) Evidence for grid cells in a human memory network. Nature 463: 657-661.
54. O' Keefe J, Dostrovsky J, (1971) The hippocampus as a spatial map. preliminary evidence from unit activity in the freely moving rat. Brain Research 34: 171-175.
55. Ekstrom A, Kahana M, Caplan J, Fields T, Isham E, et al. (2003) Cellular networks underlying human spatial navigation. Nature 425: 184-188.
56. Maguire E, Burgess N, Donnett J, Frackowiak R, Frith C, et al. (1998) Knowing where and getting there: a human navigation network. Science 280: 921-924.
57. Hassabis D, Chu C, Rees G, Weiskopf N, Molyneux P, et al. (2009) Decoding neuronal ensembles in the human hippocampus. Curr Biol 19: 546-554.
58. Morgan L, Macevoy S, Aguirre G, Epstein R, (2011) Distances between real-world locations are represented in the human hippocampus. J Neurosci 31: 1238-1245.
59. Taube J, (2009) Head direction cells. Scholarpedia 4: 1787.
60. Sargolini F, Fyhn M, Hafting T, McNaughton B, Witter M, et al. (2006) Conjunctive representation of position, direction, and velocity in entorhinal cortex. Science 312: 758-762.
61. O'Keefe J, Burgess N, (2005) Dual phase and rate coding in hippocampal place cells: theoretical significance and relationship to entorhinal grid cells. Hippocampus 15: 853-866.
62. Sreenivasan S, Fiete I, (2011) Grid cells generate an analog error-correcting code for singularly precise neural computation. Nature Neuroscience 14: 1330-1337.
63. Arleo A, Gerstner W, (2000) Spatial cognition and neuro-mimetic navigation: a model of hippocampal place cell activity. Biological Cybernetics 83: 287-299.
64. Hassabis D, Kumaran D, Vann S, Maguire E, (2007) Patients with hippocampal amnesia cannot imagine new experiences. Proc Natl Acad Sci U S A 104: 1726-1731.
65. Hassabis D, Maguire E, (2009) The construction system of the brain. Philosophical Transactions of the Royal Society London B 364: 1263-71.
66. Fox C, Prescott T (2009) Hippocampus as unitary coherent particle filter. In: International Joint Conference on Neural Networks. Atlanta, GA, USA.
67. Johnson A, Redish AD, (2007) Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J Neurosci 27: 12176-12189.
68. Bogacz R, Brown E, Moehlis J, Holmes P, Cohen J, (2006) The physics of optimal decision making: a formal analysis of models and performance in two-alternative forced-choice tasks. Psychological Review 113: 700-765.
69. Machens C, Romo R, Brody C, (2005) Flexible control of mutual inhibition: a neural model of two-interval discrimination. Science 307: 1121-1124.
70. Lisman J, (2007) Role of the dual entorhinal inputs to hippocampus: a hypothesis based on cue/action (non-self/self) couplets. Progress in Brain Research 163: 615-625.
71. Penny W, (2012) Bayesian models of brain and behaviour. ISRN Biomathematics 2012: 785791.
72. Ma W, Beck J, Latham P, Pouget A, (2006) Bayesian inference with probabilistic population codes. Nat Neurosci 9: 1432-1438.
73. Beck J, Latham P, Pouget A, (2011) Marginalization in neural circuits with divisive normalization. J Neurosci 31: 15310-15319.
74. Fenton A, Muller R, (1998) Place cell discharge is extremely variable during individual passes of the rat through the firing field. Proc Natl Acad Sci U S A 95: 3182-3187.
75. Lengyel M, Dayan P (2007) Uncertainty, phase and oscillatory hippocampal recall. In: Scholkopf B, Platt J, Hofmann T, editors, Advances in Neural Information Processing Systems 19, Cambridge: MIT Press.
76. Foster DJ, Wilson MA, (2006) Reverse replay of behavioural sequences in hippocampal place cells during the awake state. Nature 440: 680-683.
77. Attias H (2003) Planning by probabilistic inference. In: Bishop C, Frey B, editors, Proceedings of the 9th International Conference on Artificial Intelligence and Statistics.
78. Toussaint M (2009) Robot trajectory optimisation using approximate inference. In: Danyluk A, Bottou L, Littman M, editors, Proc. of the 26 th International Conference on Machine Learning (ICML 2009).
79. Botvinick M, Toussaint M, (2012) Planning as inference. Trends Cogn Sci 16: 485-488.
80. Solway A, Botvinick M, (2012) Goal-directed decision making as probabilistic inference: a computational framework and potential neural correlates. Psychol Rev 119: 120-154.
81. Bishop CM (1995) Neural Networks for Pattern Recognition. Oxford University Press, Oxford.
82. Ghahramani Z, Hinton G (1996) Parameter Estimation for Linear Dynamical Systems. Technical Report CRG-TR-96-2, Department of Computer Science, University of Toronto. Also available from http://www.mlg.eng.cam.ac.uk/zoubin/papers.html.
83. Beal M (2003) Variational algorithms for approximate Bayesian inference. Ph.D. thesis, University College London.
84. Daunizeau J, Friston KJ, Kiebel SJ, (2009) Variational Bayesian identification and prediction of stochastic nonlinear dynamic causal models. Physica D 238: 2089-2118.
85. Friston K, (2010) The free-energy principle: a unified brain theory? Nat Rev Neurosci 11: 127-138.
86. Li W, Todorov E, (2007) Iterative linearization methods for approximately optimal control and estimation of non-linear stochastic systems. International Journal of Control 80: 1439-1453.
87. Todorov E (2008) General duality between optimal control and estimation. In: IEEE Conference on Decision and Control. volume 47, pp. 4286-4292.
88. Muller R, Stead M, (1997) Hippocampal place cells connected by Hebbian synapses can solve spatial problems. Hippocampus 6: 709-719.
89. Johnson A, Redish A, (2005) Hippocampal replay contributes to within session learning in a temporal difference reinforcement learning model. Neural Netw 18: 1163-1171.
90. Pennartz C, Lee E, Verheul J, Lipa P, Barnes C, et al. (2004) The ventral striatum in off-line processing: ensemble reactivation during sleep and modulation by hippocampal ripples. Journal of Neuroscience 24: 6446-6456.
91. Foster DJ, Wilson MA, (2007) Hippocampal theta sequences. Hippocampus 17: 1093-1099.
92. O'Keefe J, Recce M, (1993) Phase relationship between hippocampal place units and the EEG theta rhythm. Hippocampus 3: 317-330.
93. Skaggs W, Mcnaughton B, Wilson M, Barnes C, (1996) Theta phase precession in hippocampal neuronal populations and the compression of temporal sequences. Hippocampus 6: 149-172.
94. Dragoi G, Buzsaki G, (2006) Temporal encoding of place sequences by Hippocampal Cell Assemblies. Neuron 50: 145-157.
95. Marr D, (1971) Simple memory: A theory of archicortex. Phil Trans Royal Soc B 262: 23-81.
96. Alvarez P, Squire L, (1994) Memory consolidation and the medial temporal lobe: a simple network model. Proceedings National Academy of Sciences 91: 7041-7045.
97. Mclelland J, McNaughton B, O'Reilly R, (1995) Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychology Review 102: 419-457.
98. Wilson M, McNaughton B, (1994) Reactivation of hippocampal ensemble memories during sleep. Science 265: 676-679.
99. Lee AK, Wilson MA, (2002) Memory of sequential experience in the hippocampus during slow wave sleep. Neuron 36: 1183-1194.
100. Girardeau G, Benchenane K, Wiener S, Buzsaki G, Zugaro M, (2009) Selective suppression of hippocampal ripples impairs spatial memory. Nature Neuroscience 12: 1222-1223.
101. Diba K, Buzsaki G, (2007) Forward and reverse hippocampal place-cell sequences during ripples. Nat Neurosci 10: 1241-1242.
102. Davidson T, Kloosterman F, Wilson M, (2009) Hippocampal Replay of Extended Experience. Neuron 63: 497-507.
103. Csiscvari J, O'Neill J, Allen K, Senior T, (2007) Place-selective firing contributes to the reverseorder activation of CA1 pyramidal cells during sharp waves in open-field exploration. European Journal of Neuroscience 26: 704-716.
104. Gupta A, van der Meer M, Touretzky D, Redish A, (2010) Hippocampal replay is not a simple function of experience. Neuron 65: 695-705.
105. Karlsson MP, Frank LM, (2009) Awake replay of remote experiences in the hippocampus. Nat Neurosci 12: 913-918.
106. Jadhav S, Kemere C, German PW, Frank L, (2012) Awake Hippocampal Sharp-Wave Ripples Support Spatial Memory. Science 336: 1454-1458.
107. Dragoi G, Tonegawa S, (2011) Preplay of future place cell sequences by hippocampal cellular assemblies. Nature 469: 397-401.
108. Nardini M, Jones P, Bedford R, Braddick O, (2008) Development of Cue Integration in Human Navigation. Current Biology 18: 689-693.
109. Gray J, McNaughton N (1987) The Neuropsychology of Anxiety: An enquiry into the functions of the septo-hippocampal system. Oxford: Oxford University Press.
110. Jezek K, Henriksen E, Treves A, Moser E, Moser M, (2011) Theta-paced ickering between placecell maps in the hippocampus. Nature 478: 246-251.
111. Courville AC, Daw ND (2007) The rat as particle filter. In: Platt J, Koller D, Singer Y, Roweis S, editors, Advances in Neural Information Processing Systems 20, Cambridge, MA: MIT Press. pp. 369-376.
112. Alvernhe A, Save E, Poucet B, (2011) Local remapping of place cell firing in the Tolman detour task. European Journal of Neuroscience 33: 1696-1705.
113. Alvernhe A, van Cauter T, Save E, Poucet B, (2008) Different CA1 and CA3 representations of novel routes in a shortcut situation. Journal of Neuroscience 28: 7324-7333.
114. Pfeiffer B, Foster D, (2013) Hippocampal place-cell sequences depict future paths to remembered goals. Nature 497: 74-79.
115. Friston KJ, Harrison L, Penny W, (2003) Dynamic causal modelling. Neuroimage 19: 1273-1302.
116. Litvak V, Mattout J, Kiebel S, Phillips C, Henson R, et al. (2011) EEG and MEG data analysis in SPM8. Comput Intell Neurosci 2011: 852961.
117. Benchenane K, Peyrache A, Khamassi M, Tierney P, Gioanni Y, et al. (2010) Coherent theta oscillations and reorganization of spike timing in the hippocampal-prefrontal network upon learning. Neuron 66: 921-36.
118. Beal M (2003) Variational Algorithms for Approximate Bayesian Inference. Ph.D. thesis, Gatsby Computational Neuroscience Unit, University College London.
119. Todorov E, (2009) Efficient computation of optimal actions. Proceedings National Academy of Sciences 106: 11478-11483.
120. Abbott L (2006) Where are the switches on this thing?, Oxford University Press, chapter 23 Problems in Systems Neuroscience.