A model of hippocampally dependent navigation, using the temporal difference learning rule

Hippocampus

Volumn 10, Issue 1, 2000, Pages 1-16

  Foster, D J ^a,b   Morris, R G M ^a   Dayan, Peter ^b,c  

^a UNIVERSITY OF EDINBURGH   (United Kingdom)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^c UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

Hippocampus; Navigation; Place cells; Spatial learning; Temporal difference learning

Indexed keywords

ALGORITHM; ANIMAL CELL; ANIMAL EXPERIMENT; ARTICLE; BRAIN CELL; CONDITIONING; CONTROLLED STUDY; ENVIRONMENT; ESCAPE BEHAVIOR; EXPERIMENTAL MODEL; HIPPOCAMPUS; LEARNING; MAZE TEST; MEMORY; NONHUMAN; PRIORITY JOURNAL; RAT; SPATIAL MEMORY; SPATIAL ORIENTATION; STATISTICS;

ANIMALS; BEHAVIOR, ANIMAL; COMPUTER SIMULATION; CONDITIONING (PSYCHOLOGY); HIPPOCAMPUS; LOCOMOTION; MAZE LEARNING; MEMORY; MODELS, NEUROLOGICAL; RATS; REWARD; SPACE PERCEPTION;

EID: 0033968832     PISSN: 10509631     EISSN: None     Source Type: Journal    
DOI: 10.1002/(SICI)1098-1063(2000)10:1<1::AID-HIPO1>3.0.CO;2-1     Document Type: Article

References (56)

1. Abbott LF, Blum KI. 1995. Functional significance of long-term potentiation for sequence learning and prediction. Cereb Cortex 6:406-416.
2. Bannerman DM, Good MA, Butcher SP, Ramsay M, Morris RGM. 1995. Distinct components of spatial learning revealed by prior training and NMDA receptor blockade. Nature 378:182-186.
3. Barnes CA. 1979. Memory deficits associated with senescence: a neurophysiological and behavioral study in the rat. J Comp Phys Psych 93:74-104.
4. Barto AG, Sutton RS, Anderson CW. 1983. Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Trans Syst Man Cybernet 13:834-846.
5. Barto AG, Sutton RS, Watkins CJCH. 1990. Learning and sequential decision making. In: Gabriel M, Moore J, editors. Learning and computational neuroscience: foundations of adaptive networks. Cambridge, MA: MIT Press, Bradford Books, p 539-602.
6. Bertsekas DP, Tsitsiklis JN. 1996. Neuro-dynamic programming. Belmont, MA: Athena Scientific.
7. Blum KI, Abbott LF. 1996. A model of spatial map formation in the hippocampus of the rat. Neural Comput 8:85-93.
8. Brown MA, Sharp PE. 1995. Simulation of spatial learning in the Morris water maze by a neural network model of the hippocampal formation and nucleus accumbens. Hippocampus 5:171-188.
9. Bunsey M, Eichenbaum H. 1996. Conservation of hippocampal memory function in rats and humans. Nature 379:255-257.
10. Burgess N, Recce M, O'Keefe J. 1994. A model of hippocampal function. Neural Networks 7:1065-1081.
11. Collett TS, Cartwright A, Smith BA. 1986. Landmark learning and visuo-spatial memories in gerbils. J Comp Physiol [A] 158:835-851.
12. Dayan P. 1991. Navigating through temporal difference. In: Lippmann RP, et al., editors. In: Lippman RP, Moody JE, Touretzky DS, editors. Advances in Neural Information Processing System 3. p 464-470.
13. Dayan P. 1993. Improving generalisation for temporal difference learning: the successor representation. Neural Comput 5:613-624.
14. Eichenbaum H. 1996. Is the rat hippocampus just for "place"? Curr Opin Neurobiol 6:187-195.
15. Gallistel CR. 1990. The organization of learning. Cambridge, MA: MIT Press.
16. Gerstner W, Abbott LF. 1996. Learning navigational maps through potentiation and modulation of hippocampal place cells. J Comput Neurosci 4:79-94.
17. Kentros C, Hargreaves E, Hawkins RD, Kandel ER, Shapiro M, Muller RV. 1998. Abolition of long-term stability of new hippocampal place cell maps by NMDA receptor blockade. Science 280:2121-2126.
18. Levy WB. 1996. A sequence predicting CA3 is a flexible associator that learns and uses context to solve hippocampal-like tasks. Hippocampus 6:579-590.
19. McClelland JL, Goddard NH. 1996. Considerations arising from a complementary learning systems perspective on hippocampus and neocortex. Hippocampus 6:654-665.
20. Mehta MR, McNaughton BL, Barnes CA, Suster MS, Weaver KL, Gerrard JL. 1996. Rapid changes in the hippocampal population code during behavior: a case for Hebbian learning in vivo. Soc Neurosci Abstr 22:724.
21. Menzel EW 1973. Chimpanzee spatial memory organization. Science 182:943-945.
22. Montague PR, Dayan P, Sejnowski TJ. 1996. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16:1936-1947.
23. Morris RGM. 1981. Spatial localisation does not require the presence of local cues. Learn Motiv 12:239-260.
24. Morris RGM. 1983. An attempt to dissociate spatial-mapping and working-memory theories of hippocampal function. In: Seifert W, editor. The neurobiology of the hippocampus. London: Academic Press.
25. Morris RGM, Garrud P, Rawlins JNP, O'Keefe J. 1982. Place navigation impaired in rats with hippocampal lesions. Nature 297:681-683.
26. Muller RU, Kubie JL, Ranck JB. 1987. Spatial firing patterns of hippocampal complex-spike cells in a fixed environment. J Neurosci 7:1935-1950.
27. O'Keefe J, Burgess N. 1996. Geometrical determinants of the place fields of hippocampal neurons. Nature 381:425-428.
28. O'Keefe J, Dostrovsky J. 1971. The hippocampus as a spatial map: preliminary evidence from unit activity in the freely moving rat. Brain Res 34:171-175.
29. O'Keefe J, Nadel L. 1978. The hippocampus as a cognitive map. London: Clarendon.
30. O'Keefe J, Recce ML. 1993. Phase relationship between hippocampal place cells and the EEG theta rhythm. Hippocampus 3:317-330.
31. O'Mara SM, Rolls ET, Berthoz A, Kesner RP. 1994. Neurons responding to whole-body motion in the primate hippocampus. J Neurosci 14:6511-6523.
32. Panakhova E, Buresova O, Bures J. 1984. Persistence of spatial memory in the Morris water maze tank task. Int J Psychophysiol 2:5-10.
33. Poggio T, Girosi F. 1990. Networks for approximation and learning. Proc IEEE 78:1481-1497.
34. Redish AD, Touretzky DS. 1997. Navigating with landmarks: computing goal locations from place codes. In Ikeuchi K, Veloso M, editors. Symbolic Visual Learning. Oxford University Press. Chapter 12, pp. 325-351.
35. Rudy JW, Sutherland RJ. 1995. Configural association theory and the hippocampal formation: an appraisal and reconfiguration. Hippocampus 5:375-389.
36. Saucier D, Cain DP. 1995. Spatial learning without NMDA receptor dependent long-term potentiation. Nature 378:186-189.
37. Schultz W, Dayan P, Montague PR. 1997. A neural substrate of prediction and reward. Science 275:1593-1599.
38. Singh SP. 1992. Transfer of learning by composing solutions of elemental sequential tasks. Machine Learning, 8:323-339.
39. Skaggs WE, McNaughton BL, Wilson MA, Barnes CA. 1996. Theta phase precession in hippocampal neuronal populations and the compression of temporal sequences. Hippocampus 6:149-172.
40. Speakman A, O'Keefe J. 1991. Hippocampal complex spike cells do not change their place fields if the goal is moved within a cue controlled environment. Eur J Neurosci 2:544-555.
41. Steele RJ, Morris RGMM. 1999. Delay-dependent impairment of a matching-to-place task with chronic and intrahippocampal infusion of the NMDA-antagonist D-AP5. Hippocampus 9:118-136.
42. Sutherland RJ, Whishaw IQ, Kolb B. 1983. A behavioural analysis of spatial localisation following electrolytic, kainate or colchicine induced damage to the hippocampal formation in the rat. Behav Brain Res 7:133-153.
43. Sutton RS. 1988. Learning to predict by the methods of temporal difference learning. Machine Learn 3:9-44.
44. Sutton RS, Barto AG. 1988. Reinforcement learning: an introduction. MIT Press. A temporal-difference model of classical conditioning. Tech Report, GTE Labs, TR87-509.2, 1987.
45. Taube JS. 1995. Head direction cells recorded in the anterior thalamic nuclei of freely moving rats. J Neurosci 15:70-86.
46. Trullier O, Wiener SI, Berthoz A, Meyer J. 1997. Biologically-based artificial navigation systems - review and prospects. Prog Neurobiol 51:483-544.
47. Wan HS, Touretzky DS, Redish AD. 1994. Towards a computational theory of rat navigation. In: Mozer M, Smolensky P, Toutetzky D, Elman J, Weigend A, editors. Proceedings of the 1993 Connectionist Models Summer School. Hillsdale, NJ: Lawrence Erlbaum, p 11-19.
48. Watkins CJCH. 1989. Learning from delayed rewards. Ph.D. thesis, University of Cambridge.
49. Whishaw IQ. 1985. Formation of a place learning set in the rat: a new paradigm for neurobehavioral studies. Physiol Behav 35:139-145.
50. Whishaw IQ. 1991. Latent learning in a swimming pool place task by rats: evidence for the use of associative and not cognitive mapping processes. QJ Exp Psychol [B] 43:83-103.
51. Wiener SI. 1996. Spatial, behavioral and sensory correlates of hippocampal CA1 complex spike cell activity: implications for information processing functions. Prog Neurobiol 49:335-361.
52. Wilkie DM, Palfrey R. 1987. A computer simulation model of rats' place navigation in the Morris water maze. Behav Res Methods Instr Comput 19:400-403.
53. Wilson MA, McNaughton BL. 1993. Dynamics of the hippocampal ensemble code for space. Science 261:1055-1058.
54. Witten IH. 1977. An adaptive optimal controller for discrete-time Markov environments. Information Control 34:286-295.
55. Wood ER, Dudchenko PA, Eichenbaum H. 1999. The global record of memory in hippocampal neuronal activity. Nature 397:613-616.
56. Zipser D. 1986. Biologically plausible models of place recognition and goal location. In: Rumelhart DE, McClelland JL, editors. Parallel distributed processing, volume 1. Cambridge, MA: MIT Press.