Erratum: Path-finding in real and simulated rats: Assessing the influence of path characteristics on navigation learning (Journal of Computational Neuroscience DOI: 10.1007/s10827-008-0094-6);Path-finding in real and simulated rats: Assessing the influence of path characteristics on navigation learning

Journal of Computational Neuroscience

Volumn 25, Issue 3, 2008, Pages 562-582

  Tamosiunaite, Minija ^a,c   Ainge, James ^b   Kulvicius, Tomas ^c   Porr, Bernd ^d   Dudchenko, Paul ^b   Wörgötter, Florentin ^b,c  

^a VYTAUTAS MAGNUS UNIVERSITY   (Lithuania)

^b UNIVERSITY OF STIRLING   (United Kingdom)

^c UNIVERSITY OF GÖTTINGEN   (Germany)

^d UNIVERSITY OF GLASGOW   (United Kingdom)

Author keywords

Function approximation; Place field system; Reinforcement learning; SARSA; Weight decay

Indexed keywords

ECONOMIC AND SOCIAL EFFECTS; NAVIGATION; RATS;

FUNCTIONS APPROXIMATIONS; PATH CHARACTERISTIC; PATH FINDING; PATH SEGMENTS; PLACE FIELD SYSTEM; REINFORCEMENT LEARNING ALGORITHMS; REINFORCEMENT LEARNINGS; SARSA; STATE-SPACE; WEIGHT DECAY;

REINFORCEMENT LEARNING;

ANIMAL EXPERIMENT; ARTICLE; COMPUTER SIMULATION; CONTROLLED STUDY; DOPAMINERGIC SYSTEM; EXPLORATORY BEHAVIOR; LEARNING ALGORITHM; MALE; MATHEMATICAL ANALYSIS; NONHUMAN; PRIORITY JOURNAL; PROBABILITY; REINFORCEMENT; REPRODUCIBILITY; TASK PERFORMANCE;

EID: 53149129441     PISSN: 09295313     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10827-010-0217-8     Document Type: Erratum

References (66)

1. Ainge, J. A., Tamosiunaite, M., Wörgötter, F., & Dudchenko, P. A. (2007). Hippocampal CA1 place cells encode intended destination on a maze with multiple choice points. The Journal of Neuroscience, 27(36), 9769-9779.
2. Arleo, A., & Gerstner, W. (2000). Spatial cognition and neuro-mimetic navigation: A model of hippocampal place cell activity. Biological Cybernetics, 83(3), 287-299.
3. Arleo, A., Smeraldi, F., & Gerstner, W. (2004). Cognitive navigation based on nonuniform Gabor space sampling, unsupervised growing networks, and reinforcement learning. IEEE Transactions on Neural Networks, 15(3), 639-652.
4. Asfour, T., Regenstein, K., Azad, P., Schröder, J., Bierbaum, A., Vahrenkamp, N., et al. (2006). ARMAR-III: An integrated humanoid plattfrom for sensory-motor control. In: In IEEE-RAS International Conference on Humanoid Robots.
5. Barto, A. (1995). Adaptive critics and the basal ganglia. In: J. C. Houk, J. L. Davis, & D. G. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 215-232). Cambridge: MIT.
6. Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983). Neuronlike elements that can solve difficult learning control problems. IEEE Transactions on Systems, Man, and Cybernetics, 13, 835-846.
7. Becker, S. (2005). A computational principle for hippocampal learning and neurogenesis. Hippocampus, 15(6), 722-738.
8. Berns, G. S., & Sejnowski, T. J. (1998). A computational model of how the basal ganglia produce sequences. The Journal of Cognitive Neuroscience, 10(1), 108-121.
9. Brown, J., Bullock, D., & Grossberg, S. (1999). How the basal ganglia use parallel excitatory and inhibitory learning pathways to selectively respond to unexpected rewarding cues. The Journal of Neuroscience, 19(23), 10502-10511.
10. Brown, M. A., & Sharp, P. E. (1995). Simulation of spatial learning in the Morris water maze by a neural network model of the hippocampal formation and nucleus accumbens. Hippocampus, 5(3), 171-188.
11. Burgess, N., Jackson, A., Hartley, T., & O'Keefe, J. (2000). Predictions derived from modelling the hippocampal role in navigation. Biological Cybernetics, 83, 301-312.
12. Contreras-Vidal, J. L., & Schultz, W. (1999). A predictive reinforcement model of dopamine neurons for learning approach behavior. Journal of Computational Neuroscience, 6, 191-214.
13. Dayan, P., & Abbott, L. F. (2005). Theoretical neuroscience: computational and mathematical modeling of neural systems. Cambridge: MIT.
14. Dudchenko, P. A., & Taube, J. S. (1997). Correlation between head-direction single unit activity and spatial behavior on a radial arm maze. Behavioral Neuroscience, 111, 3-19.
15. Eilam, D. (2004). Locomotor activity in common spiny mice (Acomys cahirinuse): The effect of light and environmental complexity. BMC Ecology, 4(16), 4-16.
16. Eilam, D., & Golani, I. (1989). Home base behavior of rats (rattus norvegicus) exploring a novel environment. Behavioural Brain Research, 34, 199-211.
17. Etienne, A. S., Maurer, R., & Seguinot, V. (1996). Path integration in mammals and its interaction with visual landmarks. Journal of Experimental Biology, 199 (Pt 1), 201-209.
18. Foster, D. J., Morris, R. G., & Dayan, P. (2000). A model of hippocampally dependent navigation, using the temporal difference learning rule. Hippocampus, 10(1), 1-16.
19. Glaubius, R., & Smart, W. D. (2004). Manifold representations for value function approximation. In: Proceedings of the AAAI-04 workshop on learning and planning in markov processes (pp. 13-189).
20. Golob, E. J., Stackman, R. W., Wong, A. C., & Taube, J. S. (2001). On the behavioural significance of head direction cells: Neural and behavioral dynamics during a spatial memory task. Behavioral Neuroscience, 115, 285-304.
21. Hartley, T., Burgess, N., Lever, C., Cacucci, F., & O'Keefe, J. (2000). Modeling place fields in terms of the cortical inputs to the hippocampus. Hippocampus, 10 (4), 369-379.
22. Hines, D. J., & Whishaw, I. Q. (2005). Home bases formed to visual cues but not to self-movement (dead reckoning) cues in exploring hippocampectomized rats. European Journal of Neuroscience, 22, 2363-2375.
23. Hollup, S. A., Molden, S., Donnett, J. G., Moser, M. B., & Moser, E. I. (2001). Accumulation of hippocampal place fields at the goal location in an annular watermaze task. The Journal of Neuroscience, 21(5), 1635-1644.
24. Houk, J. C., Adams, J. L., & Barto, A. G. (1995). A model of how the basal ganglia generate and use neural signals that predict reinforcement. In: J. C. Houk, J. L. Davis, & D. G. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 249-270). Cambridge: MIT.
25. Kaelbling, L. P., Littman, M., & Moore, A. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4, 237-285.
26. Kali, S., & Dayan, P. (2000). The involvement of recurrent connections in area CA3 in establishing the properties of place fields: A model. The Journal of Neuroscience, 20, 7463-7477.
27. Krichmar, J. L., Seth, A. K., Nitz, D. A., Fleischer, J. G., & Edelman, G. M. (2005). Spatial navigation and causal analysis in a brain-based device modeling cortical-hippocampal interactions. Neuroinformatics, 3(3), 197-221.
28. Kulvicius, T., Tamosiunaite, M., Ainge, J., Dudchenko, P., & Wörgötter, F. (2008). Odor supported place cell model and goal navigation in rodents. Journal of Computational Neuroscience. doi: 10.1007/s10827-008-0090-x.
29. Mehta, M. R., Barnes, C. A., & McNaughton, B. (1997). Experience-dependent, asymmetric expansion of hippocampal place fields. Proceedings of the Natianal Academy of Sciences, 94, 8918-8921.
30. Morris, R. (1981). Spatial localization does not require the presence of local cues. Learning and Motivation, 12, 239-260.
31. Morris, R. (1984). Developments of a water-maze procedure for studying spatial learning in the rat. Journal of Neuroscience Methods, 11(1), 47-60.
32. Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006). Midbrain dopamine neurons encode decisions for future action. Nature Neuroscience, 9(8), 1057-1063.
33. Muller, R. (1996). A quarter of a century of place cells. Neuron, 17, 813-822.
34. Nemati, F., & Whishaw, I. Q. (2007). The point of entry contributes to the organization of exploratory behaviour of rats on an open field: An example of spontaneous episodic memory. Behavioural Brain Research, 182, 119-128.
35. Niv, Y., Daw, N. D., & Dayan, P. (2006). Choice values. Nature Neuroscience, 9(8), 987-988.
36. O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science, 304, 452-454.
37. O'Keefe, J., & Burgess, N. (1996). Geometric determinants of the place fields of hippocampal neurons. Nature, 381 (6581), 425-428.
38. Okhawa, K., Shibata, T., & Tanie, K. (1998). Method for generating of global cooperation based on local communication. In: Proceedings of the 1998 IEEE/RSJ intl. conference on intelligent robots and systems (pp. 108-113).
39. Recce, M., & Harris, K. D. (1996). Memory for places: A navigational model in support of Marr's theory of hippocampal function. Hippocampus, 6(6), 735-748.
40. Redgrave, P., & Gurney, K. N. (2006). The short-latency dopamine signal: A role in discovering. Nature Reviews Neuroscience, 7(12), 967-975.
41. Redgrave, P., Prescott, T. J., & Gurney, K. (1999). Is the short-latency dopamine response too short to signal reward error? Trends in Neurosciences, 22(4), 146-151.
42. Reynolds, S. I. (2002). The stability of general discounted reinforcement learning with linear function approximation. In: UK workshop on computational intelligence (UKCI-02) (pp. 139-146).
43. Samsonovich, A., & McNaughton, B. L. (1997). Path integration and cognitive mapping in a continuous attractor neural network model. The Journal of Neuroscience, 17(15), 5900-5920.
44. Save, E., Nerad, L., & Poucet, B. (2000). Contribution of multiple sensory information to place field stability in hippocampal place cells. Hippocampus, 10(1), 64-76.
45. Schultz, W. (2002). Getting formal with dopamine and reward. Neuron, 36, 241-263.
46. Schultz, W. (2007). Reward Signals. Scholarpedia, http://www.scholarpedia.org/article/Reward_Signals.
47. Sharp, P. E., Blair, H. T., & Cho, J. (2001). The anatomical and computational basis of the rat head-direction cell signal. Trends in Neurosciences, 24(5), 289-94.
48. Sheynikhovich, D., Chavarriaga, R., Strösslin, T., & Gerstner, W. (2005). Spatial representation and navigation in a bio-inspired robot. In: S. Wermter (Ed.), Biomimetic Neural Learning for Intelligent Robots: Intelligent Systems, Cognitive Robotics, and Neuroscience (pp. 245-264). New York: Springer.
49. Strösslin, T., Sheynikhovich, D., Chavarriaga, R., & Gerstner, W. (2005). Robust self-localisation and navigation based on hippocampal place cells. Neural Networks, 18(9), 1125-1140.
50. Stuart, A., Ord, K., & Arnold, S. (1999). Kendall's advanced theory of statistics. London: Arnold, a member of the Hodder Headline Group.
51. Sutton, R. S. (1988). Learning to predict by the methods of temporal differences. Machine Learning, 3, 9-44.
52. Sutton, R., & Barto, A. (1998). Reinforcement learning: An introduction. Cambridge: MIT.
53. Szepesvari, C., & Smart, W. D. (2004). Interpolation-based Q-learning. In: Twenty-First international conference on machine learning (ICML04) (vol. 21, pp. 791-798).
54. Tesauro, G. (1995). Temporal difference learning and TD-gammon. Communications of the ACM, 38(3), 58-67.
55. Tsitsiklis, J. N., & Van Roy, B. (1997). An Analysis of temporal-difference learning with function approximation. IEEE Transactions on Automatic Control, 42(5), 674-690.
56. Tsodyks, M. (1999). Attractor neural network models of spatial maps in hippocampus. Hippocampus, 9(4), 481-489.
57. Wallace, D. G., Gorny, B., & Whishaw, I. Q. (2002). Rats can track odors, other rats, and themselves: Implications for the study of spatial behavior. Behavioural Brain Research, 131(1-2), 185-192.
58. Watkins, C. J., & Dayan, P. (1992). Q-learning. Machine Learning, 8, 279-292.
59. Werbos, P. J. (1988). Backpropagation: Past and future. In: Proceedings of the IEEE international conference on neural networks (pp. 343-353).
60. Whishaw, I. Q., Hines, D. J., & Wallace, D. G. (2001). Dead reckoning (path integration) requires the hippocampal formation: Evidence from spontaneous exploration and spatial learning tasks in light (allothetic) and dark (idiothetic) tests. Behavioural Brain Research, 127(1-2), 49-69.
61. Wiering, M. (2004). Convergence and divergence in standard averaging reinforcement learning. In: J. Boulicaut, F. Esposito, F. Giannotti, & D. Pedreschi (Eds.), Proceedings of the 15th European conference on machine learning ECML'04 (pp. 477-488).
62. Wilson, M. A., & McNaughton, B. L. (1993). Dynamics of the hippocampal ensemble code for space. Science, 261(5124), 1055-1058.
63. Wörgötter, F., & Porr, B. (2005). Temporal sequence learning, prediction, and control: A review of different models and their relation to biological mechanisms. Neural Computation, 17(2), 245-319.
64. Wörgötter, F., & Porr, B. (2007). Reinforcement Learning. Scholar pedia, http://www.scholarpedia.org/article/ Reinforcement_Learning.
65. Yen, G. G., & Hickey, T. (2004). Reinforcement learning algorithms for robotic navigation in dynamic environments. ISA Transactions, 43(2), 217-230.
66. Zadicario, P., Avni, R., Zadicario, E., & Eilam, D. (2005). 'Looping': An exploration and navigation mechanism in a dark open field. Behavioural Brain Research, 159, 27-36.