Path planning versus cue responding: A bio-inspired model of switching between navigation strategies

Biological Cybernetics

Volumn 103, Issue 4, 2010, Pages 299-317

  Dollé, Laurent ^a   Sheynikhovich, Denis ^a,b   Girard, Benoît ^a   Chavarriaga, Ricardo ^c   Guillot, Agnès ^a  

^a UNIVERSITÉ PIERRE ET MARIE CURIE   (France)

^b Groupe Enzymologie Moléculaire et Fonctionnelle   (France)

^c EPFL   (Switzerland)

Author keywords

Action selection; Computational model; Parallel memory systems; Spatial navigation; Strategy switch

Indexed keywords

ACTION SELECTION; COMPUTATIONAL MODEL; PARALLEL MEMORY; SPATIAL NAVIGATION; STRATEGY SWITCH;

NAVIGATION;

LEARNING ALGORITHMS;

RATTUS;

ALGORITHM; ANIMAL; ARTICLE; ASSOCIATION; ATTENTION; BIOLOGICAL MODEL; COMPUTER SIMULATION; DEPTH PERCEPTION; HUMAN; LEARNING; PHYSIOLOGY; TIME;

ALGORITHMS; ANIMALS; ATTENTION; COMPUTER SIMULATION; CUES; HUMANS; LEARNING; MODELS, NEUROLOGICAL; SPACE PERCEPTION; TIME FACTORS;

EID: 77957752233     PISSN: 03401200     EISSN: 14320770     Source Type: Journal    
DOI: 10.1007/s00422-010-0400-z     Document Type: Article

References (76)

1. A Arleo W Gerstner 2000 Spatial cognition and neuro-mimetic navigation: a model of hippocampal place cell activity Biol Cybern 83 3 287 299 1:STN:280:DC%2BD3cvltFKitg%3D%3D 10.1007/s004220000171 11007302
2. A Arleo L Rondi-Reig 2007 Multimodal sensory integration and concurrent navigation strategies for spatial cognition in real and artificial organisms J Integr Neurosci 6 3 327 366 10.1142/S0219635207001593 17933016 (Pubitemid 47570594)
3. Barrera A, Weitzenfeld A (2007) Bio-inspired model of robot spatial cognition: topological place recognition and target learning. In: CIRA, pp 61-66
4. Blaisdell A (2009) The role of associative processes in spatial, temporal, and causal cognition. In: Watanabe SB, Blaisdell AP, Huber L, Young A (eds) Rational animals, irrational humans. Keio University Press, Tokyo, pp 153-172
5. M Brown P Sharp 1995 Simulation of spatial learning in the Morris water maze by a neural network model of the hippocampal formation and nucleus accumbens Hippocampus 5 3 171 188 1:STN:280:DyaK28%2FktVKrtw%3D%3D 10.1002/hipo.450050304 7550613
6. N Burgess 2008 Spatial cognition and the brain Ann N Y Acad Sci 1124 77 97 10.1196/annals.1440.002 18400925 (Pubitemid 351506780)
7. Y Burnod 1991 Organizational levels of the cerebral cortex: an integrated model Acta Biotheor 39 3-4 351 361 1:STN:280:DyaK383otVehuw%3D%3D 10.1007/BF00114187 1816714
8. C Canal S Stutz P Gold 2005 Glucose injections into the dorsal hippocampus or dorsolateral striatum of rats prior to T-maze training: modulation of learning rates and strategy selection Learn Mem 12 4 367 374 10.1101/lm.88205 16027177 (Pubitemid 41126849)
9. V Chamizo 2003 Acquisition of knowledge about spatial location: assessing the generality of the mechanism of learning Q J Exp Psychol 56 1 102 113 1:STN:280:DC%2BD3s7htFeksg%3D%3D 10.1080/02724990244000205 (Pubitemid 37353306)
10. Q Chang PE Gold 2003 Switching memory systems during learning: changes in patterns of brain acetylcholine release in the hippocampus and striatum in rats J Neurosci 23 7 3001 1:CAS:528:DC%2BD3sXjtFCnur0%3D 12684487
11. Q Chang PE Gold 2004 Inactivation of dorsolateral striatum impairs acquisition of response learning in cue-deficient, but not cue-available, conditions Behav Neurosci 118 2 383 388 10.1037/0735-7044.118.2.383 15113264 (Pubitemid 38445615)
12. R Chavarriaga T Strösslin D Sheynikhovich W Gerstner 2005 A computational model of parallel navigation systems in rodents Neuroinformatics 3 3 223 242 10.1385/NI:3:3:223 16077160 (Pubitemid 41114886)
13. F Conde E Maire-Lepoivre E Audinat F Crepel 1995 Afferent connections of the medial frontal cortex of the rat. II. Cortical and subcortical afferents J Comp Neurol 352 4 567 593 1:STN:280:DyaK2M3jvVKjug%3D%3D 10.1002/cne.903520407 7722001
14. N Cuperlier M Quoy P Gaussier 2007 Neurobiologically inspired mobile robot navigation and planning Front Neurorobotics 1 1 15
15. ND Daw Y Niv P Dayan 2005 Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control Nat Neurosci 8 12 1704 1711 1:CAS:528:DC%2BD2MXht1GisrfL 10.1038/nn1560 16286932 (Pubitemid 41683198)
16. L Descarries B Lemay G Doucet B Berger 1987 Regional and laminar density of the dopamine innervation in adult rat cerebral cortex Neuroscience 21 3 807 824 1:CAS:528:DyaL2sXkvFSksr0%3D 10.1016/0306-4522(87)90038-8 3627435 (Pubitemid 17106386)
17. B Devan N White 1999 Parallel information processing in the dorsal striatum: relation to hippocampal function J Neurosci 19 7 2789 2798 1:CAS:528:DyaK1MXitFShtbw%3D 10087090
18. B Devan R McDonald N White 1999 Effects of medial and lateral caudate-putamen lesions on place-and cue-guided behaviors in the water maze: relation to thigmotaxis Behav Brain Res 100 1-2 5 14 1:STN:280: DyaK1M3islSgsw%3D%3D 10.1016/S0166-4328(98)00107-7 10212049 (Pubitemid 29130236)
19. E Dijkstra 1959 A note on two problems in connection with graphs Numer Math 1 269-270 269 271 10.1007/BF01386390
20. CF Doeller N Burgess 2008 Distinct error-correcting and incidental learning of location relative to landmarks and boundaries Proc Natl Acad Sci USA 105 15 5909 5914 1:CAS:528:DC%2BD1cXltVynt7k%3D 10.1073/pnas.0711433105 18413609 (Pubitemid 351758472)
21. CF Doeller JA King N Burgess 2008 Parallel striatal and hippocampal systems for landmarks and boundaries in spatial memory Proc Natl Acad Sci USA 105 15 5915 5920 1:CAS:528:DC%2BD1cXltVynu7w%3D 10.1073/pnas.0801489105 18408152 (Pubitemid 351758473)
22. L Dolle M Khamassi B Girard A Guillot R Chavarriaga 2008 Analyzing interactions between navigation strategies using a computational model of action selection LNAI 5248 71 86
23. DJ Foster RG Morris P Dayan 2000 A model of hippocampally dependent navigation, using the temporal difference learning rule Hippocampus 10 1 1 16 1:STN:280:DC%2BD3c7mvVOhuw%3D%3D 10.1002/(SICI)1098-1063(2000)10:1<1::AID- HIPO1>3.0.CO;2-1 10706212 (Pubitemid 30107585)
24. MO Franz HA Mallot 2000 Biomimetic robot navigation Rob Auton Syst 30 1 133 153 10.1016/S0921-8890(99)00069-X (Pubitemid 30546262)
25. B Gibson S Shettleworth 2003 Competition among spatial cues in a naturalistic food-carrying task Learn Behav 31 2 143 159 12882373
26. B Gibson S Shettleworth 2005 Place versus response learning revisited: tests of blocking on the radial maze Behav Neurosci 119 2 567 586 10.1037/0735-7044.119.2.567 15839803 (Pubitemid 40547962)
27. B Girard D Filliat J Meyer A Berthoz A Guillot 2005 Integration of navigation and action selection functionalities in a computational model of cortico-basal-thalamo-cortical loops Adapt Behav 13 2 115 130 10.1177/105971230501300204 (Pubitemid 40907084)
28. P Gold 2004 Coordination of multiple memory systems Neurobiol Learn Mem 82 3 230 242 10.1016/j.nlm.2004.07.003 15464406 (Pubitemid 39298718)
29. J Grahn J Parkinson A Owen 2008 The cognitive functions of the caudate nucleus Prog Neurobiol 86 3 141 155 10.1016/j.pneurobio.2008.09.004 18824075
30. S Granon B Poucet 1995 Medial prefrontal lesions in the rat and spatial navigation: evidence for impaired planning Behav Neurosci 109 3 474 484 1:STN:280:DyaK2MzptFWhtQ%3D%3D 10.1037/0735-7044.109.3.474 7662158
31. Groenewegen H, Berendse H, Meredith G, Haber S, Voorn P, Wolters J, Lohman A (1991) The mesolimbic dopamine system: from motivation to action. In: Willner P, Scheel-Kriiger J (eds) Functional anatomy of the ventral, limbic system-innervated striatum. Wiley, Chichester, pp 19-59
32. A Guazzelli F Corbacho M Bota M Arbib 1998 Affordances, motivation, and the world graph theory Adapt Behav 6 3 435 471 10.1177/105971239800600305 (Pubitemid 128184953)
33. D Hamilton C Rosenfelt I Whishaw 2004 Sequential control of navigation by locale and taxon cues in the morris water task Behav Brain Res 154 2 385 397 10.1016/j.bbr.2004.03.005 15313026 (Pubitemid 39093989)
34. T Hartley N Burgess 2005 Complementary memory systems: competition, cooperation and compensation Trends Neurosci 28 4 169 170 1:CAS:528: DC%2BD2MXivV2htr4%3D 10.1016/j.tins.2005.02.004 15808348 (Pubitemid 40445520)
35. ME Hasselmo 2005 A model of prefrontal cortical mechanisms for goal-directed behavior J Cogn Neurosci 17 7 1115 1129 10.1162/0898929054475190 16102240 (Pubitemid 41045582)
36. J Jankowski L Scheef C Hüppe H Boecker 2009 Distinct striatal regions for planning and executing novel and automated movement sequences Neuroimage 44 4 1369 1379 1:STN:280:DC%2BD1M%2FntlOnsA%3D%3D 10.1016/j.neuroimage.2008.10.059 19059350
37. D Kelly B Gibson 2007 Spatial navigation: spatial learning in real and virtual environments Comp Cogn Behav Rev 2 111 124
38. Khamassi M (2007) Complementary roles of the rat prefrontal cortex and striatum in reward-based learning and shifting navigation strategies. PhD thesis, University Paris 6
39. J Kim M Baxter 2001 Multiple brain-memory systems: the whole does not equal the sum of its parts Trends Neurosci 24 6 324 330 1:CAS:528: DC%2BD3MXjsFamu7w%3D 10.1016/S0166-2236(00)01818-X 11356503 (Pubitemid 32455432)
40. K Leising A Blaisdell 2009 Associative basis of landmark learning and integration in vertebrates Comp Cogn Behav Rev 4 80 102 20607101
41. LE Martinet JB Passot B Fouque JA Meyer A Arleo 2008 Map-based spatial navigation: a cortical column model for action planning LNAI 5248 39 55
42. R McDonald N White 1993 A triple dissociation of memory systems: hippocampus, amygdala, and dorsal striatum Behav Neurosci 107 1 3 22 1:STN:280:DyaK3s7ptF2msw%3D%3D 10.1037/0735-7044.107.1.3 8447956
43. R McDonald N White 1994 Parallel information processing in the water maze: evidence for independent memory systems involving dorsal striatum and hippocampus Behav Neural Biol 61 3 260 270 1:STN:280:DyaK2czkslSksw%3D%3D 10.1016/S0163-1047(05)80009-3 8067981 (Pubitemid 24176851)
44. R McDonald B Devan N Hong 2004 Multiple memory systems: the power of interactions Neurobiol Learn Mem 82 3 333 346 10.1016/j.nlm.2004.05.009 15464414 (Pubitemid 39298725)
45. Mizumori S (2008) Hippocampal place fields. Oxford University Press, USA
46. J O'Keefe J Dostrovsky 1971 The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat Brain Res 34 1 171 175 10.1016/0006-8993(71)90358-1 5124915
47. O'Keefe J, Nadel L (1978) The hippocampus as a cognitive map. Oxford University Press, Oxford
48. M Packard J McGaugh 1992 Double dissociation of fornix and caudate nucleus lesions on acquisition of two water maze tasks: further evidence for multiple memory systems Behav Neurosci 106 3 439 446 1:STN:280: DyaK38zhtlGjtg%3D%3D 10.1037/0735-7044.106.3.439 1616610
49. M Packard J McGaugh 1996 Inactivation of hippocampus or caudate nucleus with lidocaine differentially affects expression of place and response learning Neurobiol Learn Mem 65 1 65 72 1:CAS:528:DyaK28Xht1Clu7o%3D 10.1006/nlme.1996. 0007 8673408 (Pubitemid 26112655)
50. M Packard R Hirsh N White 1989 Differential effects of fornix and caudate nucleus lesions on two radial maze tasks: evidence for multiple memory systems J Neurosci 9 1465 1472 1:STN:280:DyaL1M3lslGqtQ%3D%3D 2723738
51. J Pearce 2009 The 36th Sir Frederick Bartlett Lecture: an associative analysis of spatial learning Q J Exp Psychol 62 9 1665 1684 10.1080/ 17470210902805589
52. J Pearce A Roberts M Good 1998 Hippocampal lesions disrupt navigation based on cognitive maps but not heading vectors Nature 396 6706 75 77 1:CAS:528:DyaK1cXnsVSjtb4%3D 10.1038/23941 9817202 (Pubitemid 28520451)
53. B Pouzet W Zhang J Feldon J Rawlins 2002 Hippocampal lesioned rats are able to learn a spatial position using non-spatial strategies Behav Brain Res 133 2 279 291 10.1016/S0166-4328(02)00007-4 12110461 (Pubitemid 34754403)
54. M Ragozzino S Detrick R Kesner 1999 Involvement of the prelimbic-infralimbic areas of the rodent prefontal cortex in behavioral flexibility for place and response learning J Neurosci 19 11 4585 4594 1:CAS:528:DyaK1MXjsFahtbc%3D 10341256
55. Redish A (1999) Beyond the cognitive map: from place cells to episodic memory. The MIT Press, Cambridge
56. Rescorla R, Wagner A (1972) A theory of pavlovian conditioning: the effectiveness of reinforcement and non-reinforcement. In: Black A, Prokasy W (eds) Classical conditioning II: current research and theory. Appleton-Century-Crofts, New York, pp 64-69
57. E Rich M Shapiro 2007 Prelimbic/infralimbic inactivation impairs memory for multiple task switches, but not flexible selection of familiar tasks J Neurosci 27 17 4747 1:CAS:528:DC%2BD2sXltF2ju7k%3D 10.1523/JNEUROSCI.0369-07. 2007 17460087 (Pubitemid 46663585)
58. E Rich M Shapiro 2009 Rat prefrontal cortical neurons selectively code strategy switches J Neurosci 29 22 7208 7219 1:CAS:528:DC%2BD1MXntFGktb0%3D 10.1523/JNEUROSCI.6068-08.2009 19494143
59. A Roberts J Pearce 1999 Blocking in the Morris swimming pool J Exp Psychol Anim Behav Process 25 2 225 235 1:STN:280:DyaK1M3mtVOrug%3D%3D 10.1037/0097-7403.25.2.225 10331921
60. E Save B Poucet 2000 Involvement of the hippocampus and associative parietal cortex in the use of proximal and distal landmarks for navigation Behav Brain Res 109 2 195 206 1:STN:280:DC%2BD3c3ltVOhtQ%3D%3D 10.1016/S0166-4328(99) 00173-4 10762689 (Pubitemid 30179669)
61. D Sheynikhovich R Chavarriaga T Strösslin A Arleo W Gerstner 2009 Is there a geometric module for spatial orientation? Insights from a rodent navigation model Psychol Rev 116 3 540 566 10.1037/a0016170 19618986
62. W Skaggs J Knierim H Kudrimoti B McNaughton 1995 A model of the neural basis of the rat's sense of direction Adv Neural Inf Process Syst 7 173 182 1:STN:280:DC%2BD3MnlslOgsw%3D%3D 11539168
63. W Stahlman A Blaisdell 2009 Blocking of spatial control by landmarks in rats Behav Processes 81 1 114 118 10.1016/j.beproc.2009.02.007 19429205
64. T Strösslin D Sheynikhovich R Chavarriaga W Gerstner 2005 Robust self-localisation and navigation based on hippocampal place cells Neural Netw 18 9 1125 1140 10.1016/j.neunet.2005.08.012 16263241 (Pubitemid 41643898)
65. Sutton R, Barto A (1998) Reinforcement learning: an introduction. Bradford Book. The MIT Press, Cambridge
66. JS Taube RU Muller JB Ranck Jr 1990 Head-direction cells recorded from the postsubiculum in freely moving rats. I. Description and quantitative analysis J Neurosci 10 2 420 1:STN:280:DyaK3c7lsFCisA%3D%3D 2303851
67. D Touretzky A Redish 1996 Theory of rodent navigation based on interacting representations of space Hippocampus 6 3 247 270 1:STN:280:DyaK28vjvFGisw%3D%3D 10.1002/(SICI)1098-1063(1996)6:3<247::AID- HIPO4>3.0.CO;2-K 8841825 (Pubitemid 26275786)
68. Uchibe E, Doya K (2005) Reinforcement learning with multiple heterogeneous modules: a framework for developmental robot learning. In: The 4th international conference on development and learning. IEEE Computer Society Press, pp 87-92
69. B Ujfalussy P Eros Z Somogyvari T Kiss 2008 Episodes in space: a modelling study of hippocampal place representation LNAI 5040 123 136
70. N Voermans K Petersson L Daudey B Weber K Van Spaendonck H Kremer G Fernández 2004 Interaction between the human hippocampus and the caudate nucleus during route recognition Neuron 43 3 427 435 1:CAS:528: DC%2BD2cXmvFOisbs%3D 10.1016/j.neuron.2004.07.009 15294149 (Pubitemid 39061133)
71. N White 2004 The role of stimulus ambiguity and movement in spatial navigation: a multiple memory systems analysis of location discrimination Neurobiol Learn Mem 82 216 229 10.1016/j.nlm.2004.05.004 15464405 (Pubitemid 39298717)
72. N White 2009 Some highlights of research on the effects of caudate nucleus lesions over the past 200 years Behav Brain Res 199 1 3 23 10.1016/j.bbr.2008.12.003 19111791
73. N White R McDonald 2002 Multiple parallel memory systems in the brain of the rat Neurobiol Learn Mem 77 125 184 10.1006/nlme.2001.4008 11848717 (Pubitemid 34971650)
74. H Yin B Knowlton 2004 Contributions of striatal subregions to place and response learning Learn Mem 11 4 459 463 10.1101/lm.81004 15286184 (Pubitemid 39079705)
75. J Young M Shapiro 2009 Double dissociation and hierarchical organization of strategy switches and reversals in the rat PFC Behav Neurosci 123 5 1028 1035 10.1037/a0016822 19824768
76. K Zhang 1996 Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: a theory J Neurosci 16 6 2112 1:CAS:528:DyaK28XhvVGns7w%3D 8604055