Spatial learning and action planning in a prefrontal cortical network model

PLoS Computational Biology

Volumn 7, Issue 5, 2011, Pages

  Martinet, Louis Emmanuel ^a   Sheynikhovich, Denis ^a   Benchenane, Karim ^a   Arleo, Angelo ^a  

^a Groupe Enzymologie Moléculaire et Fonctionnelle   (France)

Author keywords

[No Author keywords available]

Indexed keywords

BRAIN; DECISION MAKING; MAMMALS;

ACTION PLANNING; COLUMNAR ORGANIZATION; CORTICAL NETWORK MODEL; DECISIONS MAKINGS; DISTRIBUTED INFORMATION PROCESSING; PREFRONTAL CORTEX; SPATIAL ACTION; SPATIAL COGNITION; SPATIAL LEARNING; SYNAPTIC ADAPTATION;

NEURONS;

ADAPTATION; ARTICLE; BEHAVIOR; COGNITION; HIPPOCAMPUS; INFORMATION PROCESSING; LEARNING; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; MOLECULAR DYNAMICS; NERVE CELL NETWORK; NEUROSCIENCE; NONHUMAN; PREFRONTAL CORTEX; RECEPTIVE FIELD; SYNAPSE; TASK PERFORMANCE; ANIMAL; BIOLOGICAL MODEL; PHYSIOLOGY; RAT;

RODENTIA;

ANIMALS; COGNITION; HIPPOCAMPUS; LEARNING; MODELS, NEUROLOGICAL; NERVE NET; PREFRONTAL CORTEX; RATS; SPATIAL BEHAVIOR;

EID: 79958130868     PISSN: 1553734X     EISSN: 15537358     Source Type: Journal    
DOI: 10.1371/journal.pcbi.1002045     Document Type: Article

References (118)

1. O'Keefe J, Nadel L, (1978) The hippocampus as a cognitive map. Oxford Oxford University Press.
2. Trullier O, Wiener SI, Berthoz A, Meyer JA, (1997) Biologically based artificial navigation systems: review and prospects. Prog Neurobiol 51: 483-544.
3. Arleo A, Rondi-Reig L, (2007) Multimodal sensory integration and concurrent navigation strategies for spatial cognition in real and artificial organisms. J Integr Neurosci 6: 327-366.
4. Tolman EC, Honzik CH, (1930) Insight in rats. Publ Psychol 4: 215-232.
5. Poucet B, (1993) Spatial cognitive maps in animals: new hypotheses on their structure and neural mechanisms. Psychol Rev 100: 163-182.
6. Meyer J, Filliat D, (2003) Map-based navigation in mobile robots: II. a review of map-learning and path-planning strategies. Cogn Syst Res 4: 283-317.
7. Mallot HA, Basten K, (2009) Embodied spatial cognition: Biological and artificial systems. Image Vision Comput 27: 1658-1670.
8. Etienne AS, Jeffery KJ, (2004) Path integration in mammals. Hippocampus 14: 180-192.
9. White NM, McDonald RJ, (2002) Multiple parallel memory systems in the brain of the rat. Neurobiol Learn Mem 77: 125-184.
10. Poldrack RA, Packard MG, (2003) Competition among multiple memory systems: converging evidence from animal and human brain studies. Neuropsychologia 41: 245-51.
11. Dollé L, Sheynikhovich D, Girard B, Chavarriaga R, Guillot A, (2010) Path planning versus cue responding: a bio-inspired model of switching between navigation strategies. Biol Cybern 103: 299-317.
12. McNaughton BL, Battaglia FP, Jensen O, Moser EI, Moser M, (2006) Path integration and the neural basis of the 'cognitive map'. Nat Rev Neurosci 7: 663-678.
13. Hafting T, Fyhn M, Molden S, Moser MB, Moser EI, (2005) Microstructure of a spatial map in the entorhinal cortex. Nature 436: 801-806.
14. Wiener SI, Taube JS, (2005) Head Direction Cells and the Neural Mechansims of Spatial Orientation. Cambridge, MA MIT Press.
15. Poucet B, Lenck-Santini PP, Hok V, Save E, Banquet JP, et al. (2004) Spatial navigation and hippocampal place cell firing: the problem of goal encoding. Rev Neurosci 15: 89-107.
16. Muller RU, Stead M, Pach J, (1996) The hippocampus as a cognitive graph. J Gen Physiol 107: 663-694.
17. Amaral DG, Witter MP, (1989) The three-dimensional organization of the hippocampal formation: a review of anatomical data. Neuroscience 31: 571-591.
18. Wilson MA, McNaughton BL, (1993) Dynamics of the hippocampal ensemble code for space. Science 261: 1055-1058.
19. Nitz DA, (2006) Tracking route progression in the posterior parietal cortex. Neuron 49: 747-756.
20. Hok V, Save E, Lenck-Santini PP, Poucet B, (2005) Coding for spatial goals in the prelimbic/infralimbic area of the rat frontal cortex. Proc Natl Acad Sci U S A 102: 4602-4607.
21. Fuster JM, (1991) The prefrontal cortex: Anatomy, physiology and neurophysiology of the frontal lobe. New York Raven.
22. Knierim JJ, (2006) Neural representations of location outside the hippocampus. Learn Mem 13: 405-415.
23. Spiers HJ, Maguire EA, (2007) The neuroscience of remote spatial memory: a tale of two cities. Neuroscience 149: 7-27.
24. Shallice T, Burgess PW, (1991) Deficits in strategy application following frontal lobe damage in man. Brain 114: 727-741.
25. Granon S, Poucet B, (1995) Medial prefrontal lesions in the rat and spatial navigation: evidence for impaired planning. Behav Neurosci 109: 474-484.
26. Spiers HJ, Maguire EA, (2006) Thoughts, behaviour, and brain dynamics during navigation in the real world. Neuroimage 31: 1826-1840.
27. Spiers HJ, Maguire EA, (2007) A navigational guidance system in the human brain. Hippocampus 17: 618-626.
28. Jay TM, Witter MP, (1991) Distribution of hippocampal CA1 and subicular efferents in the prefrontal cortex of the rat studied by means of anterograde transport of phaseolus vulgaris-leucoagglutinin. J Comp Neurol 313: 574-586.
29. Vertes RP, (2006) Interactions among the medial prefrontal cortex, hippocampus and midline thalamus in emotional and cognitive processing in the rat. Neuroscience 142: 1-20.
30. Kita H, Kitai ST, (1990) Amygdaloid projections to the frontal cortex and the striatum in the rat. J Comp Neurol 298: 40-49.
31. Thierry AM, Blanc G, Sobel A, Stinus L, Golwinski J, (1973) Dopaminergic terminals in the rat cortex. Science 182: 499-501.
32. Uylings HBM, Groenewegen HJ, Kolb B, (2003) Do rats have a prefrontal cortex? Behav Brain Res 146: 3-17.
33. Aggleton JP, (1992) The amygdala: neurobiological aspects of emotion, memory, and mental dysfunction. Wiley-Liss New York.
34. Schultz W, (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80: 1-27.
35. Jung MW, Qin Y, McNaughton BL, Barnes CA, (1998) Firing characteristics of deep layer neurons in prefrontal cortex in rats performing spatial working memory tasks. Cereb Cortex 8: 437-450.
36. Otani S, (2003) Prefrontal cortex function, quasi-physiological stimuli, and synaptic plasticity. J Physiol Paris 97: 423-430.
37. Rich EL, Shapiro M, (2009) Rat prefrontal cortical neurons selectively code strategy switches. J Neurosci 29: 7208-7219.
38. Fuster JM, (2001) The prefrontal cortex-an update: time is of the essence. Neuron 30: 319-333.
39. Cohen JD, Braver TS, Brown JW, (2002) Computational perspectives on dopamine function in prefrontal cortex. Curr Opin Neurobiol 12: 223-229.
40. Mountcastle VB, (1957) Modality and topographic properties of single neurons of cat's somatic sensory cortex. J Neurophysiol 20: 408-434.
41. Sheynikhovich D, Chavarriaga R, Strösslin T, Arleo A, Gerstner W, (2009) Is there a geometric module for spatial orientation? insights from a rodent navigation model. Psychol Rev 116: 540-566.
42. Lei G, (1990) A neuron model with fluid properties for solving labyrinthian puzzle. Biol Cybern 64: 61-67.
43. Mataric MJ, (1992) Integration of representation into goal-driven behavior-based robots. IEEE Trans Rob Autom 8: 304-312.
44. Hasselmo ME, (2005) A model of prefrontal cortical mechanisms for goal-directed behavior. J Cogn Neurosci 17: 1115-1129.
45. Mountcastle VB, (1997) The columnar organization of the neocortex. Brain 120: 701-722.
46. Gabbott PL, Bacon SJ, (1996) The organisation of dendritic bundles in the prelimbic cortex (area 32) of the rat. Brain Res 730: 75-86.
47. Gabbott PLA, Warner TA, Jays PRL, Salway P, Busby SJ, (2005) Prefrontal cortex in the rat: projections to subcortical autonomic, motor, and limbic centers. J Comp Neurol 492: 145-177.
48. Buxhoeveden DP, Casanova MF, (2002) The minicolumn hypothesis in neuroscience. Brain 125: 935-951.
49. Burnod Y, (1988) An adaptative neural network: the cerebral cortex. Masson.
50. Szentágothai J, (1975) The 'module-concept' in cerebral cortex architecture. Brain Res 95: 475-496.
51. Eccles JC, (1981) The modular operation of the cerebral neocortex considered as the material basis of mental events. Neuroscience 6: 1839-1856.
52. Burnod Y, (1991) Organizational levels of the cerebral cortex: an integrated model. Acta Biotheor 39: 351-361.
53. Markram H, (2006) The blue brain project. Nat Rev Neurosci 7: 153-160.
54. Rao SG, Williams GV, Goldman-Rakic PS, (1999) Isodirectional tuning of adjacent interneurons and pyramidal cells during working memory: evidence for microcolumnar organization in PFC. J Neurophysiol 81: 1903-1916.
55. Asaad WF, Rainer G, Miller EK, (1998) Neural activity in the primate prefrontal cortex during associative learning. Neuron 21: 1399-1407.
56. Lewis DA, Melchitzky DS, Burgos G, (2002) Specificity in the functional architecture of primate prefrontal cortex. J Neurocytol 31: 265-276.
57. Arleo A, Gerstner W, (2000) Spatial cognition and neuro-mimetic navigation: a model of hippocampal place cell activity. Biol Cybern 83: 287-299.
58. Arleo A, Smeraldi F, Gerstner W, (2004) Cognitive navigation based on nonuniform gabor space sampling, unsupervised growing networks, and reinforcement learning. IEEE Trans Neural Netw 15: 639-651.
59. Peyrache A, Khamassi M, Benchenane K, Wiener SI, Battaglia FP, (2009) Replay of rule-learning related neural patterns in the prefrontal cortex during sleep. Nat Neurosci 12: 919-926.
60. Benchenane K, Peyrache A, Khamassi M, Tierney PL, Gioanni Y, et al. (2010) Coherent theta oscillations and reorganization of spike timing in the hippocampal- prefrontal network upon learning. Neuron 66: 921-936.
61. Watanabe M, (1996) Reward expectancy in primate prefrontal neurons. Nature 382: 629-632.
62. Rainer G, Rao SC, Miller EK, (1999) Prospective coding for objects in primate prefrontal cortex. J Neurosci 19: 5493-5505.
63. Tremblay L, Schultz W, (1999) Relative reward preference in primate orbitofrontal cortex. Nature 398: 704-708.
64. Miller EK, Cohen JD, (2001) An integrative theory of prefrontal function. Annu Rev Neurosci 24: 167-202.
65. Averbeck BB, Chafee MV, Crowe DA, Georgopoulos AP, (2002) Parallel processing of serial movements in prefrontal cortex. Proc Natl Acad Sci U S A 99: 13172-13177.
66. Mulder AB, Nordquist RE, Orgüt O, Pennartz CMA, (2003) Learning-related changes in response patterns of prefrontal neurons during instrumental conditioning. Behav Brain Res 146: 77-88.
67. Skaggs WE, McNaughton BL, Gothard KM, Markus EJ, (1992) An information-theoretic approach to deciphering the hippocampal code. Adv Neural Inf Process Syst pp. 1030-1037.
68. Bezzi M, Samengo I, Leutgeb S, Mizumori SJ, (2001) Measuring information spatial densities. Neural Comput 14: 405-420.
69. Horton JC, Adams DL, (2005) The cortical column: a structure without a function. Philos Trans R Soc Lond B Biol Sci 360: 837-862.
70. Rakic P, (2008) Confusing cortical columns. Proc Natl Acad Sci U S A 105: 12099-12100.
71. Vogels TP, Rajan K, Abbott LF, (2005) Neural network dynamics. Annu Rev Neurosci 28: 357-376.
72. Wu J, Huang X, Zhang C, (2008) Propagating waves of activity in the neocortex: what they are, what they do. Neuroscientist 14: 487-502.
73. Rubino D, Robbins KA, Hatsopoulos NG, (2006) Propagating waves mediate information transfer in the motor cortex. Nat Neurosci 9: 1549-1557.
74. Graybiel AM, (2008) Habits, rituals, and the evaluative brain. Annu Rev Neurosci 31: 359-387.
75. Packard MG, McGaugh JL, (1996) Inactivation of hippocampus or caudate nucleus with lidocaine differentially affects expression of place and response learning. Neurobiol Learn Mem 65: 65-72.
76. Schmajuk N, Voicu H, (2006) Exploration and navigation using hierarchical cognitive maps. Animal Spatial Cognition: Comparative, Neural, and Computational Approaches, M.F. Brown and R.G. Cook. p. Available:http://www.pigeon.psy.tufts.edu/asc/Schmajuk/Default.htm.
77. Bieszczad A, (1994) Neurosolver: a step toward a neuromorphic general problem solver. Proc World Congr Comput Intell volume 3: 1313-1318.
78. Frezza-Buet H, Alexandre F, (1999) Modeling prefrontal functions for robot navigation. Proc Int Jt Conf Neural Netw volume 1: 252-257.
79. Lieblich I, Arbib MA, (1982) Multiple representations of space underlying behavior. Behav Brain Sci 5: 627-640.
80. Schmajuk NA, Thieme AD, (1992) Purposive behavior and cognitive mapping: a neural network model. Biol Cybern 67: 165-174.
81. Franz MO, Schölkopf B, Mallot HA, Bülthoff HH, (1998) Learning view graphs for robot navigation. Auton Robots 5: 111-125.
82. Dehaene S, Changeux JP, (1997) A hierarchical neuronal network for planning behavior. Proc Natl Acad Sci U S A 94: 13293-13298.
83. Voicu H, (2003) Hierarchical cognitive maps. Neural Netw 16: 569-576.
84. Banquet JP, Gaussier P, Quoy M, Revel A, Burnod Y, (2005) A hierarchy of associations in hippocampo-cortical systems: cognitive maps and navigation strategies. Neural Comput 17: 1339-1384.
85. Fleuret F, Brunet E, (2000) DEA: an architecture for goal planning and classification. Neural Comput 12: 1987-2008.
86. Koene RA, Hasselmo ME, (2005) An integrate-and-fire model of prefrontal cortex neuronal activity during performance of goal-directed decision making. Cereb Cortex 15: 1964-1981.
87. McNamara TP, Hardy JK, Hirtle SC, (1989) Subjective hierarchies in spatial memory. J Exp Psychol Learn Mem Cogn 15: 211-227.
88. Roberts WA, (1979) Spatial memory in the rat on a hierarchical maze. Learn Motiv 10: 117-140.
89. Dallal NL, Meck WH, (1990) Hierarchical structures: chunking by food type facilitates spatial memory. J Exp Psychol Anim Behav Process 16: 69-84.
90. Fountain SB, Rowan JD, (1995) Coding of hierarchical versus linear pattern structure in rats and humans. J Exp Psychol Anim Behav Process 21: 187-202.
91. Macuda T, Roberts WA, (1995) Further evidence for hierarchical chunking in rat spatial memory. J Exp Psychol Anim Behav Process 21: 20-32.
92. Meck WH, Williams CL, (1997) Perinatal choline supplementation increases the threshold for chunking in spatial memory. Neuroreport 8: 3053-3059.
93. Kjelstrup KB, Solstad T, Brun VH, Hafting T, Leutgeb S, et al. (2008) Finite scale of spatial representation in the hippocampus. Science 321: 140-143.
94. Botvinick MM, Niv Y, Barto AC, (2009) Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective. Cognition 113: 262-280.
95. Sutton RS, Precup D, Singh S, (1999) Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning. Artif Intell 112: 181-211.
96. Koechlin E, Ody C, Kouneiher F, (2003) The architecture of cognitive control in the human prefrontal cortex. Science 302: 1181-1185.
97. Dragoi G, Buzsáki G, (2006) Temporal encoding of place sequences by hippocampal cell assemblies. Neuron 50: 145-157.
98. Rondi-Reig L, Petit GH, Tobin C, Tonegawa S, Mariani J, et al. (2006) Impaired sequential egocentric and allocentric memories in forebrain-specific-NMDA receptor knock-out mice during a new task dissociating strategies of navigation. J Neurosci 26: 4071-4081.
99. Doeller CF, Opitz B, Krick CM, Mecklinger A, Reith W, (2005) Prefrontal-hippocampal dynamics involved in learning regularities across episodes. Cereb Cortex 15: 1123-1133.
100. Doeller CF, Opitz B, Krick CM, Mecklinger A, Reith W, (2006) Differential hippocampal and prefrontal-striatal contributions to instance-based and rule-based learning. Neuroimage 31: 1802-1816.
101. Winocur G, Moscovitch M, Sekeres M, (2007) Memory consolidation or transformation: context manipulation and hippocampal representations of memory. Nat Neurosci 10: 555-557.
102. Goto Y, Grace AA, (2008) Dopamine modulation of hippocampal-prefrontal cortical interaction drives memory-guided behavior. Cereb Cortex 18: 1407-1414.
103. Schacter DL, Addis DR, Buckner RL, (2008) Episodic simulation of future events: concepts, data, and applications. Ann N Y Acad Sci 1124: 39-60.
104. Mushiake H, Saito N, Sakamoto K, Itoyama Y, Tanji J, (2006) Activity in the lateral prefrontal cortex reflects multiple steps of future events in action plans. Neuron 50: 631-641.
105. Mehta M, (2000) Experience-Dependent asymmetric shape of hippocampal receptive fields. Neuron 25: 707-715.
106. Ferbinteanu J, Shapiro ML, (2003) Prospective and retrospective memory coding in the hippocampus. Neuron 40: 1227-1239.
107. Becker S, Lim J, (2003) A computational model of prefrontal control in free recall: strategic memory use in the california verbal learning task. J Cogn Neurosci 15: 821-832.
108. Byrne P, Becker S, Burgess N, (2007) Remembering the past and imagining the future: a neural model of spatial memory and imagery. Psychol Rev 114: 340-375.
109. Tolman EC, (1948) Cognitive maps in rats and men. Psychol Rev 55: 189-208.
110. Rivard B, Li Y, Lenck-Santini P, Poucet B, Muller RU, (2004) Representation of objects in space by two classes of hippocampal pyramidal cells. J Gen Physiol 124: 9-25.
111. Alvernhe A, Cauter TV, Save E, Poucet B, (2008) Different CA1 and CA3 representations of novel routes in a shortcut situation. J Neurosci 28: 7324-7333.
112. Johnson A, Redish AD, (2007) Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J Neurosci 27: 12176-12189.
113. O'Keefe J, Recce ML, (1993) Phase relationship between hippocampal place units and the EEG theta rhythm. Hippocampus 3: 317-330.
114. Jones MW, Wilson MA, (2005) Theta rhythms coordinate hippocampal-prefrontal interactions in a spatial memory task. PLoS Biol 3: e402.
115. Rudebeck PH, Walton ME, Smyth AN, Bannerman DM, Rushworth MFS, (2006) Separate neural pathways process different decision costs. Nat Neurosci 9: 1161-1168.
116. Burgess N, Recce M, O'Keefe J, (1994) A model of hippocampal function. Neural Netw 7: 1065-1081.
117. Foster DJ, Morris RG, Dayan P, (2000) A model of hippocampally dependent navigation, using the temporal difference learning rule. Hippocampus 10: 1-16.
118. Girardeau G, Benchenane K, Wiener SI, Buzsaki G, Zugaro MB, (2009) Selective suppression of hippocampal ripples impairs spatial memory. Nat Neurosci 12: 1222-1223.