An integrate-and-fire model of prefrontal cortex neuronal activity during performance of goal-directed decision making

Cerebral Cortex

Volumn 15, Issue 12, 2005, Pages 1964-1981

  Koene, Randal A ^a   Hasselmo, Michael E ^a  

^a BOSTON UNIVERSITY   (United States)

Author keywords

Learning; Minicolumns; Orbitofrontal; Reinforcement; Selective activity

Indexed keywords

ARTICLE; BRAIN FUNCTION; DECISION MAKING; LEARNING; MOTIVATION; MOTOR PERFORMANCE; NERVE CELL EXCITABILITY; NERVE CELL NETWORK; NERVE CELL PLASTICITY; NEUROBIOLOGY; PREFRONTAL CORTEX; PRIORITY JOURNAL; REINFORCEMENT; SENSORY CORTEX; TASK PERFORMANCE;

ACTION POTENTIALS; ANIMALS; DECISION MAKING; DISCRIMINATION (PSYCHOLOGY); GOALS; HUMANS; LEARNING; MODELS, NEUROLOGICAL; NEURONS; PREFRONTAL CORTEX;

EID: 24944467165     PISSN: 10473211     EISSN: 14602199     Source Type: Journal    
DOI: 10.1093/cercor/bhi072     Document Type: Article

References (99)

1. Alonso A, Gaztelu J, Bruno W Jr, Garcia-Austt E (1987) Cross-correlation analysis of septohippocampal neurons during theta-rhythm. Brain Res 413:135-146.
2. Andrade R (1991) The effect of carbachol which affects muscarinic receptors was investigated in prefrontal layer v neurons. Brain Res 541:81-93.
3. Arana F, Parkinson JEH, Holland A, Owen A, Roberts A (2003) Dissociable contributions of the human amygdala and orbitofrontal cortex to incentive motivation and goal selection. J Neurosci 23:9632-9638.
4. Artola A, Brocher S, Singer W (1990) Different voltage-dependent thresholds for inducing long-term depression and long-term potentiation in slices of rat visual cortex. Nature 347:69-72.
5. Balleine B, Dickinson A (1998) Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 37:407-419.
6. Barbas H (2000) Connections underlying the synthesis of cognition, memory, and emotion in primate prefrontal cortices. Brain Res Bull 52:319-330.
7. Barbas H, Pandya D (1989) Architecture and intrinsic connections of the prefrontal cortex in the rhesus monkey. J Comp Neurol 286(3):353-375.
8. Barto A (1995a) Adaptive critics and the basal ganglia. In: Models of information processing in the basal ganglia (Houk J, Davis JL, Beiser, DG, eds), pp. 215-232. Cambridge, MA: MIT Press.
9. Barto A (1995b) Reinforcement learning. In: Handbook of brain theory and neural networks (Arbib M, ed.), pp. 804-809. Cambridge, MA: MIT Press.
10. Bartus R, Johnson H (1976) Short-term memory in rhesus monkey: disruption from the anti-cholinergic scopolamine. Pharmacol Biochem Behav 5:39-46.
11. Bechara A, Damasio A, Damasio H, Anderson S (1994) Insensitivity to future consequences following damage to human prefrontal cortex. Cognition 50:7-15.
12. Bechara A, Damasio H, Tranel D, Damasio A (1997) Deciding advantageously before knowing the advantageous strategy. Science 275:1293-1295.
13. Bertsekas D (1995) Dynamic programming and optimal control. Belmont, MA: Athena.
14. Bi G, Poo M (1998) Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J Neurosci 18:10464-10472.
15. Bliss T, Collingridge G (1993) A synaptic model of memory: long-term potentiation in the hippocampus. Nature 361:31-39.
16. Bliss T, Lømo T (1973) Long-lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path. J Physiol 232:331-356.
17. Bragin A, Jando G, Nadasdy Z, Hetke J (1995) Gamma (40-100 Hz) oscillation in the hippocampus of the behaving rat. J Neurosci 15:47-60.
18. Brazhnik E, Fox S (1999) Action potentials and relations to theta rhythm of septohippocampal neurons in vivo. Exp Brain Res 127:244-258.
19. Cannon R, Hasselmo M, Koene R (2003) From biophysics to behaviour: Catacomb2 and the design of biologically plausable models for spatial navigation. Neuroinformatics 1: 1:3-42.
20. Cohen J, Servan-Schreiber D (1992) Context, cortex and dopamine: a connectionist approach to behavior and biology in schizophrenia. Psychol Rev 99:45-77.
21. Constantinidis C, Goldman-Rakic P (2002) Correlated discharges among putative pyramidal neurons and interneurons in the primate prefrontal cortex. J Neurophysiol 88:3487-3497.
22. Dayan P, Hinton G (1996) Varieties of helmholtz machine. Neural Networks 9:1385-1403.
23. Douglas R, Martin K, Whitteridge D (1989) A canonical microcircuit for neocortex. Neural Comput 1:480-488.
24. Elman J (1990) Finding structure in time. Cogn Sci 14:179-211.
25. Elman J (1991) Distributed representations, simple recurrent networks, and grammatical structure. Machine Learn 7:195-224.
26. Felleman D, Van Essen D (1991) Distributed hierarchical processing in the primate cerebral cortex. Cereb Cortex 1:1-47.
27. Fransen E, Alonso A, Hasselmo M (2002) Simulations of the role of the muscarinic activated calcium-sensitive nonspecific cation current iNCM in entorhinal neuronal activity during delayed matching tasks. J Neurosci 22:1081-1097.
28. Frey S, Petrides M (1997) Orbitofrontal cortex: a key prefrontal regions for encoding information. Proc Natl Acad Sci USA 15:8723-8727.
29. Funahashi S, Bruce C, Goldman-Rakic P (1989) Mnemonic coding of visual space by neurons in the monkey's dorsolateral prefrontal cortex revealed by an oculomotor delayed response task. J Neurophysiol 61:331-349.
30. Fuster J (1973) Unit activity in prefrontal cortex during delayed-response performance: neuronal correlates of transient memory. J Neurophysiol 36:61-78.
31. Fuster J (2000) Prefrontal neurons in networks of executive memory. Brain Res Bull 52:331-336.
32. Fuster J, Bauer R, Jervey J (1982) Cellular discharge in the dorsolateral prefrontal cortex of the monkey in cognitive tasks. Ex Neurol 77:679-694.
33. Gerstner W (2002) Integrate-and-fire neurons and networks. Cambridge, MA: MIT Press.
34. Gerstner W, Kistler W (2002) Spiking neuron models: single neurons, populations, plasticity. Cambridge, UK: Cambridge University Press.
35. Grossberg S, Williamson J (2001) A neural model of how horizontal and interlaminar connections of visual cortex develop into adult circuits that carry out perceptual grouping and learning. Cereb Cortex 11:37-58.
36. Gutkin B, Ermentrout G, O'Sullivan J (2000) Layer 3 patchy recurrent excitatory connections may determine the spatial organization of sustained activity in the primate prefrontal cortex. Neurocomputing 32-33:391-400.
37. Hancock P, Smith L, Phillips W (1991) A biologically supported errorcorrecting learning rule. Neural Comput 3:201-212.
38. Harel D (1987) Statecharts: a visual formalism for complex systems. Sci Comput Prog 8:231-274.
39. Hasselmo M (2005) A model of prefrontal cortical mechanisms for goal directed behavior. J Cogn Neurosci (in press).
40. Hasselmo M, Bodelon C, Wyble B (2002) A proposed function for hippocampal theta rhythm: separate phases of encoding and retrieval enhance reversal of prior learning. Neural Comput 14:793-817.
41. Izquierdo A, Murray E (2004) Combined unilateral lesions of the amygdala and orbital prefrontal cortex impair affective processing in rhesus monkeys. J Neurophysiol 91:2023-2039.
42. Jensen O, Lisman J (1996) Novel lists of 7±2 known items can be reliably stored in an oscillatory short-term memory network: Interaction with longterm memory. Learn Mem 3:257-263.
43. Jensen O, Idiart M, Lisman J (1996) Physiologically realistic formation of autoassociative memory in networks with theta/gamma oscillations: role of fast NMDA channels. Learn Mem 3:243-256.
44. Jung M, Qin Y, McNaughton B, Barnes C (1998) Firing characteristics of deep layer neurons in prefrontal cortex in rats performing spatial working memory tasks. Cereb Cortex 8:437-450.
45. Kawato M, Hayakama H, Inui T (1993) A forward-inverse optics model of reciprocal connections between visual cortical areas. Network 4:415-422.
46. Klink R, Alonso A (1997a) Morphological characteristics of layer ii projection neurons in the rat medial entorhinal cortex. Hippocampus 7:571-583.
47. Klink R, Alonso A (1997b) Muscarinic modulation of the oscillatory and repetitive firing properties of entorhinal cortex layer ii neurons. J Neurophysiol 77:1813-1828.
48. Koene R, Gorchetchnikov A, Cannon R, Hasselmo M (2003) Modeling goaldirected spatial navigation in the rat based on physiological data from the hippocampal formation. Neural Networks 16:577-584.
49. Konorski J (1948) Conditioned reflexes and neuron organization. Cambridge: Cambridge University Press.
50. Levy W, Steward D (1983) Temporal contiguity requirements for longterm associative potentiation/depression in the hippocampus. Neuroscience 8:791-797.
51. Lisman J, Idiart M (1995) Storage of 7±2 short-term memories in oscillatory subcylces. Science 267:1512-1515.
52. Lübke J, von der Malsburg C (2004) Rapid processing and unsupervised learning in a model of the cortical macrocolumn. Neural Comput 16:501-533.
53. Lund J, Yoshioka T, Levitt J (1993) Comparison of intrinsic connectivity in different areas of macaque monkey cerebral cortex. Cereb Cortex 3:148-162.
54. Manns I, Alonso A, Jones B (2000) Discharge properties of juxtacellularly labeled and immunohistochemically identified cholinergic basal forebrain neurons recorded in association with the electroencephalogram in anesthetized rats. J Neurosci 20:1505-1518.
55. Markram H, Lübke J, Frotscher M, Sakmann B (1997) Regulation of synaptic efficacy by coincidence of postsynaptic aps and epsps. Science 225:213-215.
56. Maunsell J, Van Essen D (1983) The connections of the middle temporal visual area (mt) and their relationship to a cortical hierarchy in the macaque monkey. J Neurosci 3:2563-2586.
57. McGaughy J, Koene R, Eichenbaum H, Hasselmo M (2004) Effects of cholinergic deafferentation of prefrontal cortex on working memory: a convergence of behavioral and modeling results. In: Program No. 551.7, 2004 Abstract Viewer/Itinerary Planner. Washington, DC: Society for Neuroscience.
58. Miller E, Cohen J (2001) An integrative theory of prefrontal cortex function. Annu Rev Neurosci 24:167-202.
59. Montague P, Sejnowski T (1994) The predictive brain: temporal coincidence and temporal order in synaptic learning mechanisms. Learn Mem 1:1-33.
60. Montague P, Dayan P, Nowlan S, Pouget A, Sejnowski T (1993) Using aperiodic reinforcement for directed self-organization. In: Advances in neural information processing systems (Giles C, Hanson S, Cowan J, eds), vol. 5, pp. 969-977. San Mateo, CA: Morgan Kaufmann.
61. Montague P, Dayan P, Sejnowski T (1996) A framework for mesencephalic dopamine systems based on predictive hebbian learning. J Neurosci 16:1936-1947.
62. Mountcastle V (1997) The columnar organization of the neocortex. Brain 120:701-722.
63. Mulder A, Nordquist R, Örgüt O, Pennartz C (2003) Learning-related changes in response patterns of prefrontal neurons during instrumental conditioning. Behav Brain Res 146:77-88.
64. Mumford D (1991) On the computational architecture of the neocortex. I. The role of the thalamo-cortical loop. Biol Cybernet 65:135-145.
65. Mumford D (1992) On the computational architecture of the neocortex. II. The role of cortico-cortical loops. Biol Cybernet 66:241-251.
66. Mumford D (1994) Neuronal architectures for pattern-theoretic problems. In: Large-scale neuronal theories of the brain (Koch C, Davis J, eds), pp. 125-152. Cambridge, MA: MIT Press.
67. O'Reilly R, Munakata Y (2000) Computational explorations in cognitive neuroscience: understanding the mind by simulating the brain. Cambridge, MA: MIT Press.
68. Pears A, Parkinson A, Hopewell L, Everitt B, Roberts A (2003) Lesions of the orbitofrontal but not medial prefrontal cortex disrupt conditioned reinforcement in primates. J Neurosci 23:11189-11201.
69. Penetar D, McDonough J Jr (1977) Effects of cholinergic drugs on delayed match-to-sample performance of rhesus monkeys. Pharmacol Biochem Behav 19:963-967.
70. Quintana J, Fuster J (1992) Mnemonic and predictive functions of cortical neurons in a memory task. Neuroreport 3:721-724.
71. Rao R, Sejnowski T (2001) Spike-timing-dependent Hebbian plasticity as temporal difference learning. Neural Comput 13:2221-2237.
72. Rescorla R, Wagner A (1972) A theory of pavlovian conditioning: the effectiveness of reinforcement and non-reinforcement. In: Classical conditioning. II. Current research and theory (Black A, Prokasy W, eds), pp. 64-69. New York: Appleton-Century-Crofts.
73. Rolls E (1999) The functions of the orbitofrontal cortex. Neurocase 5:301-312.
74. Ross S (1983) Introduction to stochastic dynamic programming. New York: Academic Press.
75. Schoenbaum G, Eichenbaum H (1995a) information coding in the rodent prefrontal cortex. i. Single-neuron activity in orbitofrontal cortex compared with that in pyriform cortex. J Neurophysiol 74:733-750.
76. Schoenbaum G, Eichenbaum H (1995b) Information coding in the rodent prefrontal cortex. ii. Ensemble activity in orbitofrontal cortex. J Neurophysiol 74:751-762.
77. Schoenbaum G, Chiba A, Gallagher M (1998) Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nat Neurosci 1:155-159.
78. Schoenbaum G, Chiba A, Gallagher M (2000) Rapid changes in functional connectivity in orbitofrontal cortex and basolateral amygdala during learning and reversal. J Neurosci 20:5179-5189.
79. Schoenbaum G. Setlow B, Ramus S (2003) A systems approach to orbitofrontal cortex function: recordings in rat orbitofrontal cortex reveal interactions with different learning systems. Behav Brain Res 146:19-29.
80. Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80:1-27.
81. Schultz W, Dickinson A (2000) Neuronal coding of prediction errors. Annu Rev Neurosci 23:473-500.
82. Schultz W, Dayan P, Montague P (1997) A neural substrate of prediction and reward. Science 275:1593-1598.
83. Schultz W, Tremblay L, Hollerman J (2000) Reward processing in primate orbitofrontal cortex and basal ganglia. Cereb Cortex 10:272-283.
84. Somers D, Nelson S, Sur M (1995) An emergent model of orientation selectivity in cat visual cortical simple cells. J Neurosci 15:5448-5465.
85. Stein R (1967) Some models of neuronal variability. Biophys J 7:37-68.
86. Stewart M, Fox S (1990) Do septal neurons pace the hippocampal theta rhythm? Neuron 13:163-168.
87. Sutton R (1988) Learning to predict by the methods of temporal difference. Machine Learn 3:9-44.
88. Sutton R (1996) Generalization in reinforcement learning: successful examples using sparse coarse coding. In: Advances in neural information processing systems 8, pp. 1038-1044. Cambridge, MA: MIT Press.
89. Sutton R, Barto A (1981) Toward a modern theory of adaptive networks: expectation and prediction. Psychol Rev 88:135-140.
90. Sutton R, Barto A (1998) Reinforcement learning: an introduction. Cambridge, MA: MIT Press.
91. Terrace H, Son L, Brannon E (2003) Serial expertise of rhesus macaques. Psychol Sci 14:66-73.
92. Thorpe S, Rolls E, Maddison S (1983) The orbitofrontal cortex: neuronal activity in the behaving monkey. Exp Brain Res 49:93-115.
93. Tremblay L, Schultz W (1999) Relative reward preference in primate orbitofrontal cortex. Nature 398:704-708.
94. Tremblay L, Schultz W (2000) Reward related neuronal activity during go-nogo task performance in primate orbitofrontal cortex. J Neurophysiol 83:1877-1885.
95. Wallis J, Anderson K, Miller E (2001) Single neurons in prefrontal cortex encode abstract rules. Nature 411:953-956.
96. Wallis J, Miller E (2003) Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task. Eur J Neurosci 18:2069-2081.
97. White D (1969) Dynamic programming. San Francisco, CA Holden-Day.
98. Wood J, Grafman J (2003) Human prefrontal cortex: processing and representational perspectives. Nat Rev Neurosci 4:139-147.
99. Yishai B, Baror R, Sompolinsky H (1995) Theory of orientation tuning in visual cortex. Proc Natl Acad Sci USA 92:3844-3848.