Banishing the homunculus: Making working memory work

Neuroscience

Volumn 139, Issue 1, 2006, Pages 105-118

  Hazy, T E ^a   Frank, M J ^a   O'Reilly, R C ^a  

^a UNIVERSITY OF COLORADO   (United States)

Author keywords

basal ganglia; computational modeling; dopamine; Pavlovian conditioning; prefrontal cortex; reinforcement learning

Indexed keywords

DOPAMINE;

AMYGDALOID NUCLEUS; ARTICLE; BASAL GANGLION; COGNITION; DOPAMINE RELEASE; DOPAMINERGIC SYSTEM; ELECTROPHYSIOLOGY; HIPPOCAMPUS; INTELLIGENCE QUOTIENT; LEARNING; MESENCEPHALON; PREFRONTAL CORTEX; PRIORITY JOURNAL; REINFORCEMENT; THALAMUS; THEORETICAL MODEL; WORKING MEMORY;

AMYGDALA; BASAL GANGLIA; DOPAMINE; HUMANS; LEARNING; MEMORY, SHORT-TERM; MODELS, NEUROLOGICAL; NEURAL PATHWAYS; PREFRONTAL CORTEX; REINFORCEMENT (PSYCHOLOGY);

EID: 33645889031     PISSN: 03064522     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neuroscience.2005.04.067     Document Type: Article

References (103)

1. Alexander G.E., DeLong M.R., and Strick P.L. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Ann Rev Neurosci 9 (1986) 357-381
2. Amos A. A computational model of information processing in the frontal cortex and basal ganglia. J Cogn Neurosci 12 (2000) 505-519
3. Anderson J.R. The architecture of cognition (1983), Harvard University Press, Cambridge, MA
4. Atallah H.E., Frank M.J., and O'Reilly R.C. Hippocampus cortex and basal ganglia. insights from computational models of complementary learning systems. Neurobiol Learn Mem 82 3 (2004) 253-267
5. Aubert I., Ghorayeb I., Normand E., and Bloch B. Phenotypical characterization of the neurons expressing the D1 and D2 dopamine receptors in the monkey striatum. J Comp Neurol 418 (2000) 22-32
6. Baddeley A.D. Working memory (1986), Oxford University Press, New York
7. Beiser D.G., and Houk J.C. Model of cortical-basal ganglionic processing. encoding the serial order of sensory events. J Neurophysiol 79 (1998) 3168-3188
8. Berns G.S., and Sejnowski T.J. How the basal ganglia make decisions. In: Damasio A., Damasio H., and Christen Y. (Eds). Neurobiology of decision-making (1995), Springer-Verlag, Berlin 101-113
9. Berns G.S., and Sejnowski T.J. A computational model of how the basal ganglia produces sequences. J Cogn Neurosci 10 (1998) 108-121
10. Bogacz R., and Cohen J.D. Parameterization of connectionist models. Behav Res Methods Instr Comput 36 4 (2004) 732-741
11. Braver T.S., Barch D.M., and Cohen J.D. Cognition and control in schizophrenia. A computational model of dopamine and prefrontal function. Biol Psychiatry 46 (1999) 312-328
12. Braver T.S., and Bongiolatti S.R. The role of frontopolar cortex in subgoal processing during working memory. Neuroimage 15 (2002) 523-536
13. Braver T.S., and Cohen J.D. On the control of control. The role of dopamine in regulating prefrontal function and working memory. In: Monsell S., and Driver J. (Eds). In Control of cognitive processes. Attention and performance XVIII (2000), MIT Press, Cambridge, MA 713-737
14. Braver T.S., Cohen J.D., Nystrom L.E., Jonides J., Smith E.E., and Noll D.C. A parametric study of frontal cortex involvement in human working memory. Neuroimage 5 (1997) 49-62
15. Brown J., Bullock D., and Grossberg S. How the basal ganglia use parallel excitatory and inhibitory learning pathways to selectively respond to unexpected rewarding cues. J Neurosci 19 (1999) 10502-10511
16. Burgess N., and Hitch G.J. Memory for serial order. A network model of the phonological loop and its timing. Psychol Rev 106 (1999) 551-581
17. Centonze D., Picconi B., Gubellini P., Bernardi G., and Calabresi P. Dopaminergic control of synaptic plasticity in the dorsal striatum. Eur J Neurosci 13 (2001) 1071-1077
18. Cohen J.D., Braver T.S., and O'Reilly R.C. A computational approach to prefrontal cortex, cognitive control, and schizophrenia. Recent developments and current challenges. Philos Trans R Soc (Lond) B 351 (1996) 1515-1527
19. Cohen J.D., Dunbar K., and McClelland J.L. On the control of automatic processes. A parallel distributed processing model of the Stroop effect. Psychol Rev 97 3 (1990) 332-361
20. Cohen J.D., Perlstein W.M., Braver T.S., Nystrom L.E., Noll D.C., Jonides J., and Smith E.E. Temporal dynamics of brain activation during a working memory task. Nature 386 (1997) 604-608
21. Cohen J.D., and Servan-Schreiber D. Context, cortex, and dopamine. A connectionist approach to behavior and biology in schizophrenia. Psychol Rev 99 (1992) 45-77
22. Cohen J.D., Servan-Schreiber D., and McClelland J.L. A parallel distributed processing approach to automaticity. Am J Psychol 105 (1992) 239-269
23. Contreras-Vidal J.L., and Schultz W. A predictive reinforcement model of dopamine neurons for learning approach behavior. J Comp Neurosci 6 (1999) 191-214
24. Dominey P., Arbib M., and Joseph J.-P. A model of corticostriatal plasticity for learning oculomotor associations and sequences. J Cogn Neurosci 7 (1995) 311-336
25. Dominey P.F., and Georgieff N. Schizophrenics learn surface but not abstract structure in a serial reaction time task. Neuroreport 8 (1997) 2877
26. Durstewitz D., Kelc M., and Gunturkun O. A neurocomputational theory of the dopaminergic modulation of working memory functions. J Neurosci 19 (1999) 2807
27. Durstewitz D., Seamans J.K., and Sejnowski T.J. Dopamine-mediated stabilization of delay-period activity in a network model of prefrontal cortex. J Neurophysiol 83 (2000) 1733-1750
28. Eriksen B.A., and Eriksen C.W. Effects of noise letters upon the identification of a target letter in a nonsearch task. Percept Psychophys 16 (1974) 143-149
29. Fellous J.M., Wang X.J., and Lisman J.E. A role for NMDA-receptor channels in working memory. Nat Neurosci 1 (1998) 273-275
30. Frank M.J. Dynamic dopamine modulation in the basal ganglia. A neurocomputational account of cognitive deficits in medicated and non-medicated Parkinsonism. J Cogn Neurosci 17 (2005) 51-72
31. Frank M.J., Loughry B., and O'Reilly R.C. Interactions between the frontal cortex and basal ganglia in working memory. A computational model. Cogn Affect Behav Neurosci 1 (2001) 137-160
32. Frank M.J., Rudy J.W., and O'Reilly R.C. Transitivity, flexibility, conjunctive representations and the hippocampus: II. a computational analysis. Hippocampus 13 (2003) 341-354
33. Frank M.J., Seeberger L., and O'Reilly R.C. By carrot or by stick. Cognitive reinforcement learning in Parkinsonism. Science 306 (2004) 1940-1943
34. Freedman D.J., Riesenhuber M., Poggio T., and Miller E.K. Visual categorization and the primate prefrontal cortex. Neurophysiology and behavior. J Neurophysiol 88 (2002) 929-941
35. Funahashi S., Bruce C.J., and Goldman-Rakic P.S. Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex. J Neurophysiol 61 (1989) 331-349
36. Fuster J.M. The prefrontal cortex. Anatomy, physiology and neuropsychology of the frontal lobe (1989), Raven Press, New York
37. Fuster J.M., and Alexander G.E. Neuron activity related to short-term memory. Science 173 (1971) 652-654
38. Gerfen C. The neostriatal mosaic. multiple levels of compartmental organization in the basal ganglia. Ann Rev Neurosci 15 (1992) 285-320
39. Graybiel A.M., and Kimura M. Adaptive neural networks in the basal ganglia. In: Houk J.C., Davis J.L., and Beiser D.G. (Eds). Models of information processing in the basal ganglia (1995), MIT Press, Cambridge, MA 103-116
40. 2+ conductance. J Neurosci 17 (1997) 3334-3342
41. 3-calcineurin-signaling cascade. J Neurosci 20 (2000) 8987-8995
42. Houk J.C., Adams J.L., and Barto A.G. A model of how the basal ganglia generate and use neural signals that predict reinforcement. In: Houk J.C., Davis J.L., and Beiser D.G. (Eds). Models of information processing in the basal ganglia (1995), MIT Press, Cambridge, MA 233-248
43. Houk J.C., and Wise S.P. Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex. their role in planning and controlling action. Cereb Cortex 5 (1995) 95-110
44. Ivry R. The representation of temporal information in perception and motor control. Curr Opin Neurobiol 6 (1996) 851-857
45. Jackson S., and Houghton G. Sensorimotor selection and the basal ganglia. A neural network model. In: Houk J.C., Davis J.L., and Beiser D.G. (Eds). Models of information processing in the basal ganglia (1995), MIT Press, Cambridge, MA 337-368
46. Joel D., Niv Y., and Ruppin E. Actor-critic models of the basal ganglia. new anatomical and computational perspectives. Neural Netw 15 (2002) 535-547
47. Kimberg D.Y., D'Esposito M., and Farah M.J. Effects of bromocriptine on human subjects depend on working memory capacity. Neuroreport 8 (1997) 3581-3585
48. Koechlin E., Corrado G., and Grafman J. Dissociating the role of the medial and lateral anterior prefrontal cortex in human planning. Proc Natl Acad Sci U S A 97 (2000) 7651
49. Koechlin E., Ody C., and Kouneiher F. Neuroscience. The architecture of cognitive control in the human prefrontal cortex. Science 424 (2003) 1181-1184
50. Kropotov J.D., and Etlinger S.C. Selection of actions in the basal ganglia-thalamocoritcal circuits. Review and model. Int J Psychophysiol 31 (1999) 197-217
51. Kubota K., and Niki H. Prefrontal cortical unit activity and delayed alternation performance in monkeys. J Neurophysiol 34 (1971) 337-347
52. Levitt J.B., Lewis D.A., Yoshioka T., and Lund J.S. Topography of pyramidal neuron intrinsic connections in macaque monkey prefrontal cortex (areas 9 46). J Comp Neurol 338 (1993) 360-376
53. Lovett M.C., Reder L.M., and Lebiere C. Modeling working memory in a unified architecture. In: Miyake A., and Shah P. (Eds). Models of working memory. Mechanisms of active maintenance and executive control (1999), Cambridge University Press, New York 135-182
54. Marr D. Simple memory. A theory for archicortex. Philos Trans R Soc (Lond) B 262 (1971) 23-81
55. Mauk M.D., and Buonomano D.V. The neural basis of temporal processing. Ann Rev Neurosci 27 (2004) 307-340
56. McClelland J.L., McNaughton B.L., and O'Reilly R.C. Why there are complementary learning systems in the hippocampus and neocortex. Insights from the successes and failures of connectionist models of learning and memory. Psychol Rev 102 (1995) 419-457
57. McNaughton B.L., and Morris R.G.M. Hippocampal synaptic enhancement and information storage within a distributed memory system. Trends Neurosci 10 10 (1987) 408-415
58. Middleton F.A., and Strick P.L. Basal ganglia output and cognition. Evidence from anatomical, behavioral, and clinical studies. Brain Cogn 42 (2000) 183-200
59. Miller E.K., and Cohen J.D. An integrative theory of prefrontal cortex function. Ann Rev Neurosci 24 (2001) 167-202
60. Miller E.K., Erickson C.A., and Desimone R. Neural mechanisms of visual working memory in prefontal cortex of the macaque. J Neurosci 16 (1996) 5154
61. Mink J.W. The basal ganglia. Focused selection and inhibition of competing motor programs. Prog Neurobiol 50 (1996) 381-425
62. Miyashita Y., and Chang H.S. Neuronal correlate of pictorial short-term memory in the primate temporal cortex. Nature 331 (1988) 68-70
63. Montague P.R., Dayan P., and Sejnowski T.J. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16 (1996) 1936-1947
64. Munakata Y., and O'Reilly R.C. Developmental and computational neuroscience approaches to cognition. The case of generalization. Cogn Studies 10 (2003) 76-92
65. Nakahara H., and Doya K. Near saddle-node bifurcation behavior as dynamics in working memory for goal-directed behavior. Neural Comput 10 (1998) 113-132
66. Nakahara H., Doya K., and Hikosaka O. Parallel cortico-basal ganglia mechanisms for acquisition and execution of visuomotor sequences-a computational approach. J Cogn Neurosci 13 (2001) 626-647
67. Newell A. Unified theories of cognition (1990), Harvard University Press, Cambridge, MA
68. Nieder A., Freedman D.J., and Miller E.K. Representation of the quantity of visual items in the primate prefrontal cortex. Science 298 (2002) 1708-1711
69. Nishi A., Snyder G., and Greengard P. Bidirectional regulation of DARPP-32 phosphorylation by dopamine. J Neurosci 17 (1997) 8147-8155
70. Norman K.A., and O'Reilly R.C. Modeling hippocampal and neocortical contributions to recognition memory. A complementary learning systems approach. Psychol Rev 110 (2003) 611-646
71. O'Reilly R.C. Biologically plausible error-driven learning using local activation differences. The generalized recirculation algorithm. Neural Comput 8 5 (1996) 895-938
72. O'Reilly R.C. Six principles for biologically-based computational models of cortical cognition. Trends Cogn Sci 2 11 (1998) 455-462
73. O'Reilly R.C. Generalization in interactive networks. The benefits of inhibitory competition and Hebbian learning. Neural Comput 13 (2001) 1199-1242
74. O'Reilly R.C., Braver T.S., and Cohen J.D. A biologically based computational model of working memory. In: Miyake A., and Shah P. (Eds). Models of working memory. Mechanisms of active maintenance and executive control (1999), Cambridge University Press, New York 375-411
75. O'Reilly RC, Frank MJ (in press). Making working memory work: A computational model of learning in the frontal cortex and basal ganglia. Neural Computation, in press.
76. O'Reilly R.C., and McClelland J.L. Hippocampal conjunctive encoding, storage, and recall. Avoiding a tradeoff. Hippocampus 4 6 (1994) 661-682
77. O'Reilly R.C., and Munakata Y. Computational explorations in cognitive neuroscience. Understanding the mind by simulating the brain (2000), MIT Press, Cambridge, MA
78. O'Reilly R.C., Noelle D., Braver T.S., and Cohen J.D. Prefrontal cortex and dynamic categorization tasks. Representational organization and neuromodulatory control. Cereb Cortex 12 (2002) 246-257
79. O'Reilly R.C., and Norman K.A. Hippocampal and neocortical contributions to memory. Advances in the complementary learning systems framework. Trends Cogn Sci 6 (2002) 505-510
80. O'Reilly R.C., and Rudy J.W. Conjunctive representations in learning and memory. Principles of cortical and hippocampal function. Psychol Rev 108 (2001) 311-345
81. Pucak M.L., Levitt J.B., Lund J.S., and Lewis D.A. Patterns of intrinsic and associational circuitry in monkey prefrontal cortex. J Comp Neurol 376 (1996) 614-630
82. Rescorla R.A., and Wagner A.R. A theory of Pavlovian conditioning. Variation in the effectiveness of reinforcement and non-reinforcement. In: Black A.H., and Prokasy W.F. (Eds). Classical conditioning II. Theory and research (1972), Appleton-Century-Crofts, New York 64-99
83. Rolls E.T. Functions of neuronal networks in the hippocampus and neocortex in memory. In: Byrne J.H., and Berry W.O. (Eds). Neural models of plasticity. Experimental and theoretical approaches (1989), Academic Press, San Diego, CA 240-265
84. Rougier N.P., Noelle D., Braver T.S., Cohen J.D., and O'Reilly R.C. Prefrontal cortex and the flexibility of cognitive control. Rules without symbols. Proc Natl Acad Sci U S A 102 20 (2005) 7338-7343
85. Rougier N.P., and O'Reilly R.C. Learning representations in a gated prefrontal cortex model of dynamic task switching. Cogn Sci 26 (2002) 503-520
86. PDP Research Group, Rumelhart D.E., and McClelland J.L. Parallel distributed processing. Volume 1: Foundations (1986), MIT Press, Cambridge, MA
87. Schultz W. Predictive reward signal of dopamine neurons. J Neurophysiol 80 (1998) 1
88. Schultz W., Apicella P., and Ljungberg T. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci 13 (1993) 900-913
89. Schultz W., Romo R., Ljungberg T., Mirenowicz J., Hollerman J.R., and Dickinson A. Reward-related signals carried by dopamine neurons. In: Houk J.C., Davis J.L., and Beiser D.G. (Eds). Models of information processing in the basal ganglia (1995), MIT Press, Cambridge, MA 233-248
90. Seamans J., and Yang C. The principal features and mechanisms of dopamine modulation in the prefrontal cortex. Prog Neurobiol 74 (2004) 1-57
91. Sternberg S. High speed scanning in human memory. Science 153 (1966) 652-654
92. Suri R.E., Bargas J., and Arbib M.A. Modeling functions of striatal dopamine modulation in learning and planning. Neuroscience 103 (2001) 65-85
93. Sutton R.S. Learning to predict by the method of temporal diferences. Machine Learn 3 (1988) 9-44
94. Sutton R.S., and Barto A.G. Reinforcement learning. An introduction (1998), MIT Press, Cambridge, MA
95. Tanaka S. Dopamine controls fundamental cognitive operations of multi-target spatial working memory. Neural Netw 15 (2002) 573-582
96. Wallis J.D., Anderson K.C., and Miller E.K. Single neurons in prefrontal cortex encode abstract rules. Nature 411 (2001) 953-956
97. Wang X.-J. Synaptic basis of cortical persistent activity. The importance of NMDA receptors to working memory. J Neurosci 19 (1999) 9587
98. Wickens J. A theory of the striatum (1993), Pergamon Press, Oxford
99. Wickens J.R., Kotter R., and Alexander M.E. Effects of local connectivity on striatal function. simulation and analysis of a model. Synapse 20 (1995) 281-298
100. Widrow B, Hoff ME (1960) Adaptive switching circuits. Institute of Radio Engineers, Western Electronic Show and Convention, Convention Record, Part 4 (pp 96-104).
101. Wise S.P., Murray E.A., and Gerfen C.R. The frontal cortex-basal ganglia system in primates. Crit Rev Neurobiol 10 (1996) 317-356
102. Wu Q., Reith M., Walker Q., Kuhn C., Caroll F., and Garris P. Concurrent autoreceptor-mediated control of dopamine release and uptake during neurotransmission. an in vivo voltammetric study. J Neurosci 22 (2002) 6272-6281
103. Yeung N., Botvinick M.M., and Cohen J.D. The neural basis of error detection. Conflict monitoring and the error-related negativity. Psychol Rev 111 4 (2004) 931-959