Learning to use working memory: A reinforcement learning gating model of rule acquisition in rats

Frontiers in Computational Neuroscience

Volumn , Issue OCTOBER 2012, 2012, Pages

  Lloyd, Kevin ^a   Becker, Nadine ^a   Jones, Matthew W ^a   Bogacz, Rafal ^a  

^a UNIVERSITY OF BRISTOL   (United Kingdom)

Author keywords

Gating models; Reinforcement learning; Working memory

Indexed keywords

ACTOR CRITIC; COMPLEX PROCESSES; INDIVIDUAL DIFFERENCES; MULTIPLE STRATEGY; RULE ACQUISITION; RULE LEARNING; SPATIAL LEARNING; WORKING MEMORIES;

ALGORITHMS; COGNITIVE SYSTEMS; RATS; REINFORCEMENT LEARNING;

BRAIN MODELS;

ANIMAL BEHAVIOR; ANIMAL EXPERIMENT; ARTICLE; CONTROLLED STUDY; DISCRIMINATION LEARNING; LEARNING ALGORITHM; LEARNING CURVE; MALE; MATHEMATICAL MODEL; MAZE TEST; NONHUMAN; RAT; REINFORCEMENT; SENSORY GATING; SHORT TERM MEMORY; TASK PERFORMANCE; WORKING MEMORY;

EID: 84867605841     PISSN: 16625188     EISSN: None     Source Type: Journal    
DOI: 10.3389/fncom.2012.00087     Document Type: Article

References (28)

1. Baddeley A. 2012. Working Memory: Theories, Models, and Controversies. Ann Rev Psychol 63: 1-29.
2. Baird AL, Futter JE, Muir JL, Aggleton JP. 2004. On the Transience of Egocentric Working Memory: Evidence From Testing the Contribution of Limbic Brain Regions. Behav Neurosci 118(4): 785-797.
3. Barto AG, Sutton RS, Anderson CW. 1983. Neuronlike adaptive elements that can solve difficult learning control problems. IEEE T Syst Man Cyb 13: 835-846.
4. Bogacz R, Larsen T. 2011. Integration of Reinforcement Learning and Optimal Decision-Making see Theories of the Basal Ganglia. Neural Comput 23(4): 817-851.
5. Bogacz R, Cohen JD. 2004. Parameterization of Connectionist Models. Behav Res Meth Ins C 36(4): 732-741.
6. Braver TS, Cohen JD. 1999. Dopamine, cognitive control, and schizophrenia: the gating model. Prog Brain Res 121: 327-49.
7. Braver TS, Cohen JD. 2000. On the Control of Control: The Role of Dopamine in Regulating Prefrontal Function and Working Memory. In Control of Cognitive Processes: Attention and Per-formance 7 (eds S Monsell and JS Driver), pp. 713-737. Cambridge, MA: MIT Press.
8. Busoniu L, Babuska R, De Schutter B. 2008. A Comprehensive Survey of Multiagent Reinforce-ment Learning. IEEE T Syst Man Cy C 38(2): 156-172
9. Dayan P. 2007. Bilinearity, rules, and prefrontal cortex. Front Comput Neurosci 1:1.
10. Dayan P. (2008). Simple substrates for complex cognition. Front Neurosci 2(2): 255-263.
11. Dufort RH, Guttman N, Kimble GA. 1954. One-trial discrimination reversal in the white rat. J Comp Physiol Psych 47(3): 248-249.
12. Ferbinteanu J, Shapiro ML. 2003. Prospective and retrospective memory coding in the hippocampus. Neuron 40(6): 1227-1239.
13. Futter JE, Aggleton JP. 2006. How rats perform spatial working memory tasks: Limitations in the use of egocentric and idiothetic working memory. Q J Exp Psychol-B 59(1): 77-99.
14. Harvey CD, Coen P, Tank DW 2012. Choice-specific sequences in parietal cortex during a virtual-navigation decision task. Nature 484: 62-68.
15. Jones MW, Wilson MA. 2005. Theta rhythms coordinate hippocampal-prefrontal interactions in a spatial memory task. PLoS biology 3(12): e402.
16. Lalonde R. 2002. The neurobiological basis of spontaneous alternation. Neurosci Biobehav R 26(1): 91-104.
17. Morris G, Nevet A, Arkadir D, Vaadia E, Bergman H. 2006. Midbrain dopamine neurons encode decisions for future action. Nat Neurosci 9(8): 1057-1063.
18. Nelder JA, Mead R. 1965. A simple method for function minimization. Computer J7(4): 308-313.
19. O'Reilly RC, Frank MJ. 2006. Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Comput 18(2): 283-328.
20. Restle F. 1957. Discrimination of cues in mazes: A resolution of the "place-vs-response" question. Psych Rev 64(4): 217-228.
21. Roesch MR, Calu DJ, Schoenbaum G. 2007. Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat Neurosci 10(12): 1615-1624.
22. Rougier NP, Noelle DC, Braver TS, Cohen JD, O'Reilly RC. 2005. Prefrontal cortex and flexible cognitive control: rules without symbols. P Natl Acad Sci USA 102(20): 7338-7343.
23. Rummery GA, Niranjan M. 1994. On-line Q-learning using connectionist systems. Technical Report, Department of Engineering, University of Cambridge.
24. Sutton RS. 1988. Learning to Predict by the Methods of Temporal Differences. Mach Learn 3(1): 9-44.
25. Sutton RS, Barto AG. 1998. Reinforcement learning: an introduction. MIT Press.
26. Todd MT, Niv Y, Cohen JD. 2009. Learning to Use Working Memory in Partially Observable Environments through Dopaminergic Reinforcement. In Advances in Neural Information Processing Systems 21 (eds D Koller, D Schuurmans, Y Bengio, L. Bottou), pp. 1689-1696.
27. Wang XJ. 1999. Synaptic Basis of Cortical Persistent Activity: the Importance of NMDA Receptors to Working Memory. J Neurosci 19(21): 9587-9603.
28. Zilli EA, Hasselmo ME. 2008. Modeling the role of working memory and episodic memory in behavioral tasks. Hippocampus 18(2): 193-209.