Neurally plausible reinforcement learning of working memory tasks

Advances in Neural Information Processing Systems

Volumn 3, Issue , 2012, Pages 1871-1879

  Rombouts, Jaldert O ^a   Bohte, Sander M ^a   Roelfsema, Pieter R ^b  

^a CWI   (Netherlands)

^b NETHERLANDS INSTITUTE FOR NEUROSCIENCE   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

ACTION SELECTION; LEARNING SCHEMES; PARTIALLY OBSERVABLE MARKOV DECISION PROBLEMS; PERCEPTUAL DECISION MAKING; RELEVANT FEATURES; RELEVANT INFORMATIONS; SYNAPTIC PLASTICITY; TRIAL AND ERROR LEARNING;

NEURONS;

REINFORCEMENT LEARNING;

EID: 84877770904     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (36)

1. Gnadt, J. and Andersen, R. A. Memory Related motor planning activity in posterior parietal cortex of macaque. Experimental brain research., 70(1):216-220, 1988.
2. Sutton, R. S. and Barto, A. G. Reinforcement learning. MIT Press, Cambridge, MA, 1998.
3. Gottlieb, J. and Goldberg, M. E. Activity of neurons in the lateral intraparietal area of the monkey during an antisaccade task. Nature neuroscience, 2(10):906-12, 1999.
4. Bisley, J. W. and Goldberg, M. E. Attention, intention, and priority in the parietal lobe. Annual review of neuroscience, 33:1-21, 2010.
5. Gold, J. I. and Shadlen, M. N. The neural basis of decision making. Annual review of neuroscience, 30:535-74, 2007.
6. Yang, T. and Shadlen, M. N. Probabilistic reasoning by neurons. Nature, 447(7148):1075-80, 2007.
7. O'Reilly, R. C. and Frank, M. J. Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural computation, 18(2):283-328, 2006.
8. Izhikevich, E. M. Solving the distal reward problem through linkage of STDP and dopamine signaling. Cerebral cortex, 17(10):2443-52, 2007.
9. Montague, P. R., Hyman, S. E., et al. Computational roles for dopamine in behavioural control. Nature, 431(7010):760-7, 2004.
10. Rummery, G. A. and Niranjan, M. Online Q-learning using connectionist systems. Technical report, Cambridge University Engineering Department, 1994.
11. Funahashi, S., Bruce, C. J., et al. Mnemonic Coding of Visual Space in the Monkey's Dorsolateral Prefrontal Cortex. Journal of Neurophysiology, 6(2):331-349, 1989.
12. Cassenaer, S. and Laurent, G. Conditional modulation of spike-timing- dependent plasticity for olfactory learning. Nature, 482(7383):47-52, 2012.
13. Roelfsema, P. R. and van Ooyen, A. Attention-Gated Reinforcement Learning of Internal Representations for Classification. Neural Computation, 2214(17):2176-2214, 2005.
14. Schultz, W. Multiple dopamine functions at different time courses. Annual review of neuroscience, 30:259-88, 2007.
15. Wiering, M. and Schmidhuber, J. HQ-Learning. Adaptive Behavior, 6(2):219-246, 1997.
16. Rumelhart, D. E., Hinton, G. E., et al. Learning representations by back-propagating errors. Nature, 323(6088):533-536, 1986.
17. Krueger, K. A. and Dayan, P. Flexible shaping: how learning in small steps helps. Cognition, 110(3):380-94, 2009.
18. Sommer, M. A. and Wurtz, R. H. Frontal Eye Field Sends Delay Activity Related to Movement, Memory, and Vision to the Superior Colliculus. Journal of Neurophysiology, 85(4):1673-1685, 2001.
19. Soltani, A. and Wang, X.-J. Synaptic computation underlying probabilistic inference. Nature Neuroscience, 13(1):112-119, 2009.
20. Watkins, C. J. and Dayan, P. Q-learning. Machine learning, 292:279-292, 1992.
21. Morris, G., Nevet, A., et al. Midbrain dopamine neurons encode decisions for future action. Nature neuroscience, 9(8):1057-63, 2006.
22. Roesch, M. R., Calu, D. J., et al. Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nature neuroscience, 10(12):1615-24, 2007.
23. van Seijen, H., van Hasselt, H., et al. A theoretical and empirical analysis of Expected Sarsa. 2009 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, pages 177-184, 2009.
24. Baird, L. Residual algorithms: Reinforcement learning with function approximation. In Proceedings of the 26th International Conference on Machine Learning (ICML), pages 30-37, 1995.
25. Todd, M. T., Niv, Y., et al. Learning to use working memory in partially observable environments through dopaminic reinforcement. In NIPS, volume 21, pages 1689-1696, 2009.
26. Zipser, D. Recurrent network model of the neural mechanism of short-term active memory. Neural Computation, 3(2):179-193, 1991.
27. Moody, S. L., Wise, S. P., et al. A model that accounts for activity in primate frontal cortex during a delayed matching-to-sample task. The journal of Neuroscience, 18(1):399-410, 1998.
28. Nassi, J. J. and Callaway, E. M. Parallel processing strategies of the primate visual system. Nature reviews. Neuroscience, 10(5):360-72, 2009.
29. Hikosaka, O., Nakamura, K., et al. Basal ganglia orient eyes to reward. Journal of neurophysiology, 95(2):567-84, 2006.
30. Samejima, K., Ueda, Y., et al. Representation of action-specific reward values in the striatum. Science, 310(5752):1337-40, 2005.
31. Wang, X.-J. Synaptic reverberation underlying mnemonic persistent activity. Trends in neurosciences, 24(8):455-63, 2001.
32. Roelfsema, P. R., van Ooyen, A., et al. Perceptual learning rules based on reinforcers and attention. Trends in cognitive sciences, 14(2):64-71, 2010.
33. Deubel, H. and Schneider, W. Saccade target selection and object recognition: Evidence for a common attentional mechanism. Vision Research, 36(12):1827-1837, 1996.
34. Frey, U. and Morris, R. Synaptic tagging and long-term potentiation. Nature, 385(6616):533-536, 1997.
35. Moncada, D., Ballarini, F., et al. Identification of transmitter systems and learning tag molecules involved in behavioral tagging during memory formation. PNAS, 108(31):12931-6, 2011.
36. Sajikumar, S. and Korte, M. Metaplasticity governs compartmentalization of synaptic tagging and capture through brain-derived neurotrophic factor (BDNF) and protein kinase Mzeta (PKMzeta). PNAS, 108(6):2551-6, 2011.