Scaled free-energy based reinforcement learning for robust and efficient learning in high-dimensional state spaces

Frontiers in Neurorobotics

Volumn 7, Issue FEB, 2013, Pages

  Elfwing, Stefan ^a   Uchibe, Eiji ^a   Doya, Kenji ^a  

^a OKINAWA INSTITUTE OF SCIENCE AND TECHNOLOGY GRADUATE UNIVERSITY   (Japan)

Author keywords

Free energy; Function approximation; Reinforcement learning; Restricted boltzmann machine; Robot navigation

Indexed keywords

ARTICLE; CAMERA; COLOR; DELAY DISCOUNTING; FREE ENERGY BASED REINFORCEMENT LEARNING; LEARNING ALGORITHM; MACHINE LEARNING; NONHUMAN; REWARD; ROBOTICS; SENSOR; SIMULATION; TEMPERATURE;

EID: 84892587881     PISSN: None     EISSN: 16625218     Source Type: Journal    
DOI: 10.3389/fnbot.2013.00003     Document Type: Article

References (21)

1. Arleo, A., and Gerstner, W. (2000). Spatial cognition and neuro-mimetic navigation: a model of hippocampal place cell activity. Biol. Cybern. 83, 287-299.
2. Barrera, A., and Weitzenfeld, A. (2008). Biologically-inspired robot spatial cognition based on rat neurophysiological studies. Auton. Robots 25, 147-169.
3. Caluwaerts, K., Staffa, M., N'Guyen, S., Grand, C., Dollé, L., Favre-Félix, A., et al. (2012). A biologically inspired meta-control navigation system for the psikharpax rat robot. Bioinspir. Biomim. 7, 1-29.
4. Doya, K., Samejima, K., Katagiri, K., and Kawato, M. (2002). Multiple model-based reinforcement learning. Neural Comput. 14, 1347-1369.
5. Doya, K., and Uchibe, E. (2005). The cyber rodent project: exploration of adaptive mechanisms for self-preservation and self-reproduction. Adapt. Behav. 13, 149-160.
6. Elfwing, S., Otsuka, M., Uchibe, E., and Doya, K. (2010). "Free-energy based reinforcement learning for vision-based navigation with high-dimensional sensory inputs," in Proceedings of the International Conference on Neural Information Processing (ICONIP2010) (Berlin, Heidelberg), 215-222.
7. Fleischer, J. G., Gally, J. A., Edelman, G. M., and Krichmar, J. L. (2007). Retrospective and prospective responses arising in a modeled hippocampus during maze navigation by a brain-based device. Proc. Natl. Acad. Sci. U.S.A. 104, 3556-3561.
8. Freund, Y., and Haussler, D. (1992). "Unsupervised learning of distributions on binary vectors using two layer networks," in Advances in Neural Information Processing Systems 4, eds J. E. Moody, S. J. Hanson, and R. P. Lippmann (Denver, CO: Morgan Kaufmann), 912-919.
9. Giovannangeli, C., and Gaussier, P. (2008). "Autonomous vision-based navigation: goal-oriented action planning by transient states prediction, cognitive map building, and sensory-motor learning," in Proceedings of the International Conference on Intelligent Robots and Systems (IROS2008) (Nice, France), 676-683.
10. Hinton, G. E. (2002). Training products of experts by minimizing contrastive divergence. Neural Comput. 12, 1771-1800.
11. Krichmar, J. L., Seth, A. K., Nitz, D. A., Fleischer, J., and Edelman, G. M. (2005). Spatial navigation and causal analysis in a brain-based device modeling cortical-hippocampal interactions. Neuroinformatics 3, 197-221.
12. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998). Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278-2324.
13. MATLAB. (2010). version 7.14.0 (R2012a). Natick, MA: The MathWorks Inc.
14. Milford, M., and Wyeth, G. (2010). Persistent navigation and mapping using a biologically inspired slam system. Int. J. Rob. Res. 29, 1131-1153.
15. Otsuka, M., Yoshimoto, J., and Doya, K. (2010). "Free-energy-based reinforcement learning in a partially observable environments," in Proceedings of the European Symposium on Artificial Neural Networks (ESANN2010) (Bruges, Belgium), 541-545.
16. Rummery, G. A., and Niranjan, M. (1994). On-line Q-learning Using Connectionist Systems. Technical Report CUED/F-INFENG/TR 166, Cambridge University Engineering Department.
17. Sallans, B., and Hinton, G. E. (2004). Reinforcement learning with factored states and actions. J. Mach. Learn. Res. 5, 1063-1088.
18. Smolensky, P. (1986). "Information processing in dynamical systems: foundations of harmony theory," in Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundations, eds D. E. Rumelhart and J. L. McClelland (Cambridge, MA: MIT Press), 194-281.
19. Sutton, R. S. (1996). "Generalization in reinforcement learning: successful examples using sparse coarse coding," in Advances in Neural Information Processing Systems 8, eds D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo (Denver, CO: MIT Press), 1038-1044.
20. Sutton, R. S., and Barto, A. (1998). Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press.
21. Uchibe, E., and Doya, K. (2004). "Competitive-cooperative-concurrent reinforcement learning with importance sampling," in Proceedings of the International Conference on Simulation of Adaptive Behavior: From Animals and Animats (SAB2004) (Santa Monica, CA: MIT Press), 287-296.