Low complexity proto-value function learning from sensory observations with incremental slow feature analysis

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 7553 LNCS, Issue PART 2, 2012, Pages 279-287

  Luciw, Matthew ^a   Schmidhuber, Juergen ^a  

^a DALLE MOLLE INSTITUTE FOR ARTIFICIAL INTELLIGENCE IDSIA   (Switzerland)

Author keywords

Biologically Inspired Reinforcement Learning; Incremental Slow Feature Analysis; Proto Value Functions

Indexed keywords

ADJACENCY MATRICES; BASIS SETS; BIOLOGICALLY INSPIRED; FUNCTION LEARNING; HIGH-DIMENSIONAL; LINEAR APPROXIMATIONS; SENSORY INPUT; SLOW FEATURE ANALYSIS; SPACE AND TIME; TRANSITION MODEL; VALUE FUNCTION APPROXIMATION; VALUE FUNCTIONS;

COVARIANCE MATRIX; REINFORCEMENT LEARNING;

NEURAL NETWORKS;

EID: 84867667632     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-33266-1_35     Document Type: Conference Paper

References (22)

1. Bradtke, S.J., Barto, A.G.: Linear least-squares algorithms for temporal difference learning. Machine Learning 22(1), 33-57 (1996)
2. Chung, F.R.K.: Spectral graph theory. AMS Press, Providence (1997)
3. Coifman, R.R., Lafon, S., Lee, A.B., Maggioni, M., Nadler, B., Warner, F., Zucker, S.W.: Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps. Proceedings of the National Academy of Sciences of the United States of America 102(21), 7426 (2005)
4. Coifman, R.R., Maggioni, M.: Diffusion wavelets. Applied and Computational Harmonic Analysis 21(1), 53-94 (2006)
5. da Motta Salles Barreto, A., Anderson, C.W.: Restricted gradient-descent algorithm for value-function approximation in reinforcement learning. Artificial Intelligence 172(4-5), 454-482 (2008)
6. Dayan, P., Abbott, L.F.: Theoretical neuroscience: Computational and mathematical modeling of neural systems (2001)
7. Forsythe, G.E., Henrici, P.: The cyclic Jacobi method for computing the principal values of a complex matrix. Applied Mathematics and Statistics Laboratories, Stanford University (1958)
8. Franzius, M., Sprekeler, H., Wiskott, L.: Slowness and sparseness lead to place, head-direction, and spatial-view cells. PLoS Computational Biology 3(8), e166 (2007)
9. Kompella, V.R., Luciw, M.D., Schmidhuber, J.: Incremental slow feature analysis: Adaptive low-complexity slow feature updating from high-dimensional input streams. Neural Computation (accepted and to appear, 2012)
10. Kompella, V.R., Luciw, M., Schmidhuber, J.: Incremental slow feature analysis. In: International Joint Conference of Artificial Intelligence (2011)
11. Lange, S., Riedmiller, M.: Deep auto-encoder neural networks in reinforcement learning. In: International Joint Conference on Neural Networks, Barcelona, Spain (2010)
12. Legenstein, R., Wilbert, N., Wiskott, L.: Reinforcement learning on slow features of high-dimensional input streams. PLoS Computational Biology 6(8) (2010)
13. Lin, L.J.: Reinforcement learning for robots using neural networks. School of Computer Science, Carnegie Mellon University (1993)
14. Luciw, M., Kompella, V.R., Schmidhuber, J.: Hierarchical incremental slow feature analysis. In: Workshop on Deep Hierarchies in Vision (2012)
15. Mahadevan, S.: Proto-value functions: Developmental reinforcement learning. In: Proceedings of the 22nd International Conference on Machine Learning, pp. 553-560. ACM (2005)
16. Menache, I., Mannor, S., Shimkin, N.: Basis function adaptation in temporal difference reinforcement learning. Annals of Operations Research 134(1), 215-238 (2005)
17. Santamaria, J.C., Sutton, R.S., Ram, A.: Experiments with reinforcement learning in problems with continuous state and action spaces. Adaptive Behavior 6(2), 163 (1997)
18. Schmidhuber, J.: A local learning algorithm for dynamic feedforward and recurrent networks. Connection Science 1(4), 403-412 (1989)
19. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8), 888-905 (2000)
20. Sprekeler, H.: On the relation of slow feature analysis and laplacian eigenmaps. Neural Computation, 1-16 (2011)
21. Sutton, R.S., Barto, A.G.: Reinforcement learning: An introduction, vol. 1. Cambridge Univ. Press (1998)
22. Wiskott, L., Sejnowski, T.: Slow feature analysis: Unsupervised learning of invariances. Neural Computation 14(4), 715-770 (2002)