Unifying perceptual and behavioral learning with a correlative subspace learning rule

Neurocomputing

Volumn 73, Issue 10-12, 2010, Pages 1818-1830

  Duff, Armin ^a,b   Verschure, Paul F M J ^b,c  

^a Institute of Neuroinformatics Zurich   (Switzerland)

^b UNIVERSITAT POMPEU FABRA   (Spain)

^c INSTITUCIÓ CATALANA DE RECERCA I ESTUDIS AVANÇATS ICREA   (Spain)

Author keywords

Classical conditioning; Correlative learning; Distributed adaptive control; Model; Perceptual and behavioral learning; Robot

Indexed keywords

CLASSICAL CONDITIONING; CORRELATIVE LEARNING; DISTRIBUTED ADAPTIVE CONTROL; FORAGING TASK; FORMAL ANALYSIS; NUMERICAL SIMULATION; PERCEPTUAL LEARNING; SUBSPACE LEARNING;

ADAPTIVE CONTROL SYSTEMS; COMPUTER SIMULATION; ROBOTS; SPREADSHEETS;

LEARNING ALGORITHMS;

ADAPTIVE BEHAVIOR; ANIMAL BEHAVIOR; ARTICLE; BEHAVIOR; COMPUTER SIMULATION; CONDITIONING; DISTRIBUTED ADAPTIVE CONTROL; INFORMATION PROCESSING; LEARNING ALGORITHM; MACHINE LEARNING; MATHEMATICAL ANALYSIS; MATHEMATICAL MODEL; NONHUMAN; PERCEPTION; PRIORITY JOURNAL; PROCESS OPTIMIZATION; REINFORCEMENT; SENSORIMOTOR FUNCTION; SIMULATION; ZETA POTENTIAL;

EID: 77952549819     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2009.11.048     Document Type: Article

References (115)

1. Verschure P.F.M.J., Voegtlin T., Douglas R.J. Environmentally mediated synergy between perception and behaviour in mobile robots. Nature 2003, 425(6958):620-624.
2. Dvorkin A., Benjamini Y., Golani I. Mouse cognition-related behavior in the open-field: emergence of places of attraction. PLoS Comput. Biol. 2008, 4(2):e1000027.
3. Gibson J.J. The Ecological Approach to Visual Perception 1979, Lawrence Erlbaum, New Jersey, USA.
4. Goldstone R. Perceptual learning. Annu. Rev. Psychol. 1998, 49(1):585-612.
5. Edeline J.M. Learning-induced physiological plasticity in the thalamo-cortical sensory systems: a critical evaluation of receptive field plasticity, map changes and their potential mechanisms. Prog. Neurobiol. 1999, 57(2):165-224.
6. Ohl F.W., Scheich H. Learning-induced plasticity in animal and human auditory cortex. Curr. Opin. Neurobiol. 2005, 15(4):470-477.
7. Weinberger N.M. Learning-induced changes of auditory receptive fields. Curr. Opin. Neurobiol. 1993, 3(4):570-577.
8. Rutkowski R.G., Weinberger N.M. Encoding of learned importance of sound by magnitude of representational area in primary auditory cortex. Proc. Natl. Acad. Sci. USA 2005, 102(38):13664-13669.
9. Berlau K.M., Weinberger N.M. Learning strategy determines auditory cortical plasticity. Neurobiol. Learn. Mem. 2008, 89(2):153-166.
10. Di Pellegrino G., Fadiga L., Fogassi L., Gallese V., Rizzolatti G. Understanding motor events: a neurophysiological study. Exp. Brain Res. 1992, 91(1):176-180.
11. Rizzolatti G., Craighero L. The mirror-neuron system. Annu. Rev. Neurosci. 2004, 27(1):169-192.
12. Fabbri-Destro M., Rizzolatti G. Mirror neurons and mirror systems in monkeys and humans. Physiology 2008, 23(3):171-179.
13. Olshausen B.A., Field D.J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 1996, 381(6583):607-609.
14. Oja E., Ogawa H., Wangviwattana J. Principal component analysis by homogeneous neural networks. Part I: the weighted subspace criterion. IEICE Trans. Inf. Syst. 1992, 75:366-375.
15. Sutton R., Barto A.G. Reinforcement Learning: An Introduction 1998, MIT Press, Cambridge, USA.
16. Hopfield J.J. Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA 1982, 79(8):2554-2558.
17. Rosenblatt F. The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 1958, 65(6):386-408.
18. Becker S., Plumbley M. Unsupervised neural network learning procedures for feature extraction and classification. Appl. Intell. 1996, 6(3):185-203.
19. Rao R., Olshausen B., Lewicki M. Probabilistic Models of the Brain: Perception and Neural Function 2002, MIT Press, Cambridge, USA.
20. Jolliffe I. Principal Component Analysis 2002, Springer, New York.
21. Oja E. A simplified neuron model as a principal component analyzer. J. Math. Biol. 1982, 15(3):267-273.
22. P. Földiák, Adaptive network for optimal linear feature extraction, in: International Joint Conference on Neural Networks, IJCNN, vol. 1, 1989, pp. 401-405.
23. Sanger T. Optimal unsupervised learning in a single-layer linear feedforward neural network. Neural Netw. 1989, 2(6):459-473.
24. Karhunen J., Joutsensalo J. Representation and separation of signals using nonlinear PCA type learning. Neural Netw. 1994, 7(1):113-127.
25. Jankovic M.V., Ogawa H. Modulated Hebb-Oja learning rule: a method for principal subspace analysis. IEEE Trans. Neural Netw. 2006, 17(2):345-356.
26. Pang Y., Yuan Y., Li X. Generalised nearest feature line for subspace learning. Electron. Lett. 2007, 43(20):1079-1080.
27. Pang Y., Tao D., Yuan Y., Li X. Binary two-dimensional PCA. IEEE Trans. Syst. Man Cybernet. B Cybernet. 2008, 38(4):1176-1180.
28. Pang Y., Yuan Y., Li X. Iterative subspace analysis based on feature line distance. IEEE Trans. Image Process. 2009, 18(4):903-907.
29. Pang Y., Li X., Yuan Y., Tao D., Pan J. Fast haar transform based feature extraction for face representation and recognition. IEEE Trans. Inf. Forensics Secur. 2009, 4(3):441-450.
30. Chen T., Hua Y., Yan W.Y. Global convergence of Oja's subspace algorithm for principal component extractions. IEEE Trans. Neural Netw. 1998, 9(1):58-67.
31. Weingessel A., Hornik K. Local PCA algorithms. IEEE Trans. Neural Netw. 2000, 11(6):1242-1250.
32. Friston K.J., Frith C.D., Frackowiak R.S. Principal component analysis learning algorithms: a neurobiological analysis. Proc. Biol. Sci. 1993, 254(1339):47-54.
33. Hyvarinen A., Karhunen J., Oja E. Independent Component Analysis 2001, J. Wiley, New York.
34. Bell A., Sejnowski T. The "independent components" of natural scenes are edge filters. Vision Res. 1997, 37(23):3327-3338.
35. Hoyer P.O., Hyvärinen A. A multi-layer sparse coding network learns contour coding from natural images. Vision Res. 2002, 42(12):1593-1605.
36. Hurri J., Hyvärinen A. Simple-cell-like receptive fields maximize temporal coherence in natural video. Neural Comput. 2003, 15(3):663-691.
37. Berkes P., Wiskott L. Slow feature analysis yields a rich repertoire of complex cell properties. J. Vis. 2005, 5(6):579-602.
38. Körding K.P., Kayser C., Einhäuser W., König P. How are complex cell properties adapted to the statistics of natural stimuli?. J. Neurophysiol. 2004, 91(1):206-212.
39. Hashimoto W. Quadratic forms in natural images. Network 2003, 14(4):765-788.
40. Einhäuser W., Kayser C., König P., Körding K.P. Learning the invariance properties of complex cells from their responses to natural stimuli. Eur. J. Neurosci. 2002, 15(3):475-486.
41. Kayser C., Körding K., König P. Processing of complex stimuli and natural scenes in the visual cortex. Curr. Opin. Neurobiol. 2004, 14(4):468-473.
42. Einhäuser W., Kayser C., Körding K.P., König P. Learning distinct and complementary feature-selectivities from natural colour videos. J. Neurosci. 2003, 21:43-52.
43. Stringer S.M., Rolls E.T. Invariant object recognition in the visual system with novel views of 3D objects. Neural Comput. 2002, 14(11):2585-2596.
44. Einhäuser W., Hipp J., Eggert J., Korner E., König P. Learning viewpoint invariant object representations using a temporal coherence principle. Biol. Cybernet. 2005, 93(1):79-90.
45. Wyss R., König P., Verschure P.F.M.J. A model of the ventral visual system based on temporal stability and local memory. PLoS Biol. 2006, 4(5):e120.
46. König P., Krüger N. Symbols as self-emergent entities in an optimization process of feature extraction and predictions. Biol. Cybernet. 2006, 94(4):325-334.
47. Hipp J., Einhäuser W., Conradt J., König P. Learning of somatosensory representations for texture discrimination using a temporal coherence principle. Network 2005, 16(2-3):223-238.
48. Lewicki M.S. Efficient coding of natural sounds. Nat. Neurosci. 2002, 5(4):356-363.
49. Smith E., Lewicki M.S. Efficient coding of time-relative structure using spikes. Neural Comput. 2005, 17(1):19-45.
50. Smith E.C., Lewicki M.S. Efficient auditory coding. Nature 2006, 439(7079):978-982.
51. Klein D.J., König P., Körding K.P. Sparse spectrotemporal coding of sounds. EURASIP JASP 2003, 3:659-667.
52. A. Duff, R. Wyss, P.F.M.J. Verschure, Learning temporally stable representations from natural sounds: temporal stability as a general objective underlying sensory processing, in: J. Marques de Sá, L.A. Alexandre, W. Duch, D. Mandic (Eds.), Lecture Notes in Computer Science: Artificial Neural Networks ICANN, 2007, pp. 129-138.
53. Thorndike E. Animal Intelligence 1911, Macmillan, New York.
54. Sutton R.S., Barto A.G. Toward a modern theory of adaptive networks: expectation and prediction. Psychol. Rev. 1981, 88(2):135-170.
55. Klopf A.H. A neuronal model of classical conditioning. Psychobiology 1988, 16(2):85-125.
56. Montague P.R., Dayan P., Person C., Sejnowski T.J. Bee foraging in uncertain environments using predictive Hebbian learning. Nature 1995, 377(6551):725-728.
57. Barraclough D.J., Conroy M.L., Lee D. Prefrontal cortex and decision making in a mixed-strategy game. Nat. Neurosci. 2004, 7(4):404-410.
58. Kaelbling L., Littman M., Cassandra A. Planning and acting in partially observable stochastic domains. Artif. Intell. 1998, 101(1-2):99-134.
59. Burago D., de Rougemont M., Slissenko A. On the complexity of partially observed Markov decision processes. Theor. Comput. Sci. 1996, 157(2):161-183.
60. Verschure P.F.M.J., Pfeifer R. Categorization, representations, and the dynamics of system-environment interaction: a case study in autonomous systems. From Animals to Animates: Proceedings of the Second International Conference on Simulation of Adaptive Behavior, Honolulu, Hawaii 1992, 210-217. MIT Press, Cambridge, USA. J.A. Meyer, H. Roitblat, S. Wilson (Eds.).
61. Hebb D. The Organization of Behavior: A Neuropsychological Theory 1949, Wiley, New York.
62. Pavlov I.P. Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex 1927, Oxford University Press, Oxford, New York.
63. A. Klopf, A drive-reinforcement model of single neuron function: an alternative to the Hebbian neuronal model, in: AIP Conference Proceedings, vol. 151, 1986, pp. 265-270.
64. B. Kosko, Differential Hebbian learning, in: AIP Conference Proceedings, vol. 151, 1986, pp. 277-282.
65. Porr B., Wörgötter F. Isotropic sequence order learning. Neural Comput. 2003, 15(4):831-864.
66. Porr B., Wörgötter F. Strongly improved stability and faster convergence of temporal sequence learning by using input correlations only. Neural Comput. 2006, 18(6):1380-1412.
67. Manoonpong P., Geng T., Kulvicius T., Porr B., Wörgötter F. Adaptive, fast walking in a biped robot under neuronal control and learning. PLoS Comput. Biol. 2007, 3(7):e134.
68. Kulvicius T., Porr B., Wörgötter F. Chained learning architectures in a simple closed-loop behavioural context. Biol. Cybernet. 2007, 97(5-6):363-378.
69. Porr B., Wörgötter F. Fast heterosynaptic learning in a robot food retrieval task inspired by the limbic system. Biosystems 2007, 89(1-3):294-299.
70. P.F.M.J. Verschure, Synthetic epistemology: the acquisition, retention, and expression of knowledge in natural and synthetic systems, in: IEEE World Conference on Computational Intelligence, Proceedings, 1998, pp. 147-152.
71. Verschure P.F.M.J., Althaus P. A real-world rational agent: unifying old and new AI. Cogn. Sci. 2003, 27:561-590.
72. Verschure P.F.M.J., Coolen A.C.C. Adaptive fields: distributed representations of classically conditioned associations. Network 1991, 2(2):189-206.
73. Verschure P.F.M.J., Krose B., Pfeifer R. Distributed adaptive control: the self-organization of structured behavior. Rob. Auton. Syst. 1993, 9:181-196.
74. Voegtlin T., Verschure P.F.M.J. What can robots tell us about brains? A synthetic approach towards the study of learning and problem solving. Rev. Neurosci. 1999, 10(3-4):291-310.
75. Oja E., Ogawa H., Wangviwattana J. Principal component analysis by homogeneous neural networks, Part II: Analysis and extensions of the learning algorithms. IEICE Trans. Inf. Syst. 1992, 3:376-382.
76. Golub G., Van Loan C. Matrix Computations 1996, The Johns Hopkins University Press, Baltimore.
77. Peebles P.Z. Probability, Random Variables, and Random Signal Principles 1993, McGraw-Hill, New York.
78. Davies P.I., Higham N.J. Numerically stable generation of correlation matrices and their factors. BIT Num. Math. 2000, 40(4):640-651.
79. R. Wyss, Sensory and motor coding in the organization of behavior, Ph.D. Thesis, ETHZ, 2003.
80. Wyss R., König P., Verschure P.F.M.J. Involving the motor system in decision making. Proc. R. Soc. Lond. B Biol. Sci. Suppl. 2003, 271(3):S50-S52.
81. Braitenberg V. Vehicles, Experiments in Synthetic Psychology 1984, MIT Press, Cambridge, USA.
82. Weinberger N.M. Experience-dependent response plasticity in the auditory cortex: issues characteristics, mechanisms and functions. Plasticity of the Auditory System 2004, 173-228. Springer, New York. T.N. Parks, E.W. Rubel, R.R. Fay, A.N. Popper (Eds.).
83. Harley C.W. Norepinephrine and the dentate gyrus. Prog. Brain Res. 2007, 163:299-318.
84. Grenhoff J., Nisell M., Ferre S., Aston-Jones G., Svensson T.H. Noradrenergic modulation of midbrain dopamine cell firing elicited by stimulation of the locus coeruleus in the rat. J. Neural Transm. Gen. Sect. 1993, 93(1):11-25.
85. R. Fisher, The use of multiple measurements in taxonomic problems, Ann. Eugenics 7 (1936) 179-188.
86. Rao C. The utilization of multiple measurements in problems of biological classification. J. R. Stat. Soc. B Methodol. 1948, 10:159-203.
87. S. Roweis, L. Saul, Nonlinear dimensionality reduction by locally linear embedding, Science 290 (2000) 2323-2326.
88. H. Yu, J. Yang, A direct lda algorithm for high-dimensional data with application to face recognition, Pattern Recognition 34 (12) (2001) 2067-2070.
89. Kim T., Kittler J. Locally linear discriminant analysis for multimodally distributed classes for face recognition with a single model image. IEEE Trans. Pattern Anal. Mach. Intell. 2005, 27(3):318-327.
90. Yan S., Xu D., Zhang B., Zhang H., Yang Q., Lin S. Graph embedding and extensions: a general framework for dimensionality reduction. IEEE Trans. Pattern Anal. Mach. Intell. 2007, 29(1):40-51.
91. Li X., Lin S., Yan S., Xu D. Discriminant locally linear embedding with high-order tensor data. IEEE Trans. Syst. Man Cybernet. B Cybernet. 2008, 38(2):342-352.
92. Tao D., Li X., Wu X., Maybank S. Geometric mean for subspace selection. IEEE Trans. Pattern Anal. Mach. Intell. 2009, 31(2):260-274.
93. T. Zhang, D. Tao, J. Yang, Discriminative locality alignment, in: ECCV '08: Proceedings of the 10th European Conference on Computer Vision, Springer-Verlag, Berlin, Heidelberg, 2008, pp. 725-738.
94. W. Liu, D. Tao, J. Liu, Transductive component analysis, in: Eighth IEEE International Conference on Data Mining, ICDM '08, 2008, pp. 433-442.
95. Porr B., Ferber C., Wörgötter F. ISO learning approximates a solution to the inverse-controller problem in an unsupervised behavioral paradigm. Neural Comput. 2003, 15(4):865-884.
96. Porr B., Wörgötter F. Learning with "relevance": using a third factor to stabilize Hebbian learning. Neural Comput. 2007, 19(10):2694-2719.
97. Kalman R. A new approach to linear filtering and prediction problems. Trans. ASME J. Basic Eng. 1960, 82(1):35-45.
98. Thrun S., Burgard W., Fox D. Probabilistic Robotics (Intelligent Robotics and Autonomous Agents) 2005, MIT Press, Cambridge, USA.
99. Montemerlo M., Thrun S. FastSLAM: A Scalable Method for the Simultaneous Localization and Mapping Problem in Robotics (Springer Tracts in Advanced Robotics) 2007, Springer, New York.
100. Tovar B., Muñoz-Gómez L., Murrieta-Cid R., Alencastre-Miranda M., Monroy R., Hutchinson S. Planning exploration strategies for simultaneous localization and mapping. Rob. Auton. Syst. 2006, 54(4):314-331.
101. Rekleitis I., Meger D., Dudek G. Simultaneous planning localization, and mapping, in a camera sensor network. Rob. Auton. Syst. 2006, 54(11):921-932.
102. P. Sermanet, R. Hadsell, M. Scoffier, U. Muller, Y. LeCun, Mapping and planning under uncertainty in mobile robots with long-range perception, in: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2008, pp. 2525-2530.
103. Sahin E., Cakmak M., Dogar M., Ugur E., Ucoluk G. To afford or not to afford: a new formalization of affordances toward affordance-based robot control. Adapt. Behav. 2007, 15(4):447-472.
104. Montesano L., Lopes M., Bernardino A., Santos-Victor J. Learning object affordances: from sensory-motor coordination to imitation. IEEE Trans. Robot. 2008, 24(1):15-26.
105. Arleo A., Smeraldi F., Gerstner W. Cognitive navigation based on nonuniform gabor space sampling, unsupervised growing networks, and reinforcement learning. IEEE Trans. Neural Netw. 2004, 15(3):639-652.
106. Friston K., Stephan K. Free-energy and the brain. Syntheses 2007, 159(3):417-458.
107. Kiebel S.J., Daunizeau J., Friston K.J. A hierarchy of time-scales and the brain. PLoS Comput. Biol. 2008, 4(11):e1000209.
108. Cisek P. Cortical mechanisms of action selection: the affordance competition hypothesis. Philos. Trans. R. Soc. Lond. B Biol. Sci. 2007, 362(1485):1585-1599.
109. Connolly P.M., Bennur S., Gold J.I. Correlates of perceptual learning in an oculomotor decision variable. J. Neurosci. 2009, 29(7):2136-2150.
110. Law C., Gold J.I. Neural correlates of perceptual learning in a sensory-motor, but not a sensory, cortical area. Nat. Neurosci. 2008, 11(4):505-513.
111. Vogel E.H., Castro M.E., Saavedra M.A. Quantitative models of pavlovian conditioning. Brain Res. Bull. 2004, 63(3):173-202.
112. Yamazaki T., Tanaka S. Neural modeling of an internal clock. Neural Comput. 2005, 17(5):1032-1058.
113. Scholkopf B., Smola A., Muller K. Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 1998, 10(5):1299-1319.
114. Kim K.I., Franz M.O., Scholkopf B. Iterative kernel principal component analysis for image modeling. IEEE Trans. Pattern Anal. Mach. Intell. 2005, 27(9):1351-1366.
115. Hyvärinen A., Hoyer P. Emergence of phase- and shift-invariant features by decomposition of natural images into independent feature subspaces. Neural Comput. 2000, 12(7):1705-1720.