A multilayer in-place learning network for development of general invariances

International Journal of Humanoid Robotics

Volumn 4, Issue 2, 2007, Pages 281-320

  Weng, Juyang ^a,b   Luwang, Tianyu ^a   Lu, Hong ^a   Xue, Xiangyang ^a  

^a FUDAN UNIVERSITY   (China)

^b Michigan State University   (United States)

Author keywords

Biologically inspired network; In place learning; Incremental learning; Internal representation; Positional invariance; Size invariance; Style invariance

Indexed keywords

BIOLOGICALLY INSPIRED NETWORK; INCREMENTAL LEARNING; MULTIPLE-LAYER IN-PLACE LEARNING NETWORK; POSITIONAL INVARIANCE;

COMPUTATIONAL COMPLEXITY; INVARIANCE; LEARNING ALGORITHMS; MULTILAYER NEURAL NETWORKS; NEURONS; ROBOTICS;

LEARNING SYSTEMS;

EID: 34447310006     PISSN: 02198436     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0219843607001072     Document Type: Article

References (48)

1. H. Abut (ed.), Vector Quantization (IEEE Press, Piscataway, NJ, 1990).
2. R. A. Andersen, C. Asanuma, G. Essick and R. M. Siegel, Cortico-cortical connections of anatomically and physiologically defined subdivisions within the inferior parietal lobule, J. Compar. Neurol. 296 (1990) 65-113.
3. J. J. Atick and A. N. Redlich, Towards a theory of early visual processing, Neural Comput. 2 (1990) 308-320.
4. J. J. Atick and A. N. Redlich, Convergent algorithm for sensory receptive field development, Neural Comput. 5 (1993) 45-60.
5. C. Blakemore and G. F. Cooper, Development of the brain depends on the visual environment, Nature 228 (October 1970) 477-478.
6. D. Boussaoud, L. C. Ungerleider and R. Desimone, Pathways for motion analysis: Cortical connections of the medial superior temporal and fundus of the superior temporal visual areas in the macaque, J. Compar. Neurol. 296 (1990) 462-495.
7. C. J. C. Burges, A tutorial on support vector machines for pattern recognition, Data Min. Knowl. Disc. 2(2) (1998) 121-167.
8. V. Dragoi and M. Sur, Plasticity of orientation processing in adult visual cortex, in Visual Neurosciences, eds. L. M. Chalupa and J. S. Werner (MIT Press, Cambridge, MA, 2004), pp. 1654-1664.
9. S. E. Fahlman and C. Lebiere, The cascade-correlation learning architecture, Technical Report CMU-CS-90-100, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA (February 1990).
10. D. J. Felleman and D. C. Van Essen, Distributed hierarchical processing in the primate cerebral cortex, Cerebral Cortex 1 (1991) 1-47.
11. D. Field, What is the goal of sensory coding?, Neural Comput. 6 (1994) 559-601.
12. K. Fukushima, Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position, Biol. Cybern. 36 (1980) 193-202.
13. D. H. Hubel, Eye, Brain and Vision (Scientific American Library, distributed by Freeman, New York, 1988).
14. D. H. Hubel and T. N. Wiesel, Receptive fields, binocular interaction and functional architecture in the cat's visual cortex, J. Physiol. 160(1) (1962) 107-155.
15. M. Hubener, D. Shoham, A. Grinvald and T. Bonhoeffer, Spatial relationships among three columnar systems in cat area 17, J. Neurosci. 17 (1997) 9270-9284.
16. W. S. Hwang and J. Weng, Hierarchical discriminant regression, IEEE Trans. Pattern Anal. Mach. Intell. 22(11) (2000) 1277-1293.
17. A. Hyvarinen, Survey on independent component analysis, Neural Comput. Surv. 2 (1999) 94-128.
18. A. Hyvarinen and P. Hover, Emergence of phase- and shift-invariant features by decomposition of natural images into independent feature subspaces, Neural Comput. 12 (2000) 1705-1720.
19. A. Hyvarinen, J. Karhunen and E. Oja, Independent Component Analysis (Wiley, New York, 2001).
20. M. Ito, G. Westheimer and C. D. Gilbert, Attention and perceptual learning modulate contextual influences on visual perception, Neuron 20 (1998) 1191-1197.
21. I. T. Jolliffe, Principal Component Analysis (Springer-Verlag, New York, 1986).
22. E. R. Kandel, J. H. Schwartz and T. M. Jessell (eds.), Principles of Neural Science, 4th edn. (McGraw-Hill, New York, 2000).
23. T. Kohonen, Self-Organizing Maps, 3rd edn. (Springer-Verlag, Berlin, 2001).
24. P. A. Kolers, R. L. Duchnicky and G. Sundstroem, Size in visual processing of faces and words, J. Exp. Psychol. Human Percept. Perform. 11 (1985) 726-751.
25. D. Lee and S. Seung, Learning the parts of objects by non-negative matrix factorization, Nature 401 (1999) 788-791.
26. E. L. Lehmann, Theory of Point Estimation (John Wiley and Sons, Inc., New York, 1983).
27. R. Miikkulainen, J. A. Bednar, Y. Choe and J. Sirosh, Computational Maps in the Visual Cortex (Springer, Berlin, 2005).
28. T. A. Nazir and J. K. O'Regan, Some results on translation invariance in the human visual system Spatial Vision 5(2) (1990) 81-100.
29. K. Obermayer, H. Ritter and K. J. Schulten, A neural network model for the formation of topographic maps in the CNS: Development of receptive fields in Proc. Int. Joint Conf. Neural Networks, San Diego, California (17-21 June 1990), pp. 423-429.
30. M. J. O'Donovan, The origin of spontaneous activity in developing networks of the vertebrate nervous systems, Curr. Opin. Neurobiol. 9 (1999) 94-104.
31. W. K. Purves, D. Sadava, G. H. Orians and H. C. Heller, Life: The Science of Biology, 7th edn. (Sinauer, Sunderland, MA, 2004).
32. W. L. Sejnowski and T. Sejnowski, Slow feature analysis: Unsupervised learning of invariances, Neural Comput. 14(4) (2002) 715-770.
33. F. Sengpiel, P. Stawinski and T. Bonhoeffer, Influence of experience on orientation maps in cat visual cortex, Nature Neurosci. 2(8) (1999) 727-732.
34. D. C. Somers, S. B. Nelson and M. Sur, An emergent model of orientation selectivity in cat visual cortical simple cells, J. Neurosci. 15(8) (1995) 5448-5465.
35. D. C. Somers, E. V. Todorov, A. G. Siapas, D. S. Kim L. J. Toth and M. Sur, A local circuit approach to understanding integration of long-range inputs in primary visual cortex, Cerebral Cortex 8(3) (1998) 204-217.
36. M. Sur and C. A. Leamey, Development and plasticity of cortical areas and networks, Nature Rev. Neurosci. 2 (2001) 251-262.
37. M. Sur and J. L. R. Rubenstein, Patterning and plasticity of the cerebral cortex, Science 310 (2005) 805-810.
38. J. B. Tenenbaum, V. de Silva and J. C. Langford, A global geometric framework for nonlinear dimensionality reduction, Science 290 (2000) 2319-2323.
39. P. Thompson, Margaret Thatcher: A new illusion, Perception 9 (1980) 483-484.
40. A. Van der Heijden, Selective Attention in Vision (Routledge, New York, 1992).
41. C. von der Malsburg, Self-organization of orientation-sensitive cells in the striate cortex, Kybernetik 15 (1973) 85-100.
42. J. Weng, Developmental robotics: Theory and experiments, Int. J. Humanoid Robot. 1(2) (2004) 199-235.
43. J. Weng, N. Ahuja and T. S. Huang, Learning recognition and segmentation using the Cresceptron, Int. J. Comput. Vision 25(2) (November 1997) 109-143.
44. J. Weng and W. Hwang, Online image classification using IHDR, Int. J. Doc. Anal. Recogn. 5(2-3) (2002) 118-125.
45. J. Weng, J. McClelland, A. Pentland, O. Sporns, I. Stockman, M. Sur and E. Thelen, Autonomous mental development by robots and animals, Science 291(5504) (2001) 599-600.
46. J. Weng and N. Zhang, In-place learning and the lobe component analysis, in Proc. IEEE World Congress on Computational Intelligence, Vancouver, BC, Canada (16-21 July 2006).
47. J. Weng, Y. Zhang and W. Hwang, Candid covariance-free incremental principal component analysis, IEEE Trans. Pattern Anal. Mach. Intell. 25(8) (2003) 1034-1040.
48. N. Zhang and J. Weng, A developing sensory mapping for robots, in Proc. IEEE 2nd Int. Conf. Development and Learning (ICDL 2002), MIT, Cambridge, MA (12-15 June 2002), pp. 13-20.