Topographic map formation by maximizing unconditional entropy: A plausible strategy for "On-line" unsupervised competitive learning and nonparametric density estimation

IEEE Transactions on Neural Networks

Volumn 7, Issue 5, 1996, Pages 1299-1305

  Van Hulle, Marc M ^a  

^a UNIVERSITY OF LEUVEN   (Belgium)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; COMPUTER SIMULATION; ESTIMATION; INFORMATION THEORY; LEARNING SYSTEMS; MATHEMATICAL MODELS; MEMBERSHIP FUNCTIONS; ONLINE SYSTEMS; OPTIMIZATION; PROBABILITY DENSITY FUNCTION; TOPOLOGY; VECTORS;

EQUIPROBABLE QUANTIZATION; TOPOGRAPHIC MAP FORMATION; UNCONDITIONAL ENTROPY MAXIMIZATION; UNSUPERVISED COMPETITIVE LEARNING; VECTORIAL BOUNDARY ADAPTATION RULE;

NEURAL NETWORKS;

EID: 0030243661     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.536323     Document Type: Article

References (22)

1. H. B. Barlow, "The coding of sensory messages," in Current Problems in Animal Behavior. Cambridge, U.K.: Cambridge Univ. Press, 1961.
2. _, "Unsupervised learning," Neural Computa., vol. 1, pp. 295-311, 1989.
3. S. B. Gardner, "Application of neural network algorithms and architectures to correlation/tracking and identification," in Neural Networks for Computing, J. S. Denker, Ed. New York: Amer. Inst. Physics, 1986, pp. 153-157.
4. R. Linsker, "Self-organization in a perceptual network," Comput., vol. 21, pp. 105-117, 1988.
5. _, "How to generate ordered maps by maximizing the mutual information between input and output signals," Neural Computa., vol. 1, pp. 402-411, 1989.
6. G. Rigoll, "Maximum mutual information neural networks for hybrid connectionist-HMM speech recognition systems," IEEE Trans. Speech Audio Processing, vol. 2, pp. 175-184, 1994.
7. M. Bichsel and P. Seitz, "Minimum class entropy: A minimum information approach to layered networks," Neural Networks, vol. 2, pp. 133-141, 1989.
8. T. Kohonen, "Self-organized formation of topologically correct feature maps," Biol. Cybern., vol. 43, pp. 59-69, 1982.
9. _, Self-Organization and Associative Memory. Berlin: Springer-Verlag, 1989.
10. H. White, "Parametric statistical estimation with artificial neural networks," in Mathematical Perspectives on Neural Networks, P. Smolensky, M. C. Mozer, and D. E. Rumelhart, Eds. Hillsdale, NJ: Erlbaum, 1992.
11. D. S. Modha and Y. Fainman, "A learning law for density estimation," IEEE Trans. Neural Networks, vol. 5, pp. 519-523, 1994.
12. M. M. Van Hulle and D. Martinez, "On a novel unsupervised competitive learning algorithm for scalar quantization," IEEE Trans. Neural Networks, vol. 5, pp. 498-501, 1994.
13. D. Martinez and M. M. Van Hulle, "Generalized boundary adaptation rule for minimizing rth power law distortion in high-resolution quantization," Neural Networks, vol. 6, pp. 891-900, 1995.
14. H.-U. Bauer and K. R. Pawelzik, "Quantifying the neighborhood preservation of self-organizing feature maps," IEEE Trans. Neural Networks, vol. 3, pp. 570-579, 1992.
15. T. Geszti, Physical Models of Neural Networks. Singapore: World Scientific, 1990.
16. L. Ljung, "Analysis of recursive stochastic algorithms," IEEE Trans. Automat. Contr., vol. AC-22, pp. 551-575, 1977.
17. H. J. Kushner and D. S. Clark, Stochastic Approximation Methods for Constrained and Unconstrained Systems. New York: Springer-Verlag, 1978.
18. K. Hornik and C. Kuan, "Convergence analysis of local feature extraction algorithms," Neural Networks, vol. 5, pp. 229-240, 1992.
19. C. Kuan and K. Hornik, "Convergence of learning algorithms with constant learning rates," IEEE Trans. Neural Networks, vol. 2, pp. 484-489, 1991.
20. B. W. Silverman, Density Estimation for Statistics and Data Analysis. London: Chapman and Hall, 1986.
21. H. Ritter and K. Schulten, "On the stationary state of Kohonen's self-organizing sensory mapping," Biol. Cybern., vol. 54, pp. 99-106, 1986.
22. R. Der and M. Herrmann, "Instabilities in self-organized feature maps with short neighborhood range," in Proc. European Symp. Artificial Neural Networks - ESANN'94, Brussels, Belgium, 1994, pp. 271-276.