Entropy-based kernel mixture modeling for topographic map formation

IEEE Transactions on Neural Networks

Volumn 15, Issue 4, 2004, Pages 850-858

  Van Hulle, Marc M ^a  

^a KU LEUVEN UNIVERSITY OF LEUVEN   (Belgium)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL COMPLEXITY; INFORMATION THEORY; LEARNING ALGORITHMS; MATHEMATICAL MODELS; PROBABILITY DENSITY FUNCTION; PROBABILITY DISTRIBUTIONS;

ENTROPY ESTIMATION; INFORMATION THEORETIC LEARNING; KERNEL BASED TOPOGRAPHIC MAP FORMATION; NONDIFFERENTIABLE KERNELS;

LEARNING SYSTEMS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPARATIVE STUDY; COMPUTER SIMULATION; DECISION SUPPORT SYSTEM; ENTROPY; EVALUATION; INFORMATION RETRIEVAL; INFORMATION SCIENCE; LEARNING; METHODOLOGY; STATISTICAL MODEL; VALIDATION STUDY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; DECISION SUPPORT TECHNIQUES; ENTROPY; INFORMATION STORAGE AND RETRIEVAL; INFORMATION THEORY; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; PROBABILITY LEARNING;

EID: 3843136241     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2004.828763     Document Type: Article

References (28)

1. T. Kohonen, "Self-organized formation of topologically correct feature maps," Biol. Cybern., vol. 43, pp. 59-69, 1982.
2. T. Kohonen, Self-Organizing Maps. Berlin, Germany: Springer-Verlag, 1995.
3. S. P. Luttrell, "Code vector density in topographic mappings: scalar case," IEEE Trans. Neural Networks, vol. 2, pp. 427-436, July 1991.
4. T. M. Heskes and B. Kappen, "Error potentials for self-organization," in Proc. IEEE Int. Conf. Neural Networks, vol. 3, San Francisco, 1993, pp. 1219-1223.
5. M. M. Van Hulle, "Joint entropy maximization in kernel-based topographic maps," Neural Computat., vol. 14, no. 8, pp. 1887-1906, 2002.
6. T. Graepel, M. Burger, and K. Obermayer, "Self-organizing maps: generalizations and new optimization techniques," Neurocomputing, vol. 21, pp. 173-190, 1998.
7. T. Heskes, "Self-organizing maps, vector quantization, and mixture modeling," IEEE Trans. Neural Networks, vol. 12, pp. 1299-1305, Nov. 2001.
8. V. N. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
9. R. Linsker, "How to generate ordered maps by maximizing the mutual information between input and output signals," Neural Computat., vol. 1, pp. 402-411, 1989.
10. M. Benaim and L. Tomasini, "Competitive and self-organizing algorithms based on the minimization of an information criterion," in Proc. ICANN'91, 1991, pp. 391-396.
11. H. Yin and N. M. Allinson, "Self-organizing mixture networks for probability density estimation," IEEE Trans. Neural Networks, vol. 12, pp. 405-411, Mar. 2001.
12. A. J. Bell and T. J. Sejnowski, "An information-maximization approach to blind separation and blind deconvolution," Neural Computat., vol. 7, pp. 1129-1159, 1995.
13. S. Fiori, "Notes on Bell-Sejnowski pdf-matching neuron," Neural Comput., vol. 14, pp. 2847-2855, 2002.
14. C. E. Shannon, "A mathematical theory of communication (concluded ," Bell System Tech. J., vol. 27, pp. 629-631, 1948.
15. C. E. Shannon, "A mathematical theory of communication," Bell System Tech. J., vol. 27, pp. 379-423, 1948.
16. J. Beirlant, E. J. Dudewicz, L. Györfi, and E. C. van der Meulen, "Non-parametric entropy estimation: an overview," Int. J. Math. Statist. Sci., vol. 6, pp. 17-39, 1997.
17. P. Hall, "Limit theorems for sums of general functions of m-spacings," Math. Proc. Camb. Phil. Soc., vol. 96, pp. 517-532, 1984.
18. J. C. Principe and D. Xu, "Information-theoretic learning using Renyi's quadratic entropy," in Proc. 1st Int. Workshop Independent Component Analysis and Signal Separation, Aussois, France, Jan. 1999, pp. 407-412.
19. D. Erdogmus, J. C. Principe, and K. E. Hild II, "Do Hebbian synapses estimate entropy?," in Proc. 2002 IEEE Workshop Neural Networks for Signal Processing, Martigny, Valais, Switzerland, Sept. 2002, pp. 199-208.
20. Y. Shao and M. G. Hahn, "Limit theorems for the logarithm of sample spacings," Statist. Probability Lett., vol. 24, pp. 121-132, 1995.
21. D. P. Mandic and J. A. Chambers, Recurrent Neural Networks for Prediction: Architectures, Learning Algorithms and Stability. New York: Wiley, 2001.
22. J. Beirlant and M. C. A. Van Zuijlen, "The empirical distribution function and strong laws for functions of order statistics of uniform spacings," J. Multivariate Anal., vol. 16, pp. 300-317, 1985.
23. E. G. Miller and J. W. Fisher III, "ICA using spacings estimates of entropy," in Proc. 4th Int. Symp. Independent Component Analysis and Blind Signal Separation (ICA 2003), Nara, Japan, Apr. 2003, pp. 1047-1052,
24. K. Rose, E. Gurewitz, and G. C. Fox, "Statistical mechanics and phase transitions in clustering," Phys. Rev. Lett., vol. 65, no. 8, pp. 945-948, 1990.
25. P. J. Bickel and L. Breiman, "Sums of functions of nearest neighbor distances, moment bounds, limit theorems and a goodness of fit test," Ann. Statist., vol. 11, pp. 185-214, 1983.
26. L. F. Kozachenko and N. N. Leonenko, "Sample estimate of entropy of a random vector," Probl. Inform. Transmission, vol. 23, pp. 95-101, 1987.
27. P. P. B. Eggermont and V. N. LaRiccia, "Best asymptotic normality of the kernel density entropy estimator for smooth densities," IEEE Trans. Inform. Theory, vol. 45, pp. 1321-1326, Apr. 1999.
28. A. Rényi, Probability Theory. Amsterdam, The Netherlands: North-Holland, 1970.