Entropy penalized automated model selection on Gaussian mixture

International Journal of Pattern Recognition and Artificial Intelligence

Volumn 18, Issue 8, 2004, Pages 1501-1512

  Ma, Jinwen ^a   Wang, Taijun ^b  

^a PEKING UNIVERSITY   (China)

^b SOUTHEAST UNIVERSITY   (China)

Author keywords

Gaussian mixture; Maximum likelihood estimation; Model selection; Penalty; Shannon entropy

Indexed keywords

COMPUTER SIMULATION; DATA REDUCTION; MATHEMATICAL MODELS; PARAMETER ESTIMATION; PROBLEM SOLVING; VECTORS;

GAUSSIAN MIXTURE; MAXIMUM-LIKELIHOOD ESTIMATION; MODEL SELECTION; PENALTY; SHANNON ENTROPY;

LEARNING ALGORITHMS;

EID: 11244308002     PISSN: 02180014     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0218001404003812     Document Type: Article

References (28)

1. S. C. Ahalt, A. K. Krishnamurty, P. Chen and D. E. Melton, Competitive learning algorithm for vector quantization, Neural Networks 3(3) (1990) 277-291.
2. H. Akaike, A new look at the statistical model identification, IEEE Trans. Autom. Contr. AC-19 (1974) 716-723.
3. J. C. Bezdek and N. R. Pal, Some nex indexes of cluster validity, IEEE Trans. Syst. Man Cybern. B28(3) (1998) 301-315.
4. H. Bozdogan, Multiple Sample Cluster Analysis and Approaches to Validity Studies in Clustering Individuals, Doctoral dissertation, University of Illinois at Chicago Circle (1981).
5. H. Bozdogan, Model selection and Akaike's information criterion: the general theory and its analytical extensions, Psychometrika 52(3) (1987) 345-370.
6. P. A. Devijver and J. Kittler, Pattern Recognition: A Statistical Approach (Prentice-Hall, London, UK, 1982).
7. S. R. Gerald, Matrix Derivatives, Lecture Notes in Statistics, Vol. 2 (Marcel Dekker, NY, 1980).
8. J. A. Hartigan, Distribution problems in clustering, in Classification and Clustering, ed. J. Van Ryzin (Academic Press, NY, 1977), pp. 45-72.
9. A. K. Jain and R. C. Dubes, Algorithm for Clustering Data (Englewood Cliffs, Prentice Hall, NJ, 1988).
10. J. Ma, T. Wang and L. Xu, Convergence analysis of rival penalized competitive learning (RPCL) algorithm, in Proc. Int. Joint Conf. Neural Networks (IJCNN'02), Honolulu, Hawaii, May 12-17, 2002, Vol. 2, pp. 1596-1601.
11. J. Ma, T. Wang and L. Xu, An adaptive BYY harmony learning algorithm and its relation to rewarding and penalizing competitive learning mechanism, in Proc. 6th Int. Conf. Signal Processing (ICSP '02), Beijing, China, August 26-30, 2002, Vol. 2, pp. 1154-1158.
12. J. Ma, T. Wang and L. Xu, An iterative algorithm for BYY learning on Gaussian mixture with automated model selection, in Proc. 2003 Int. Conf. Neural Networks and Signal Processing (ICNNSP'OS), Nanjing, China, December 14-17, 2003, Vol. 1, pp. 7-10.
13. J. Ma, T. Wang and L. Xu, An annealing approach to BYY harmony learning on Gaussian mixture with automated model selection, in Proc. 2003 Int. Conf. Neural Networks and Signal Processing (ICNNSP'03), Nanjing, China, December 14-17, 2003, Vol. 1, pp. 23-28.
14. J. Mao and A. K. Jain, A self-organizing network for hyperellipsoidal clustering (HEC), IEEE Trans. Neural Networks 7(1) (1996) 16-29.
15. G. J. Mclachlan and K. E. Basford, Mixture Models: Inference and Applications to Clustering (Marcel Dekker, NY, 1988).
16. R. M. Ramze, B. P. F. Leliieveldt and J. H. C. Reiber, A new cluster validity index for the fuzzy c-means, Patt. Recogn. Lett. 19 (1998) 273-246.
17. R. A. Render and H. F. Walker, Mixture densities, maximum likelihood and the EM algorithm, SIAM Rev. 26(2) (1984) 195-239.
18. D. E. Rumelhart and D. Zipser, Feature discovery by competitive learning, Cogn. Sci. 9(1) (1985) 75-112.
19. G. Scharz, Estimating the dimension of a model, Ann. Statist. 6(2) (1978) 461-464.
20. J. Ma, T. Wang and L. Xu, A gradient BYY harmony learning rule on Gaussian mixture with automated model selection, Neurocomputing 56 (2004) 481-487.
21. X. L. Xie and G. A. Beni, A validity method for fuzzy clustering, IEEE Trans. Patt. Anal. Mach. Intell. 13(8) (1991) 843-847.
22. L. Xu, A. Krzyzak and E. Oja, Rival penalized competitive learning for clustering analysis, RBF net, and curve detection, IEEE Trans. Neural Networks 4 (1993) 636-648.
23. L. Xu et al., A unified learning scheme: Bayesian-Kullback YING-YANG machine, in Advances in Neural Information Processing Systems, eds. D. S. Touretzky et al. (MIT Press, Cambridge, MA, 1996), Vol. 8, pp. 444-450; A part of its preliminary version on Proc. 1995 Int. Conf. Neural Information Processing, pp. 977-988.
24. L. Xu et al., A unified learning scheme: Bayesian-Kullback YING-YANG machine, in Advances in Neural Information Processing Systems, eds. D. S. Touretzky et al. (MIT Press, Cambridge, MA, 1996), Vol. 8, pp. 444-450; A part of its preliminary version on Proc. 1995 Int. Conf. Neural Information Processing, pp. 977-988.
25. L. Xu, Rival penalized competitive learning, finite mixture, and multisets clustering, in Proc. 1998 IEEE Int. Joint Conf. Neural Networks, Vol. 3, pp. 251-253.
26. L. Xu, Best harmony, unified RPCL and automated model selection for unsupervised and supervised learning on Gaussian mixtures, three-layer nets and ME-RBF-SVM models, Int. J. Neural Syst. 11(1) (2002) 43-69.
27. L. Xu, BYY harmony learning, structural RPCL, and topological self-organzing on mixture modes, Neural Networks 15 (2002) 1231-1237.
28. L. Xu, Ying-Yang learning, in The Handbook of Brain Theory and Neural Networks, ed. M. A. Arbib, 2nd edn. (The MIT Press, 2002), pp. 1231-1237.