A BYY scale-incremental EM algorithm for Gaussian mixture learning

Applied Mathematics and Computation

Volumn 205, Issue 2, 2008, Pages 832-840

  Li, Lei ^a   Ma, Jinwen ^a  

^a LMAM   (China)

Author keywords

Bayesian Ying Yang (BYY) harmony learning; EM algorithm; Gaussian mixture; Model selection; Unsupervised image segmentation

Indexed keywords

ALGORITHMS; BAYESIAN NETWORKS; BOOLEAN FUNCTIONS; DATA PROCESSING; DIGITAL IMAGE STORAGE; GAUSSIAN DISTRIBUTION; IMAGE PROCESSING; IMAGE SEGMENTATION; LEARNING ALGORITHMS; METAL ANALYSIS; MIXTURES; MODAL ANALYSIS; MODEL STRUCTURES; PARAMETER ESTIMATION; PROBABILITY DENSITY FUNCTION; SET THEORY; STATISTICAL METHODS; TRELLIS CODES;

BAYESIAN YING-YANG (BYY) HARMONY LEARNING; EM ALGORITHM; GAUSSIAN MIXTURE; MODEL SELECTION; UNSUPERVISED IMAGE SEGMENTATION;

MATHEMATICAL MODELS;

EID: 54249141403     PISSN: 00963003     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.amc.2008.05.076     Document Type: Article

References (34)

1. Dempster A.P., Laird N.M., and Rubin D.B. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B 39 (1977) 1-38
2. Mao J., and Jain A.K. A self-organizing network for hyperellipsoidal clustering. IEEE Transactions on Neural Networks 7 1 (1996) 16-29
3. Render R.A., and Walker H.F. Mixture densities, maximum likelihood and the EM algorithm. SIAM Review 26 2 (1984) 195-239
4. Xu L., and Jordan M.I. On convergence properties of the EM algorithm for Gaussian mixtures. Neural Computation 8 (1996) 129-151
5. Ma J., Xu L., and Jordan M.I. Asymptotic convergence rate of the EM algorithm for Gaussian mixtures. Neural Computation 12 12 (2000) 2881-2907
6. Ma J., and Xu L. Asymptotic convergence properties of the EM algorithm with respect to the overlap in the mixture. Neurocomputing 68 (2005) 105-129
7. Ma J., and Fu S. On the correct convergence of the EM algorithm for Gaussian mixtures. Pattern Recognition l38 12 (2005) 2602-2611
8. Hartigan J.A. Distribution problems in clustering. In: Van Ryzin J. (Ed). Classification and Clustering (1977), Academic Press 45-72
9. Millgan G.W., and Copper M.C. An examination of procedures for determining the number of clusters in a data set. Psychometrika 46 (1985) 187-199
10. Akaike H. A new look at the statistical model identification. IEEE Transactions on Automatic Control AC-19 (1974) 716-723
11. Bozdogan H. Model selection and Akaike's information criterion: the general theory and its analytical extensions. Psychometrika 52 (1987) 345-370
12. Scharz G. Estimating the dimension of a model. The Annals of Statistics 6 2 (1978) 461-464
13. Rissanen J. Stochastic Complexity in Statistical Inquiry (1989), World Scientific, Singapore
14. J. Oliver, R. Baxterand, C. Wallace, Unsupervised learning using MML, in: Proceedings of the 13th International Conference on Machine Learning, 1996, pp. 364-372.
15. Escobar M.D., and West M. Bayesian density estimation and inference using mixtures. Journal of the American Statistical Association 90 430 (1995) 577-588
16. Peter J.G. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82 4 (1995) 711-732
17. Roberts S.J., et al. Bayesian approaches to Gaussian mixture modelling. IEEE Transactions on Pattern Analysis and Machine Intelligence 20 11 (1998) 1133-1142
18. Ueda N., and Ghahramani Z. Bayesian model search for mixture models based on optimizing variational bounds. Neural Networks 15 (2002) 1223-1241
19. L. Xu, Ying-Yang machine: a Bayesian-Kullback scheme for unified learnings and new results on vector quantization, in: Proceedings of the 1995 International Conference on Neural Information Processing (ICONIP'95), vol. 2, 1995, pp. 977-988.
20. Xu L. Bayesian Ying-Yang machine, clustering and number of clusters. Pattern Recognition Letters 18 (1997) 1167-1178
21. Xu L. Best harmony, unified RPCL and automated model selection for unsupervised and supervised learning on Gaussian mixtures, three-layer nets and ME-RBF-SVM models. International Journal of Neural Systems 11 1 (2001) 43-69
22. Xu L. BYY harmony learning, structural RPCL and topological self-organizing on mixture modes. Neural Networks 15 8-9 (2002) 1231-1237
23. Ma J., Wang T., and Xu L. A gradient BYY harmony learning rule on Gaussian mixture with automated model selection. Neurocomputing 56 (2004) 481-487
24. Ma J., et al. Conjugate and natural gradient rules for BYY harmony learning on Gaussian mixture with automated model selection. International Journal of Pattern Recognition and Artificial Intelligence 19 5 (2005) 701-713
25. Ma J., and Wang L. BYY harmony learning on finite mixture: adaptive gradient implementation and a floating RPCL mechanism. Neural Processing Letters 24 1 (2006) 19-40
26. Ma J., and He X. A fast fixed-point BYY harmony learning algorithm on Gaussian mixture with automated model selection. Pattern Recognition letters 29 (2008) 701-711
27. J. Ma, Automated model selection (AMS) on finite mixtures: a theoretical analysis, in: Proceedings of the 2006 International Joint Conference on Neural Networks (IJCNN06), 2006, pp. 8255-8261.
28. Ma J., and Liu J. The BYY annealing learning algorithm for Gaussian mixture with automated model selection. Pattern Recognition 40 (2007) 2029-2037
29. Ma J., and Wang T. Entropy penalized automated model selection on Gaussian mixture. International Journal of Pattern Recognition and Artificial Intelligence 18 8 (2004) 1501-1512
30. Vlassis N., and Likas A. A greedy EM algorithm for Gaussian mixture learning. Neural Processing Letters 15 (2002) 77-87
31. Verbeek J.J., Vlassis N., and Krose B. Efficient greedy learning of Gaussian mixture models. Neural Computation 15 2 (2003) 469-485
32. Zhang Z., et al. EM algorithms for Gaussian mixtures with split-and-merge operation. Pattern Recognition 36 (2003) 1973-1983
33. Robert S.J., Everson R., and Rezek I. Maximum certainty data partitioning. Pattern Recognition 33 (2000) 833-839
34. N. Boujeman, Generalized competitive clustering for image segmentation, in: Proceedings of the 19th International Conference of the North American Fuzzy Information Processing Society, 2000, pp. 133-137.