BYY harmony learning on finite mixture: Adaptive gradient implementation and a floating RPCL mechanism

Neural Processing Letters

Volumn 24, Issue 1, 2006, Pages 19-40

  Ma, Jinwen ^a   Wang, Le ^a  

^a PEKING UNIVERSITY   (China)

Author keywords

Bayesian Ying Yang system; Clustering analysis; Finite mixture; Harmony learning; Rival penalized competitive learning

Indexed keywords

ADAPTIVE SYSTEMS; ALGORITHMS; COMPUTER SIMULATION; MATHEMATICAL MODELS; PARAMETER ESTIMATION; PROBLEM SOLVING;

BAYESIAN YING-YANG SYSTEM; CLUSTERING ANALYSIS; FINITE MIXTURE; HARMONY LEARNING; RIVAL PENALIZED COMPETITIVE LEARNING;

LEARNING SYSTEMS;

EID: 33748846576     PISSN: 13704621     EISSN: 1573773X     Source Type: Journal    
DOI: 10.1007/s11063-006-9008-7     Document Type: Article

References (29)

1. Mclachlan, G. J. and Basford, K. E.: Mixture Models: Inference and Applications to Clustering, Marcel Dekker, New York, 1988.
2. Devijver, P. A. and Kittter, J.: Pattern Recognition: A Statistical Approach, Prentice Hall, Englewood Cliffs, NJ, 1982.
3. Render, R. A. and Walker, H. F.: Mixture densities, maximum, likelihood and the EM algorithm, SIAM Review 26(2) (1984), 195-239.
4. Hartigan, J. A.: Distribution problems in clustering, In: J. Van Ryzin (ed), Classification and Clustering, pp. 45-72, Academic press, New York, 1977.
5. Akaike, H.: A new look at the statistical model identification, IEEE Trans. on Automatic Control AC-19 (1974), 716-723.
6. Scharz, G.: Estimating the dimension of a model, The Annals of Statistics, 6 (1978), 461-464.
7. Bozdogan, H: Model selection and Akaike's information criterion: the general theory and its analytical extensions, Psychometrika 52 (1987), 345-370.
8. Ball, G. H. and Hall, D. J.: ISODATA: a novel method of data analysis and pattern classification, Technique Report No. 699616, 1965, Stanford Research International.
9. Makhoul, J., Rpucos, S. and Gish, H.: Vector quantization in speech coding, Proceedings of IEEE 73 (1985), 1551-1558.
10. Rumelhart, D. E. and Zipser, D.: Feature discovery by competitive learning, Cognitive Sciences 9 (1985), 75-112.
11. Ahalt, S. C., Krishnamurty, A. K., Chen, P. and Melton, D. E.: Competitive learning algorithm for vector quantization, Neural Networks 3 (1990), 277-291.
12. Xu, L., Krzyzak, A. and Oja, E.: Rival penalized competitive learning for clustering analysis, RBF net, and curve detection, IEEE Transactions on Neural networks 4 (1993), 636-648.
13. Xu, L.: Rival penalized competitive learning, finite mixture, and multisets clustering, In: Proceedings of 1998 IEEE International Joint Conference On Neural Networks 3 (1998), 251-2530.
14. Xu, L.: Ying-Yang machine: a Bayesian-Kullback scheme for unified learnings and new results on vector quantization, In: Proceedings of the 1995 International Conference on Neural Information Processing (ICONIP'95) 2 (1995), 977-988.
15. Xu, L.: Best harmony, unified RPCL and automated model selection for unsupervised and supervised learning on Gaussian mixtures, three-layer nets and ME-RBF-SVM models, International Journal of Neural Systems 11 (2001), 43-69.
16. Xu, L.: Ying-Yang learning, In: M. A. Arbib (2nd ed.), The Handbook of Brain Theory and Neural Networks, pp. 1231-1237. The MIT Press, Cambridge, MA, 2002.
17. Xu, L.: BYY harmony learning, structural RPCL, and topological self-organizing on mixture modes, Neural Networks 15 (2002), 1231-1237.
18. Xu, L.: Bayesian-Kullback YING-YANG learning scheme: reviews and new results, In: Proceedings of the 1996 International Conference on Neural Information Processing (ICONIP'96) 1 (1996) 59-67.
19. Hu, X. and Xu, L.: A comparative study of several cluster number selection criteria, In: Lecture Notes of Computer Science, vol. 2690 pp. 195-202, 2003.
20. Liu, Z. and Xu, L.: Smoothed local PCA by BYY data smoothing learning, In: Proceedings of the International Conference on Control, Automation, and System (ICCAS'01), 621-622 (2001).
21. Ma, J., Wang, T. and Xu, L.: An annealing approach to BYY learning on Gaussian mixture with automated model selection, In: Proceedings of 2003 International Conference on Neural Networks and Signal Processing (ICNN&SP'03) 1 (2003), 23-28.
22. Ma, T., Wang, T., and Xu, L.: A gradient BYY harmony learning rule on Gaussian mixture with automated model selection, Neurocomputing 56 (2004), 481-487.
23. Ma, J., Gao, B., Wang, Y., and Cheng Q.: Conjugate and natural gradient rules for BYY harmony learning on Gaussian mixture with automated model selection, International Journal of Pattern Recognition and Artificial Intelligence 19 (2005), 701-713.
24. Robbins, H. and Monro, S.: A stochastic approximation method, The Annals of Mathematical Statistics 22 (1951), 400-407.
25. Robert, S. J., Everson, R., and Rezek, I.: Maximum certainty data partitioning, Pattern Recognition 33 (2000), 833-839.
26. Ueda, N. and Ghahramani, Z.: Bayesian model search for mixture models based on optimizing variational bounds, Neural Network 15 (2002), 1223-1241.
27. Blake, C. L. and Merz, C. J.: UCI repository of machine learning databases University of California, Irvine, Department of Information and Computer Science, 1998, http://www.ics.uci.edu/~mlearn/MLRepository.html.
28. Loo, C. K., and Rao, M. V. C.: Accurate and reliable diagnosis and classification using probablistic ensemble simplified fuzzy ARTMAP, IEEE Transactions on Knowledge and Data engineering 17 (2005), 1589-1593.
29. Boujeman, N.: Generalized competitive clustering for image segmentation, In: Proceedings of 19th International Conference of the North American Fuzzy Information Processing Society, 133-137 (2000).