Unsupervised learning of Gaussian mixtures based on variational component splitting

IEEE Transactions on Neural Networks

Volumn 18, Issue 3, 2007, Pages 745-755

  Constantinopoulos, Constantinos ^a   Likas, Aristidis ^a  

^a UNIVERSITY OF IOANNINA   (Greece)

Author keywords

Clustering; Mixture models; Model selection; Variational Bayes methods

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE; CLUSTER ANALYSIS; ITERATIVE METHODS; MATHEMATICAL MODELS; STATISTICAL MECHANICS; VARIATIONAL TECHNIQUES;

MIXTURE MODELS; MODEL SELECTION; VARIATIONAL BAYES METHODS; VARIATIONAL COMPONENT SPLITTING;

LEARNING SYSTEMS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; BAYES THEOREM; COMPUTER SIMULATION; DECISION SUPPORT SYSTEM; INFORMATION RETRIEVAL; METHODOLOGY; NORMAL DISTRIBUTION; THEORETICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; BAYES THEOREM; COMPUTER SIMULATION; DECISION SUPPORT TECHNIQUES; INFORMATION STORAGE AND RETRIEVAL; MODELS, THEORETICAL; NEURAL NETWORKS (COMPUTER); NORMAL DISTRIBUTION; PATTERN RECOGNITION, AUTOMATED;

EID: 34248648503     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2006.891114     Document Type: Article

References (28)

1. G. McLachlan and D. Peel, Finite Mixture Models. New York:WIley, 2000.
2. H. Hajji, "Statistical analysis of network traffic for adaptive faults detection," IEEE Trans. Neural Netw., vol. 16, no. 5, pp. 1053-1063, Sep. 2005.
3. L. A. G. N. P. Blekas, K. and I. E. Lagaris, "A spatially constrained mixture model for image segmentation," IEEE Trans. Neural Netw., vol. 16, no. 2, pp. 494-498, Mar. 2005.
4. A. Dempster,N. Laird, and D. Rubin, "Maximum likelihood estimation from incomplete data via the EM algorithm," J. Roy. Statist. Soc., Ser. B vol. 39, no. 1, pp. 1-38, 1977.
5. G. McLachlan and T. Krishnan, The EM Algorithm and Extensions. New York: Wiley, 1997.
6. P. Smyth, "Model selection for probabilistic clustering using cross- validated likelihood," Statist. Comput., vol. 10, no. 1, pp. 63-72, 2000.
7. T. Lange, M. L. Braun, V. Roth, and J. M. Buhmann, "Stability-based model selection," in Advances in Neural Information Processing Systems 15. Cambridge, MA: MIT Press, 2002.
8. M. Hareven and V. L. Brailovsky, "Probabilistic validation approach for clustering," Pattern Recognit. Lett., vol. 16, pp. 1189-1196, 1995.
9. M. A. T. Figueiredo and A. K. Jain, "Unsupervised learning of finite mixture models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 3, pp. 381-396, Mar. 2002.
10. Y. M. Cheung, "Maximum weighted likelihood via rival penalized EM for density mixture clustering with automatic model selection," IEEE Trans. Knowl. Data Eng., vol. 17, no. 6, pp. 750-761, Jun. 2005.
11. S. Richardson and P. Green, "On Bayesian analysis of mixtures with unknown number of components," J. Roy. Statist. Soc.i], Ser. B, vol. 59, no. 4, pp. 731-792, 1997.
12. T. Jaakkola, "Tutorial on variational approximation methods," in Advanced Mean Field Methods: Theory and Practice, M. Opper and D. Saad, Eds. Cambridge, MA: MIT Press, 2000, pp. 129-160.
13. Z. Ghahramani and M. Beal, "Variational inference for Bayesian mixtures of factor analysers," in Advances in Neural Information Processing Systems 12. Cambridge, MA: MIT Press, 2000, pp. 449-455.
14. Z. Ghahramani and M. Beal, "Propagation algorithms for variational Bayesian learning," in Advances in Neural Information Processing Systems 13. Cambridge, MA: MIT Press, 2001, pp. 507-513.
15. R. Neal and G. Hinton, "A view of the EM algorithm that justifies incremental, sparse, and other variants," in Learning in Graphical Models, M. I. Jordan, Ed. Norwell, MA: Kluwer, 1998, pp. 355-370.
16. M. Sato, "On-line model selection based on the variational Bayes," Neural Comput., vol. 13, no. 7, pp. 1649-1681, 2001.
17. H. Attias, "Inferring parameters and structure of latent variable models by variational Bayes," in Proc. 15th Conf. Uncertainty Artif. Intel., 1999, pp. 21-30.
18. H. Attias, "A variational Bayesian framework for graphical models," in Advances in Neural Information Processing Systems 12. Cambridge, MA: MIT Press, 2000, pp. 209-215.
19. A. Corduneanu and C. M. Bishop, "Variational Bayesian model selection for mixture distributions," in Artificial Intelligence and Statistics 2001. San Mateo, CA: Morgan Kaufmann, 2001, pp. 27-34.
20. N. Ueda and Z. Ghahramani, "Bayesian model search for mixture models based on optimizing variational bounds," Neural Netw., vol. 15, no. 10, pp. 1223-1241, 2002.
21. N. Ueda, R. Nakano, Z. Ghahramani, and G. E. Hinton, "SMEM algorithm for mixture models," Neural Comput., vol. 12, no. 9, pp. 2109-2128, 2000.
22. S. Kotz, N. Balakrishnan, and N. L. Johnson, Continuous multivariate distributions, 2nd ed. New York: Wiley, 2000, vol. 1.
23. G. B. Arfken and H. J. Weber, Mathematical Methods For Physicists, 6th ed. New York: Elsevier, 2005.
24. K. Fukumizu, S. Akaho, and S. Amari, "Critical lines in symmetry of mixture models and its application to component splitting," in Advances in Neural Information Processing Systems 15. Cambridge, MA: MIT Press, 2003, pp. 865-872.
25. Z. Zhang, C. Chen, J. Sun, and K. L. Chan, "EM algorithms for Gaussian mixtures with split-and-merge operation," Pattern Recognit., vol. 36, no. 9, pp. 1973-1983, 2003.
26. S. Dasgupta, "Learning mixtures of Gaussians," in Proc. 40th Symp. Found. Comput. Sci., 1999, pp. 634-644.
27. J. J. Hull, "A database for handwritten text recognition research," IEEE Trans. Pattern Anal. Mach. Intell., vol. 16, no. 5, pp. 550-554, May 1994.
28. C. Constantinopoulos and A. Likas, "An incremental training method for the probabilistic RBF network," IEEE Trans. Neural Netw., vol. 17, no. 4, pp. 966-974, Jul. 2006.