Entropy-based incremental variational bayes learning of gaussian mixtures

IEEE Transactions on Neural Networks and Learning Systems

Volumn 23, Issue 3, 2012, Pages 534-540

  Penalver, Antonio ^a   Escolano, Francisco ^b  

^a MIGUEL HERNÁNDEZ UNIVERSITY   (Spain)

^b UNIVERSITY OF ALICANTE   (Spain)

Author keywords

Clustering; entropy estimation; mixture models; model order selection; variational Bayes methods

Indexed keywords

CLUSTERING; ENTROPY ESTIMATION; MIXTURE MODEL; MODEL-ORDER SELECTION; VARIATIONAL-BAYES METHOD;

ITERATIVE METHODS; OPTIMIZATION; PATTERN RECOGNITION;

ENTROPY;

EID: 84876936071     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2011.2177670     Document Type: Article

References (25)

1. A. Jain, R. Dubes, and J. Mao, "Statistical pattern recognition: A review," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 1, pp. 4-38, Jan. 2000.
2. D. Titterington, A. Smith, and U. Makov, Statistical Analysis of Finite Mixture Distributions. Chichester, U.K.: Wiley, 2002.
3. G. Box and G. Tiao, Bayesian Inference in Statistical Models. Reading, MA: Addison-Wesley, 1992.
4. A. Dempster, N. Laird, and D. Rubin, "Maximum likelihood estimation from incomplete data via the EM algorithm," J. Royal Stat. Soc., vol. 39, no. 1, pp. 1-38, 1977.
5. D. Husmeier, "The Bayesian evidence scheme for regularizing probability-density estimating neural networks," Neural Comput., vol. 12, no. 11, pp. 2685-2717, 2000.
6. D. MacKay, Introduction to Monte Carlo Methods, M. I. Jordan, Ed. Cambridge, MA: MIT Press, 1999.
7. Z. Ghahramani and M. J. Beal, "Variational inference for Bayesian mixtures of factor analysers," in Advances in Neural Information Processing Systems 12. Cambridge, MA: MIT Press, 2000, pp. 449-455.
8. N. Nasios and A. Bors, "Variational learning for Gaussian mixture models," IEEE Trans. Syst., Man, Cybern. Part B: Cybern., vol. 36, no. 4, pp. 849-862, Aug. 2006.
9. N. Nasios and A. Bors, "Blind source separation using variational expectation-maximization algorithm," in Proc. Int. Conf. Comput. Anal. Images Patterns, vol. 2756. 2003, pp. 442-450.
10. M. A. T Figueiredo, J. M. N Leitao, and A. K Jain, "On fitting mixture models," in Energy Minimization Methods in Computer Vision and Pattern Recognition (Lecture Notes in Computer Science), vol. 1654. New York: Springer-Verlag, 1999, pp. 54-69.
11. M. A. T. Figueiredo and A. K. Jain, "Unsupervised selection and estimation of finite mixture models," in Proc. Int. Conf. Pattern Recognit., 2000, pp. 87-90.
12. M. Figueiredo and A. Jain, "Unsupervised learning of finite mixture models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 3, pp. 381-399, Mar. 2002.
13. A. Peñalver, F. Escolano, and J. M. Sáez, "Learning Gaussian mixture models with entropy-based criteria," IEEE Trans. Neural Netw., vol. 20, no. 11, pp. 1756-1772, Nov. 2009.
14. A. Corduneau and C. Bishop, Variational Bayesian Model Selection for Mixture Distributions. San Mateo, CA: Morgan Kaufmann, 2001.
15. C. Constantinopoulos and A. Likas, "Unsupervised learning of Gaussian mixtures based on variational component splitting," IEEE Trans. Neural Netw., vol. 18, no. 3, pp. 745-755, May 2007.
16. A. Peñalver, F. Escolano, and B. Bonev, "Entropy-based variational scheme for fast Bayes learning of Gaussian mixtures," in Proc. Joint IAPR Int. Conf. Struct., Synt., Stat. Pattern Recognit., 2010, pp. 100-108.
17. H. Attias, "Inferring parameters and structure of latent variable models by variational Bayes," in Proc. Uncertain. Artif. Intell., 1999, pp. 21-30.
18. G. Arfken and H. Weber, Mathematical Methods for Physicists. New York: Elsevier, 2005.
19. A. Hero and O. J. J. Michel, "Estimation of Rényi information divergence via pruned minimal spanning trees," in Proc. IEEE Workshop Higher Order Stat., Caesarea. Israel, Jun. 1999, pp. 264-268.
20. N. Leonenko and L. Pronzato, "A class of Rényi information estimators for multi-dimensional densities," Ann. Stat., vol. 36, no. 5, pp. 2153-2182, 2008.
21. A. G. Bors and M. Gabbouj, "Minimal topology for a radial basis functions neural network for pattern classification," Digital Signal Process., vol. 4, no. 3, pp. 173-188, 1994.
22. S. Richardson and P. Green, "On Bayesian analysis of mixtures with unknown number of components (with discussion)," J. Royal Stat. Soc. B, vol. 59, no. 1, pp. 731-792, 1997.
23. Z. Zhang, K. Chan, Y. Wu, and C. Chen, "Learning a multivariate Gaussian mixture models with the reversible jump MCMC algorithm," Stat. Comput., vol. 14, no. 1, pp. 343-355, 2004.
24. P. Dellaportas and I. Papageorgiou, "Multivariate mixtures of normals with unknown number of components," Stat. Comput., vol. 16, no. 1, pp. 57-68, 2006.
25. C. Blake and C. Merz. (1998). UCI Repository of Machine Learning Databases. Dept. Inf. Comput. Sci., Univ. California, Irvine [Online]. Available: http://archive.ics.uci.edu/ml/