Variational Bayesian mixture model on a subspace of exponential family distributions

IEEE Transactions on Neural Networks

Volumn 20, Issue 11, 2009, Pages 1783-1796

  Watanabe, Kazuho ^a   Akaho, Shotaro ^b   Omachi, Shinichiro ^c   Okada, Masato ^d  

^a NARA INSTITUTE OF SCIENCE AND TECHNOLOGY   (Japan)

^b NATIONAL INSTITUTE OF ADVANCED INDUSTRIAL SCIENCE AND TECHNOLOGY AIST   (Japan)

^c TOHOKU UNIVERSITY   (Japan)

^d UNIVERSITY OF TOKYO   (Japan)

Author keywords

Exponential family; Exponential principal component analysis (e PCA); Mixture model; Variational Bayes (VB) method.

Indexed keywords

ALGORITHMS; APPROXIMATION ALGORITHMS; CLUSTERING ALGORITHMS; DATA MINING; DATA VISUALIZATION; MIXTURES; PROBABILITY DISTRIBUTIONS;

APPROXIMATION TECHNIQUES; DIMENSIONALITY REDUCTION; EXPONENTIAL FAMILY; EXPONENTIAL FAMILY DISTRIBUTIONS; EXPONENTIAL PRINCIPAL COMPONENT ANALYSIS (E-PCA); LATENT VARIABLE MODELING; MIXTURE MODEL; VARIATIONAL BAYES;

PRINCIPAL COMPONENT ANALYSIS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; BAYES THEOREM; COMPUTER SIMULATION; MATHEMATICS; PRINCIPAL COMPONENT ANALYSIS; STATISTICAL ANALYSIS; THEORETICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; BAYES THEOREM; COMPUTER SIMULATION; DATA INTERPRETATION, STATISTICAL; MATHEMATICS; MODELS, THEORETICAL; NEURAL NETWORKS (COMPUTER); PRINCIPAL COMPONENT ANALYSIS;

EID: 75549089874     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2009.2029694     Document Type: Article

References (19)

1. S. Akaho, "e-PCA and m-PCA: Dimension reduction of parameters by information geometry, " in Proc. Int. Joint Conf. Neural Netw., 2004, pp. 129-134.
2. S. Amari and H. Nagaoka, Methods of Information Geometry. London, U.K.: AMS/Oxford Univ. Press, 2000.
3. H. Attias, "Inferring parameters and structure of latent variable models by variational Bayes, " in Proc. Uncertainty Artif. Intell., 1999, pp. 21-30.
4. A. Banerjee, S. Merugu, I. S. Dhillon, and J. Ghosh, "Clustering with Bregman divergences, " J. Mach. Learn. Res., vol. 6, pp. 1705-1749, 2005.
5. C. M. Bishop, "Bayesian PCA, " Adv. Neural Inf. Process. Syst., vol. 11, pp. 382-388, 1999.
6. C. M. Bishop, "Variational PCA, " in Proc. Int. Conf. Artif. Neural Netw., 1999, pp. 509-514.
7. M. Collins, S. Dasgupta, and R. E. Schapire, "A generalization of principal component analysis to the exponential family, " Adv. Neural Inf. Process. Syst., vol. 14, pp. 617-624, 2002.
8. I. Dhillon, S. Mallela, and R. Kumar, "A divisive information-theoretic feature clustering algorithm for text classification, " J. Mach. Learn. Res., vol. 3, pp. 1265-1287, 2003.
9. C. Ding and T. Li, "Adaptive dimension reduction using discriminant analysis and K-means clustering, " in Proc. Int. Conf. Mach. Learn., 2007, pp. 521-528.
10. N. Kato, M. Suzuki, S. Omachi, H. Aso, and Y. Nemoto, "A handwritten character recognition system using directional element feature and asymmetric Mahalanobis distance, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 3, pp. 258-262, Mar. 1999.
11. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition, " Proc. IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.
12. N. Matsumoto, M. Okada, Y. Sugase-Miyamoto, and K. Kawano, "Population dynamics of face-responsive neurons in the inferior temporal cortex, " Cerebral Cortex, vol. 15, no. 8, pp. 1103-1112, 2005.
13. S. Oba, M. Sato, and S. Ishii, "Prior hyperparameters in Bayesian PCA, " in Proc. Int. Conf. Artif. Neural Netw., 2003, pp. 271-279.
14. S. Oba, M. Sato, and S. Ishii, "Variational Bayes method for mixture of principal component analyzers, " Syst. Comput. Jpn., vol. 34, no. 11, pp. 55-66, 2003.
15. S. Omachi, M. Omachi, and H. Aso, "An approximation method of the quadratic discriminant function and its application to estimation of high-dimensional distribution, " IEICE Trans. Inf. Syst., vol. E90-D, no. 8, pp. 1160-1167, 2007.
16. Sajama and A. Orlitsky, "Semi-parametric exponential family PCA: Reducing dimensions via non-parametric latent distribution estimation, " Univ. California, San Diego, CA, Tech. Rep. CS2004-0790, 2004.
17. M. E. Tipping, "Probabilistic visualisation of high-dimensional binary data, " Adv. Neural Inf. Process. Syst., vol. 11, pp. 592-598, 1999.
18. M. E. Tipping and C. M. Bishop, "Probabilistic principal component analysis, " J. R. Statist. Soc. B, vol. 61, pt. 3, pp. 611-622, 1999.
19. K. Watanabe and S. Watanabe, "Stochastic complexities of general mixture models in variational Bayesian leaning, " Neural Netw., vol. 20, no. 2, pp. 210-219, 2007.