Online learning of a dirichlet process mixture of generalized dirichlet distributions for simultaneous clustering and localized feature selection

Journal of Machine Learning Research

Volumn 25, Issue , 2012, Pages 113-128

  Wenta, Wentao Fan ^a   Bouguila, Nizar ^b  

^a Gina Cody School of Engineering and Computer Science   (Canada)

^b NONE   (Canada)

Author keywords

Clustering; Dirichlet process; Generalized dirichlet mixtures; Localized feature selection; Nonparametric bayesian; Online learning; Variational bayes

Indexed keywords

CLUSTERING; DIRICHLET PROCESS; GENERALIZED DIRICHLET MIXTURES; LOCALIZED FEATURES; NON-PARAMETRIC BAYESIAN; ONLINE LEARNING; VARIATIONAL BAYES;

LEARNING SYSTEMS; MIXTURES;

E-LEARNING;

EID: 84876887338     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (39)

1. D. Achlioptas. Database-Friendly Random Projections. In Proc. of the twentieth ACM symposium on Principles of database systems (PODS), pages 274-281. ACM, 2001.
2. S. Amari. Natural Gradient Works Efficiently in Learning. Neural Comput., 10:251-276, 1998.
3. Y. Amit and A. Trouvé. POP: Patchwork of Parts Models for Object Recognition. International Journal of Computer Vision, 75:267-282, 2007.
4. H. Attias. A Variational Bayes Framework for Graphical Models. In Proc. of Neural Information Processing Systems (NIPS), pages 209-215, 1999.
5. E. Bingham and H. Mannila. Random Projection in Dimensionality Reduction: Applications To Image and Text Data. In Proc. of the ACM SIGKDD international conference on Knowledge discovery and data mining (KDD), pages 245-250, 2001.
6. C. M. Bishop. Pattern Recognition and Machine Learning. Springer, 2006.
7. D. M. Blei and M. I. Jordan. Variational Inference for Dirichlet Process Mixtures. Bayesian Analysis, 1:121-144, 2005.
8. N. Bouguila and D. Ziou. A Powreful Finite Mixture Model Based on the Generalized Dirichlet Distribution: Unsupervised Learning and Applications. In ICPR (1), pages 280-283, 2004.
9. N. Bouguila and D. Ziou. MML-Based Approach for High-Dimensional Learning using the Generalized Dirichlet Mixture. In CVPR '05: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops, page 53. IEEE Computer Society, 2005.
10. N. Bouguila and D. Ziou. A Dirichlet Process Mixture of Generalized Dirichlet Distributions for Proportional Data Modeling. IEEE Transactions on Neural Networks, 21(1):107-122, 2010.
11. N. Bouguila, D. Ziou, and R. I. Hammoud. A Bayesian Non-Gaussian Mixture Analysis: Application to Eye Modeling. In Proc. of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2007.
12. N. Bouguila, D. Ziou, and R. I. Hammoud. On Bayesian Analysis of a Finite Generalized Dirichlet Mixture Via a Metropolis-within-Gibbs Sampling. Pattern Analysis and Applications, 12(2):151- 166, 2009.
13. S. Boutemedjet, N. Bouguila, and D. Ziou. A Hybrid Feature Extraction Selection Approach for High-Dimensional Non-Gaussian Data Clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(8):1429-1443, 2009.
14. C. Constantinopoulos, M.K. Titsias, and A. Likas. Bayesian Feature And Model Selection for Gaussian Mixture Models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28 (6):1013-1018, 2006.
15. L. Engebretsen, P. Indyk, and R. O'Donnell. Derandomized Dimensionality Reduction With Applications. In Proc. of the thirteenth annual ACM-SIAM symposium on Discrete algorithms (SODA), pages 705-712, 2002.
16. W. Fan, N. Bouguila, and D. Ziou. Unsupervised Anomaly Intrusion Detection via Localized Bayesian Feature Selection. In Proc. of International Conference on Data Mining (ICDM), pages 1032-1037, 2011.
17. T. S. Ferguson. Bayesian Density Estimation by Mixtures of Normal Distributions. Recent Advances in Statistics, 24:287-302, 1983.
18. D. Fradkin and D. Madigan. Experiments With Random Projections For Machine Learning. In Proc. of the ACM SIGKDD international conference on Knowledge discovery and data mining (KDD), pages 517-522, 2003.
19. Y. Guan, J. G. Dy, and M. I. Jordan. A Unified Probabilistic Model for Global and Local Unsupervised Feature Selection. In Proc. of International Conference on Machine Learning (ICML), pages 1073-1080, 2011.
20. A. Hadid, M. Pietikainen, and T. Ahonen. A Discriminative Feature Space for Detecting and Recognizing Faces. In Proc. of CVPR 2004, volume 2, pages 797-804, 2004.
21. M. D. Hoffman, D. M. Blei, and F. Bach. Online Learning for Latent Dirichlet Allocation. In Proc. of Neural Information Processing Systems (NIPS), pages 856-864, 2010.
22. M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul. An Introduction to Variational Methods for Graphical Models. Machine Learning, 37(2):183-233, 1999.
23. S. Kaski. Dimensionality Reduction by Random Mapping: Fast Similarity Computation for Clustering. In Proc. of International Conference on Neural Networks (IJCNN), pages 413-418, 1998.
24. E. Keogh and M. Pazzani. Learning Augmented Bayesian Classifiers: A Comparison of Distribution- Based and Classification-Based Approaches. In Proc. of the seventh international workshop on artificial intelligence and statistics (AISTAT), pages 225-230, 1999.
25. H. Kim, P. Howland, and H. Park. Dimension Reduction in Text Classification with Support Vector Machines. Journal of Machine Learning Research, 6:37-53, 2005.
26. R. M. Korwar and M. Hollander. Contributions to the theory of Dirichlet processes. The Annals of Probability, 1:705-711, 1973.
27. H.J. Kushner and G. Yin. Stochastic Approximation Algorithms and Applications. Applications of mathematics. Springer, 1997.
28. M. H. C. Law, M. A. T. Figueiredo, and A. K. Jain. Simultaneous Feature Selection and Clustering Using Mixture Models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(9): 1154-1166, 2004.
29. D. D. Lewis, Y. Yang, T. G. Rose, and F. Li. RCV1: A New Benchmark Collection for Text Categorization Research. Journal of Machine Learning Research, 5:361-397, 2004.
30. S. Z. Li, L. Zhu, Z. Zhang, A. Blake, H. Zhang, and H. Shum. Statistical Learning of Multi-view Face Detection. In Proc. of the 7th European Conference on Computer Vision (ECCV), pages 67-81, 2002.
31. Y. Li, M. dong, and J. Hua. Simultaneous Localized Feature Selection and Model Detection for Gaussian Mixtures. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31:953- 960, 2009.
32. Z. Ma and A. Leijon. Bayesian Estimation of Beta Mixture Models with Variational Inference. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(11):2160 - 2173, 2011.
33. G.J. McLachlan and D. Peel. Finite Mixture Models. New York: Wiley, 2000.
34. R. M. Neal. Markov Chain Sampling Methods for Dirichlet Process Mixture Models. Journal of Computational and Graphical Statistics, 9(2):249-265, 2000.
35. C.P. Robert and G. Casella. Monte Carlo Statistical Methods. Springer-Verlag, 1999.
36. M. Sato. Online Model Selection Based on the Variational Bayes. Neural Comput., 13:1649-1681, 2001.
37. J. Sethuraman. A Constructive Definition of Dirichlet Priors. Statistica Sinica, 4:639-650, 1994.
38. Y. W. Teh, M. I. Jordan, M. J. Beal, and D. M. Blei. Hierarchical Dirichlet Processes. Journal of the American Statistical Association, 101:705-711, 2004.
39. C. Wang, J. W. Paisley, and D. M. Blei. Online Variational Inference for the Hierarchical Dirichlet Process. Journal of Machine Learning Research - Proceedings Track, 15:752-760, 2011.