Discrete data clustering using finite mixture models

Pattern Recognition

Volumn 42, Issue 1, 2009, Pages 33-42

  Bouguila, Nizar ^a   ElGuebaly, Walid ^a  

^a Concordia University ^*  (Canada)

Author keywords

Discrete data; EM; Finite mixture models; Generalized Dirichlet distribution; Image databases; Labeled and unlabeled images; Multinomial; Spatial color; Summarization; Text classification

Indexed keywords

ARTIFICIAL INTELLIGENCE; CLASSIFICATION (OF INFORMATION); CLUSTERING ALGORITHMS; COMPUTER NETWORKS; COMPUTER VISION; FEATURE EXTRACTION; IMAGE PROCESSING; PATTERN RECOGNITION;

DISCRETE DATA; EM; FINITE MIXTURE MODELS; GENERALIZED DIRICHLET DISTRIBUTION; IMAGE DATABASES; LABELED AND UNLABELED IMAGES; MULTINOMIAL; SPATIAL COLOR; SUMMARIZATION; TEXT CLASSIFICATION;

MIXTURES;

EID: 51649088874     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2008.06.022     Document Type: Article

References (55)

1. Bouguila N., and Ziou D. Unsupervised learning of a finite discrete mixture: applications to texture modeling and image databases summarization. J. Visual Commun. Image Representation 18 4 (2007) 295-309
2. Puzicha J., Buhmann J.M., and Hofmann T. Histogram clustering for unsupervised image segmentation. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (1999) 2602-2608
3. Juan A., and Vidal E. On the use of Bernoulli mixture models for text classification. Pattern Recognition 35 (2002) 2705-2710
4. Robertson S.E., and Walker S. Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. SIGIR '94 (1994), Springer, New York 232-241
5. Churchand K.W., and Gale W.A. Poisson mixtures. Nat. Lang. Eng. 1 (1995) 163-190
6. Nigam K., McCallum A., Thrun S., and Mitchell T. Text classification from labeled and unlabeled documents using EM. Mach. Learn. 39 2 (2000) 103-134
7. McCallum A., and Nigam K. A comparison of event models for naive Bayes text classification. AAAI-98 Workshop on Learning for Text Categorization (1998), AAAI Press 41-48
8. Rennie J.D.M., Shih L., Teevan J., and Karger D.R. Tackling the poor assumptions of naive Bayes text classifiers. Proceedings of the 20th International Conference on Machine Learning (ICML2003) (2003), Morgan Kauffmann, Los Altos, CA 616-623
9. Madsen R.E., Kauchak D., and Elkan C. Modeling word burstiness using the Dirichlet distribution. Proceedings of the 22nd International Conference on Machine Learning (ICML2005) (2005), ACM Press, New York 545-552
10. Lewis D. Naive (Bayes) at forty: the independence assumption in information retrieval. Proceedings of the 10th European Conference on Machine Learning (ECML1998) (1998), Springer, Berlin 4-15
11. Bouguila N., Ziou D., and Vaillancourt J. Novel mixtures based on the Dirichlet distribution: application to data and image classification. In: Perner P., and Rosenfeld A. (Eds). Machine Learning and Data Mining in Pattern Recognition (MLDM2003), Lecture Notes in Artificial Intelligence, vol. 2734 (2003), Springer, Berlin 172-181
12. Bouguila N., and Ziou D. Improving content based image retrieval systems using finite multinomial Dirichlet mixture. Proceedings of the IEEE Workshop on Machine Learning for Signal Processing (MLSP2004). Sao Luis, Brazil (2004) 23-32
13. Bolstad W.M. Introduction to Bayesian Statistics (2004), Wiley, New York
14. Kotz S., Ng K.W., and Fang K. Symmetric Multivariate and Related Distributions (1990), Chapman & Hall, London, New York
15. DeGroot M.H. Optimal Statistical Decisions (2004), Wiley-Interscience, New York
16. Bouguila N., Ziou D., and Vaillancourt J. Unsupervised learning of a finite mixture model based on the Dirichlet distribution and its application. IEEE Trans. Image Process. 13 11 (2004) 1533-1543
17. Bouguila N., and Ziou D. A hybrid SEM algorithm for high-dimensional unsupervised learning using a finite generalized Dirichlet mixture. IEEE Trans. Image Process. 15 9 (2006) 2657-2668
18. McLachlan G.J., and Peel D. Finite Mixture Models (2000), Wiley, New York
19. Bouguila N., and Ziou D. A powerful finite mixture model based on the generalized Dirichlet distribution: unsupervised learning and applications. Proceedings of the 17th International Conference on Pattern Recognition, ICPR2004 (2004) 280-283
20. Bouguila N., and Ziou D. High-dimensional unsupervised selection and estimation of a finite generalized Dirichlet mixture model based on minimum message length. IEEE Trans. Pattern Anal. Mach. Intell. 29 10 (2007) 1716-1731
21. T.P. Minka, Bayesian inference, entropy, and the multinomial distribution, Technical Report, CMU, 2003.
22. Katz S.M. Distribution of content words and phrases in text and language modelling. Nat. Lang. Eng. 2 (1996) 15-59
23. J. Aitchison, The Statistical Analysis of Compositional Data, second ed., Chapman & Hall, New York.
24. Connor R.J., and Mosimann J.E. Concepts of independence for proportions with a generalization of the Dirichlet distribution. J. Am. Statist. Assoc. 64 (1969) 194-206
25. Wong T. A Bayesian approach employing generalized Dirichlet priors in predicting microchip yields. J. Chin. Inst. Ind. Eng. 22 3 (2005) 210-217
26. Lochner R.H. A generalized Dirichlet distribution in Bayesian life testing. J. R. Statist. Soc. B 37 (1975) 103-113
27. Thall P.F., and Sung H.G. Some extensions and applications of a Bayesian strategy for monitoring multiple outcomes in clinical trials. Statist. Med. 17 (1998) 1563-1580
28. Wong T. Generalized Dirichlet distribution in Bayesian analysis. Appl. Math. Comput. 97 (1998) 165-181
29. Carlin B.P., and Louis T.A. Bayes and Empirical Bayes Methods for Data Analysis. second ed. (2000), Chapman & Hall, CRC, London
30. McLachlan G.J., and Krishnan T. The EM Algorithm and Extensions (1997), Wiley-Interscience, New York
31. Graybill F.A. Matrices with Applications in Statistics (1983), Wadsworth, California
32. Jamshidian M., and Jennrich R.I. Acceleration of the EM algorithm by using quasi-Newton methods. J. R. Statist. Soc. (B) 59 3 (1997) 569-587
33. Hille E. Analytic Function Theory vol. I (1959), Ginn & Company
34. Lange K. Applications of the Dirichlet distribution to forensic match probabilities. Genetica 96 (1995) 107-117
35. Duda R.O., Hart P.E., and Stork D.G. Pattern Classification (2001), Wiley, New York
36. Lin C.-T., Chen C.-B., and Wu W.-H. Fuzzy clustering algorithm for latent class model. Statist. Comput. 14 4 (2004) 299-310
37. Dhillon I.S., and Modha D.S. Concept decompositions for large sparse text data using clustering. Mach. Learn. 42 1-2 (2001) 143-175
38. Strehl A., Ghosh J., and Mooney R. Impact of similarity measures on web-pages clustering. Proceedings of the AAAI-2000 Workshop of Artificial Intelligence for Web Search (2000), AAAI, MIT Press, Cambridge 58-64
39. Salton G., and McGill M.J. Introduction to Modern Retrieval (1983), McGraw-Hill, New York
40. Bouguila N., and Ziou D. Unsupervised selection of a finite Dirichlet mixture model: an MML-based approach. IEEE Trans. Knowl. Data Eng. 18 8 (2006) 993-1009
41. Schwarz G. Estimating the dimension of a model. Ann. Statist. 6 (1978) 461-464
42. Rui Y., Huang T.S., and Chang S.-F. Image retrieval: current techniques, promising directions and open issues. J. Visual Commun. Image Representation 10 (1999) 39-62
43. Smeulders A.W.M., Worring M., Santini S., Guptaand A., and Jain R. Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell. 22 12 (2000) 1349-1380
44. Medasani S., and Krishnapuram R. Categorization of image databases for efficient retrieval using robust mixture decomposition. Comput. Vision Image Understanding 83 (2001) 216-235
45. Swain M., and Ballard D. Color Indexing. Int. J. Comput. Vision 7 1 (1991) 11-32
46. Stricker M., and Swain M. The capacity of color histogram indexing. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (1995) 704-708
47. Pass G., and Zabih R. Comparing images using joint histograms. Multimedia Syst. 7 (1999) 234-240
48. N. Bouguila, Spatial color image databases summarization, in: IEEE International Conference on Acoustics, Speech, and Signal Processing ICASSP2007, vol. 1, Honolulu, HI, USA, 2007, pp. I-953-I-956.
49. Huang J., Kumar S.R., Mitra M., Zhu W., and Zabih R. Spatial color indexing and applications. Int. J. Comput. Vision 35 3 (1999) 245-268
50. Katz S. Estimation of probabilities from sparse data for the language model component of a speech recognizer. IEEE Trans. Acoust. Speech Signal Process. 35 3 (1987) 400-401
51. Witten I.H., and Bell T.C. The zero-frequency problem: estimating the probabilities of novel events in adaptive text compression. IEEE Trans. Inform. Theory 37 4 (1991) 1085-1094
52. Chen S.F., and Goodman J. An empirical study of smoothing techniques for language modeling. In: Joshi A., and Palmer M. (Eds). Proceedings of the 34th Annual Meeting of the Association for Computational Linguistics (1996), Morgan Kaufmann Publishers, Los Altos, CA 310-318
53. Hartley H.O., and Rao J.N.K. Classification and estimation in analysis of variance problems. Rev. Int. Statist. Inst. 36 (1968) 141-147
54. Day N.E. Estimating the components of a mixture of normal distributions. Biometrika 56 3 (1969) 463-474
55. A.K. McCallum, Bow: a toolkit for statistical language modeling, text retrieval, classification and clustering 〈http://www.cs.cmu.edu/∼mccallum/bow〉, 1996.