Count data modeling and classification using finite mixtures of distributions

IEEE Transactions on Neural Networks

Volumn 22, Issue 2, 2011, Pages 186-198

  Bouguila, Nizar ^a  

^a Concordia University ^*  (Canada)

Author keywords

Count data; deterministic annealing expectation maximization; Dirichlet; finite mixture models; Fisher kernel; generalized Dirichlet; Liouville; model selection; multinomial; shape modeling; support vector machine; text categorization; texture classification

Indexed keywords

COUNT DATA; DETERMINISTIC ANNEALING EXPECTATION-MAXIMIZATION; DIRICHLET; FINITE MIXTURE MODELS; FISHER KERNELS; LIOUVILLE; MODEL SELECTION; MULTINOMIALS; SHAPE MODELING; TEXT CATEGORIZATION; TEXTURE CLASSIFICATION;

CLUSTER ANALYSIS; CLUSTERING ALGORITHMS; COMPUTER VISION; DATA MINING; INFORMATION RETRIEVAL; MAXIMUM PRINCIPLE; MIXTURES; OPTIMIZATION; RAPID THERMAL ANNEALING; SUPPORT VECTOR MACHINES; TEXT PROCESSING; TEXTURES;

CLASSIFICATION (OF INFORMATION);

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; DATA MINING; HUMAN; INFORMATION PROCESSING; MATHEMATICAL PHENOMENA; METHODOLOGY; STANDARD; STATISTICS; THEORETICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; AUTOMATIC DATA PROCESSING; COMPUTER SIMULATION; DATA MINING; HUMANS; MATHEMATICAL CONCEPTS; MODELS, THEORETICAL; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED;

EID: 79951674150     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2010.2091428     Document Type: Article

References (58)

1. B. D. Ripley, Pattern Recognition and Neural Networks. Cambridge, U.K.: Cambridge Univ. Press, 1996.
2. N. Bouguila, "A model-based approach for discrete data clustering and feature weighting using MAP and stochastic complexity," IEEE Trans. Knowl. Data Eng., vol. 21, no. 12, pp. 1649-1664, Dec. 2009.
3. G. J. McLachlan and D. Peel, Finite Mixture Models. New York: Wiley, 2000.
4. H. G. C. Tråvén, "A neural network approach to statistical pattern classification by 'semiparametric' estimation of probability density functions," IEEE Trans. Neural Netw., vol. 2, no. 3, pp. 366-377, May 1991.
5. N. Ueda and K. Saito, "Parametric mixture models for multi-labeled text," in Proc. 16th Annu. Conf. Neural Inform. Process. Syst., Vancouver, BC, Canada, Dec. 2002, pp. 721-728.
6. H. B. Barlow, "Unsupervised learning," Neural Comput., vol. 1, no. 3, pp. 295-311, 1989.
7. H. Tirri, P. Kontkanen, and P. Myllymäki, "Probabilistic instance-based learning," in Proc. 13th Int. Conf. Mach. Learn., Bari, Italy, Jul. 1996, pp. 507-515.
8. S. Eyheramendy, D. D. Lewis, and D. Madigan, "On the naïve Bayes model for text categorization," in Proc. 9th Int. Workshop Artif. Intell. Statist., Key West, FL, Jan. 2003, pp. 1-8.
9. A. McCallum and K. Nigam, "A comparison of event models for naïve Bayes text classification," in Proc. AAAI Workshop Learn. Text Categorization, Madison, WI, Jul. 1998, pp. 41-48.
10. N. Bouguila and D. Ziou, "Improving content based image retrieval systems using finite multinomial Dirichlet mixture," in Proc. 14th IEEE Workshop Mach. Learn. Signal Process., Sao Luis, Brazil, Oct. 2004, pp. 23-32. (Pubitemid 40557170)
11. R. E. Madsen, D. Kauchak, and C. Elkan, "Modeling word burstiness using the Dirichlet distribution," in Proc. 22nd Int. Conf. Mach. Learn., Bonn, Germany, Aug. 2005, pp. 545-552. (Pubitemid 43183377)
12. N. Bouguila and D. Ziou, "Unsupervised learning of a finite discrete mixture: Applications to texture modeling and image databases summarization," J. Visual Commun. Image Represent., vol. 18, no. 4, pp. 295-309, Aug. 2007. (Pubitemid 46978397)
13. N. Bouguila and O. Amayri, "A discrete mixture-based kernel for SVMs: Application to spam and image categorization," Inform. Process. Manage., vol. 45, no. 6, pp. 631-642, Nov. 2009.
14. D. Margaritis and S. Thrun, "A Bayesian multiresolution independence test for continuous variables," in Proc. 7th Conf. Uncertainty Artif. Intell., Seattle, WA, Aug. 2001, pp. 346-353.
15. P. M. Hooper, "Dependent Dirichlet priors and optimal linear estimators for belief net parameters," in Proc. 10th Conf. Uncertainty Artif. Intell., Banff, AB, Canada, Jul. 2004, pp. 251-259.
16. J. H. Albert and A. K. Gupta, "Bayesian estimation methods for 2 2 contingency tables using mixtures of Dirichlet distributions," J. Amer. Statist. Assoc., vol. 78, no. 383, pp. 708-717, Sep. 1983.
17. P. Giudici and R. Castelo, "Improving Markov chain Monte Carlo model search for data mining," Mach. Learn., vol. 50, nos. 1-2, pp. 127-158, 2003.
18. N. Bouguila, "Clustering of count data using generalized Dirichlet multinomial distributions," IEEE Trans. Knowl. Data Eng., vol. 20, no. 4, pp. 462-474, Apr. 2008. (Pubitemid 351337416)
19. C. Elkan, "Clustering documents with an exponential-family approximation of the Dirichlet compound multinomial distribution," in Proc. 23rd Int. Conf. Mach. Learn., Pittsburgh, PA, Jun. 2006, pp. 289-296.
20. N. Bouguila, "Spatial color image databases summarization," in Proc. IEEE Int. Conf. Acous., Speech, Signal Process., Honolulu, HI, Apr. 2007, pp. 953-956.
21. G. Pólya, "Sur quelques points de la théorie des probabilités," Ann. de l'Inst. Henri Poincaré, vol. 1, no. 2, pp. 117-161, 1930.
22. T.-T. Wong, "Alternative prior assumptions for improving the performance of naïve Bayesian classifiers," Data Mining Knowl. Discovery, vol. 18, no. 2, pp. 183-213, Apr. 2009.
23. N. Bouguila and W. ElGuebaly, "Discrete data clustering using finite mixture models," Pattern Recognit., vol. 42, no. 1, pp. 33-42, Jan. 2009.
24. K. T. Fang, S. Kotz, and K. W. Ng, Symmetric Multivariate and Related Distributions. London, U.K.: Chapman & Hall, 1990.
25. B. D. Sivazlian, "On a multivariate extension of the Gamma and Beta distributions," SIAM J. Appl. Math., vol. 41, no. 2, pp. 205-209, Oct. 1981. (Pubitemid 12503945)
26. R. D. Gupta and D. S. P. Richards, "Multivariate Liouville distributions," J. Multivariate Anal., vol. 23, no. 2, pp. 233-256, Dec. 1987.
27. N. Bouguila, D. Ziou, and E. Monga, "Practical Bayesian estimation of a finite Beta mixture through Gibbs sampling and its applications," Statist. Comput., vol. 16, no. 2, pp. 215-225, Jun. 2006. (Pubitemid 43742298)
28. J. Aitchison, The Statistical Analysis of Compositional Data. London, U.K.: Chapman & Hall, 1990.
29. G. J. McLachlan and T. Krishnan, The EM Algorithm and Extensions. New York: Wiley, 1997.
30. H. S. Seung, H. Sompolinsky, and N. Tishby, "Statistical mechanics of learning from examples," Phys. Rev. A, vol. 45, no. 8, pp. 6056-6091, Apr. 1992.
31. K. Rose, "Deterministic annealing for clustering, compression, classification, regression, and related optimization problems," Proc. IEEE, vol. 86, no. 11, pp. 2210-2239, Nov. 1998. (Pubitemid 128720301)
32. S. Zhong, "Semi-supervised model-based document clustering: A comparative study," Mach. Learn., vol. 65, no. 1, pp. 3-29, Oct. 2006. (Pubitemid 44451192)
33. N. Ueda and R. Nakano, "Deterministic annealing EM algorithm," Neural Netw., vol. 11, no. 2, pp. 271-282, Mar. 1998. (Pubitemid 28231473)
34. F. A. Graybill, Matrices with Applications in Statistics. Belmont, CA: Wadsworth, 1983.
35. K. Zhang and J. T. Kwok, "Simplifying mixture models through function approximation," IEEE Trans. Neural Netw., vol. 21, no. 4, pp. 644-658, Apr. 2010.
36. N. Bouguila and D. Ziou, "A Dirichlet process mixture of generalized Dirichlet distributions for proportional data modeling," IEEE Trans. Neural Netw., vol. 21, no. 1, pp. 107-122, Jan. 2010.
37. N. Bouguila and D. Ziou, "High-dimensional unsupervised selection and estimation of a finite generalized Dirichlet mixture model based on minimum message length," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 10, pp. 1716-1731, Oct. 2007.
38. R. E. Kass and A. E. Raftery, "Bayes factors," J. Amer. Statist. Assoc., vol. 90, no. 430, pp. 773-795, Jun. 1995.
39. N. Murata, S. Yoshizawa, and S.-I. Amari, "Network information criterion-determining the number of hidden units for an artificial neural network model," IEEE Trans. Neural Netw., vol. 5, no. 6, pp. 865-872, Nov. 1994.
40. D. B. Fogel, "An information criterion for optimal neural network selection," IEEE Trans. Neural Netw., vol. 2, no. 5, pp. 490-497, Sep. 1991.
41. J. Rissanen, "Modeling by shortest data description," Automatica, vol. 14, no. 5, pp. 465-471, Sep. 1987.
42. C. D. Manning, P. Raghavan, and H. Schütze, Introduction to Information Retrieval. Cambridge, U.K.: Cambridge Univ. Press, 2008.
43. A. Varma and A. Zisserman, "Texture classification: Are filter banks necessary?" in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Madison, WI, Jun. 2003, pp. 691-698.
44. E. Hayman, B. Caputo, M. Fritz, and J.-O. Eklundh, "On the significance of real-world conditions for material classification," in Proc. 8th Eur. Conf. Comput. Vis., Prague, Czech Republic, May 2004, pp. 253-266.
45. L. S. Davis, M. Clearman, and J. K. Aggarwal, "An empirical evaluation of generalized cooccurrence matrices," IEEE Trans. Pattern Anal. Mach. Intell., vol. 3, no. 2, pp. 214-221, Mar. 1981. (Pubitemid 11497389)
46. M. Tuceryan and A. K. Jain, "Texture analysis," in The Handbook of Pattern Recognition and Computer Vision, P. S. P. Wang, C. H. Chen, and L. F. Pau, Eds. Singapore: World Scientific, 1998, pp. 207-248.
47. J. Zhang, M. Marszalek, S. Lazebnik, and C. Schmid, "Local features and kernels for classification of texture and object categories: A comprehensive study," Int. J. Comput. Vis., vol. 73, no. 2, pp. 213-238, Jun. 2007. (Pubitemid 46181625)
48. S. Lazebnik, C. Schmid, and J. Ponce, "A sparse texture representation using local affine regions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 8, pp. 1265-1278, Aug. 2005. (Pubitemid 41245056)
49. B. Julesz, "Textons, the elements of texture perception, and their interactions," Nature, vol. 290, no. 5802, pp. 91-97, Mar. 1981. (Pubitemid 11149681)
50. K. J. Dana, B. van Ginneken, S. K. Nayar, and J. J. Koenderink, "Reflectance and texture of real-world surfaces," ACM Trans. Graph., vol. 18, no. 1, pp. 1-34, Jan. 1999. (Pubitemid 129547675)
51. T. Lindeberg, "Feature detection with automatic scale selection," Int. J. Comput. Vis., vol. 30, no. 2, pp. 79-116, 1998. (Pubitemid 128517917)
52. D. G. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, Nov. 2004.
53. L. Shams and C. von der Malsburg, "Are object shape primitives learnable?" Neurocomputing, vols. 26-27, pp. 855-863, Jun. 1999. (Pubitemid 29447397)
54. S. Belongie, J. Malik, and J. Puzicha, "Shape context: A new descriptor for shape matching and object recognition," in Proc. 14th Annu. Conf. Neural Inform. Process. Syst., Vancouver, BC, Canada, Dec. 2000, pp. 831-837.
55. T. S. Jaakkola and D. Haussler, "Exploiting generative models in discriminative classifiers," in Proc. 12th Annu. Conf. Neural Inform. Process. Syst., Vancouver, BC, Canada, Dec. 1998, pp. 487-493.
56. N. Bouguila, "A data-driven mixture kernel for count data classification using support vector machines," in Proc. IEEE Workshop Mach. Learn. Signal Process., Cancun, Mexico, Oct. 2008, pp. 26-31.
57. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proc. IEEE, vol. 86, no. 11, pp. 2278- 2324, Nov. 1998.
58. S. Jeannin and M. Bober, "Description of core experiments for MPEG- 7 motion/shape," MPEG-7 Visual Group, Seoul, Korea, Tech. Rep. ISO/IEC JTC 1/SC 29/WG 11 MPEG99/N2690, Mar. 1999.