A dirichlet process mixture of generalized dirichlet distributions for proportional data modeling

IEEE Transactions on Neural Networks

Volumn 21, Issue 1, 2010, Pages 107-122

  Bouguila, Nizar ^a   Ziou, Djemel ^b  

^a Concordia University ^*  (Canada)

^b UNIVERSITÉ DE SHERBROOKE   (Canada)

Author keywords

Data classification; Dirichlet processes; Generalized dirichlet distribution; Gibbs sampling; Image database categorization; MCMC; Metropolis hastings (M H); Mixture modeling; Nonparametric bayesian analysis; Proportional data

Indexed keywords

DATA CLASSIFICATION; DIRICHLET PROCESS; GENERALIZED DIRICHLET DISTRIBUTION; GIBBS SAMPLING; IMAGE DATABASE CATEGORIZATION; MIXTURE MODELING; NON-PARAMETRIC BAYESIAN;

BAYESIAN NETWORKS; CLASSIFICATION (OF INFORMATION); CLUSTERING ALGORITHMS; DATABASE SYSTEMS; IMAGE RETRIEVAL; MIXTURES; SAMPLING; VARIATIONAL TECHNIQUES;

CLUSTER ANALYSIS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; BAYES THEOREM; CLUSTER ANALYSIS; COMPUTER ASSISTED DIAGNOSIS; HUMAN; INFORMATION RETRIEVAL; STATISTICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; BAYES THEOREM; CLUSTER ANALYSIS; HUMANS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; INFORMATION STORAGE AND RETRIEVAL; MODELS, STATISTICAL;

EID: 73949130630     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2009.2034851     Document Type: Article

References (71)

1. G. J. McLachlan and D. Peel, Finite Mixture Models. New York: Wiley, 2000.
2. N. Bouguila and D. Ziou, "High-dimensional unsupervised selection and estimation of a finite generalized Dirichlet mixture model based on minimum message length," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 10, pp. 1716-1731, Oct. 2007.
3. K. Roeder and L. Wasserman, "Practical Bayesian density estimation using mixture of normals," J. Amer. Statist. Assoc., vol. 92, pp. 894-902, 1997.
4. S. Richardson and P. J. Green, "On Bayesian analysis of mixtures with an unknown number of components (with discussion)," J. R. Statist. Soc. B, vol. 59, pp. 731-792, 1997.
5. M. Stephens, "Dealing with label switching in mixture models," J. R. Statist. Soc. B, vol. 62, no. 4, pp. 795-809, 2000.
6. T. S. Ferguson, "Bayesian density estimation by mixtures of normal distributions," in Recent Advances in Statistics, H. Rizvi and J. Rustagi, Eds. New York: Academic, 1983, pp. 287-302.
7. M. D. Escobar and M. West, "Bayesian density estimation and inference using mixtures," J. Amer. Statist. Assoc., vol. 90, no. 430, pp. 577-588, 1995.
8. S. N. MacEachern and P. Müller, "Estimating mixture of Dirichlet process models," J. Comput. Graph. Statist., vol. 7, pp. 223-238, 1998.
9. R. M. Neal, "Markov chain sampling methods for Dirichlet process mixture models," J. Comput. Graph. Statist., vol. 9, pp. 249-265, 2000.
10. T. S. Ferguson, "A Bayesian analysis of some nonparametric problems," Ann. Statist., vol. 1, no. 2, pp. 209-230, 1973.
11. C. E. Antoniak, "Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems," Ann. Statist., vol. 2, no. 6, pp. 1152-1174, 1974.
12. R. M. Korwar and M. Hollander, "Contributions to the theory of Dirichlet processes," Ann. Probab., vol. 1, no. 4, pp. 705-711, 1973.
13. D. Blackwell and J. MacQueen, "Ferguson distributions via Pólya Urn schemes," Ann. Statist., vol. 1, no. 2, pp. 353-355, 1973.
14. M. West, "Modelling with mixtures (with discussion)," in Bayesian Statistics, A. P. Dawid, J. M. Bernardo, J. O. Berger, and A. F. M. Smith, Eds. Oxford, U.K.: Oxford Univ. Press, 1992, pp. 503-524.
15. M. West, P. Müller, and M. D. Escobar, "Hierarchical priors and mixture models, with application in regression and density estimation," in Aspects of Uncertainty: A Tribute to D. V. Lindley, P. R. Freeman and A. F. Smith, Eds. New York: Wiley, 1994, pp. 363-386.
16. M. D. Escobar, "Estimating normal means with a Dirichlet process prior," J. Amer. Statist. Assoc., vol. 89, pp. 268-277, 1994.
17. S. N. MacEachern, "Estimating normal means with a conjugate style Dirichlet processes prior," Commun. Statist. B, Simul. Comput., vol. 23, pp. 727-741, 1994.
18. C. A. Bush and S. N. MacEachernl, "A semiparametric Bayesian model for randomised block designs," Biometrika, vol. 83, pp. 275-285, 1996.
19. S. Jain and R. M. Neal, "A split-merge Markov chain Monte Carlo procedure for the Dirichlet process mixture model," J. Comput. Graph. Statist., vol. 13, pp. 158-182, 2004.
20. H. Ishwaran and L. F. James, "Gibbs sampling methods for stick-breaking priors," J. Amer. Statist. Assoc., vol. 96, no. 453, pp. 161-173, 2001.
21. C. E. Rasmussen, "The infinite Gaussian mixture model," in Advances in Neural Information Processing Systems (NIPS). Cambridge, MA: MIT Press, 2000, pp. 554-560.
22. A. Gelfand and A. Kottas, "A computational approach for full nonparametric Bayesian inference under Dirichlet process mixture models," J. Comput. Graph. Statist., vol. 11, pp. 289-305, 2002.
23. Y. W. Teh, M. I. Jordan, M. I. Beal, J. Matthew, and D. M. Blei, "Hierarchical Dirichlet processes," J. Amer. Statist. Assoc., vol. 101, no. 476, pp. 1566-1581, 2006.
24. J. Sethuraman, "A constructive definition of Dirichlet priors," Statistica Sinica, vol. 4, pp. 639-650, 1994.
25. P. J. Green and S. Richardson, "Modelling heterogeneity with and without the Dirichlet process," Scandinavian J. Statist., vol. 28, pp. 355-375, 2001.
26. J. Aitchison, The Statistical Analysis of Compositional Data. Caldwell, NJ: Blackburn, 2003.
27. N. Bouguila, D. Ziou, and J. Vaillancourt, "Unsupervised learning of a finite mixture model based on the Dirichlet distribution and its application," IEEE Trans. Image Process., vol. 13, no. 11, pp. 1533-1543, Nov. 2004.
28. N. Bouguila and D. Ziou, "A hybrid SEM algorithm for high-dimensional unsupervised learning using a finite generalized Dirichlet mixture," IEEE Trans. Image Process., vol. 15, no. 9, pp. 2657-2668, Sep. 2006.
29. N. Bouguila, D. Ziou, and R. I. Hammoud, "A Bayesian non-Gaussian mixture analysis: Application to eye modeling," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., 2007, pp. 1-8.
30. S. Boutemedjet and D. Ziou, "A graphical model for context-aware visual content recommendation," IEEE Trans. Multimedia, vol. 10, no. 1, pp. 52-62, Jan. 2008.
31. G. Csurka, C. R. Dance, L. Fan, J. Willamowski, and C. Bray, "Visual categorization with bags of keypoints," in Proc. Workshop Statist. Learn. Comput. Vis./8th Eur. Conf. Comput. Vis., 2004, pp. 1-22.
32. R. J. Connor and J. E. Mosimann, "Concepts of independence for proportions with a generalization of the Dirichlet distribution," J. Amer. Statist. Assoc., vol. 64, pp. 194-206, 1969.
33. S. Boutemedjet, N. Bouguila, and D. Ziou, "A hybrid feature extraction selection approach for high-dimensional non-Gaussian data clustering," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 9, pp. 1429-1443, Sep. 2009.
34. A. Gelman, "Prior distributions for variance parameters in hierarchical models," Bayesian Anal., vol. 1, no. 3, pp. 515-533, 2006.
35. B. P. Carlin and T. A. Louis, Bayes and Empirical Bayes Methods for Data Analysis, 2nd ed. Boca Raton, FL: Chapman & Hall/CRC, 2000.
36. A. O'Hagan, "Eliciting expert beliefs in substantial practical applications," The Statistician, vol. 47, no. 1, pp. 21-35, 1998.
37. J. B. Kadane and L. J.Wolfson, "Experience in elicitation," The Statistician, vol. 47, no. 1, pp. 3-19, 1998.
38. P. H. Garthwaite and S. A. Al-Awadhi, "Non-conjugate prior distribution assessment for multivariate normal sampling," J. R. Statist. Soc. B, vol. 63, pp. 1-16, 2001.
39. C. P. Robert and G. Casella, Monte Carlo Statistical Methods. New York: Springer-Verlag, 1999.
40. M. I. Jordan, "Graphical models," Statist. Sci., vol. 19, no. 1, pp. 140-155, 2004.
41. W. R. Gilks and P.Wild, "Algorithm AS 287: Adaptive rejection sampling from log-concave density functions," Appl. Statist., vol. 42, no. 4, pp. 701-709, 1993.
42. G. Celeux, M. Hurn, and C. P. Robert, "Computational and inferential difficulties with mixture posterior distributions," J. Amer. Statist. Assoc., vol. 95, no. 451, pp. 957-970, 2000.
43. G. Casella, K. Mengersen, C. Robert, and D. Titterington, "Perfect slice samplers for mixtures of distributions," J. R. Statist. Soc. B, vol. 64, no. 4, pp. 777-790, 2000.
44. G. Casella, C. P. Robert, and M. T. Wells, "Mixture models, latent variables and partitioned importance sampling," Statist. Methodol., vol. 1, no. 1-2, pp. 1-18, 2004.
45. Z. Zhang, K. L. Chan, Y. Wu, and C. Chen, "Learning a multivariate Gaussian mixture model with the reversible jump MCMC algorithm," Statist. Comput., vol. 14, pp. 343-355, 2004.
46. A. Jarsa, C. C. Holmes, and D. A. Stephens, "Markov chain Monte Carlo and the label switching problem in Bayesian mixture modelling," Statist. Sci., vol. 20, pp. 50-67, 2005.
47. R. E. Kass, B. P. Carlin, A. Gelman, and R. M. Neal, "Markov chain Monte Carlo in practice: A roundtable discussion," Amer. Statistician, vol. 52, no. 2, pp. 93-100, 1998.
48. M. K. Cowles and B. P. Carlin, "Markov chain Monte Carlo convergence diagnostics: A comparative review," J. Amer. Statist. Assoc., vol. 91, no. 434, pp. 883-904, 1996.
49. S. P. Brooks and G. O. Roberts, "Convergence assessment techniques for Markov chain Monte Carlo," Statist. Comput., vol. 8, no. 4, pp. 319-335, 1998.
50. C. P. Robert, The Bayesian Choice From Decision-Theoretic Foundations to Computational Implementation, 2nd ed. NewYork: Springer- Verlag, 2007.
51. A. E. Raftery and S. M. Lewis, "One long run with diagnostics: Implementation strategies for Markov chain Monte Carlo," Statist. Sci., vol. 7, no. 4, pp. 493-497, 1992.
52. A. E. Raftery and S. M. Lewis, "Implementing MCMC," in Markov Chain Monte Carlo in Practice, D. J. Spiegelhalter, W. R. Gilks, and S. Richardson, Eds. London, U.K.: Chapman & Hall, 1996, pp. 115-130.
53. R. E. Kass and A. E. Raftery, "Bayes factors," J. Amer. Statist. Assoc., vol. 90, pp. 773-795, 1995.
54. N. Bouguila and D. Ziou, "A Dirichlet process mixture of Dirichlet distributions for classification and prediction," in Proc. IEEE Workshop Mach. Learn. Signal Process., 2008, pp. 297-302.
55. R. E. Madsen, D. Kauchak, and C. Elkan, "Modeling word burstiness using the Dirichlet distribution," in Proc. 22nd Int. Conf. Mach. Learn., 2005, pp. 545-552.
56. C. Apte, F. J. Damerau, and S. M. Weiss, "Automated learning of decision rules for text categorization," ACM Trans. Inf. Syst., vol. 12, pp. 233-251, 1994.
57. D. M. Blei, A. Y. Ng, and M. I. Jordan, "Latent Dirichlet allocation," J. Mach. Learn. Res., vol. 3, pp. 993-1022, 2003.
58. H.-Y.Wang, H. Zha, and H. Qin, "Dirichlet aggregation: Unsupervised learning towards an optimal metric for proportional data," in Proc. Int. Conf. Mach. Learn., 2007, pp. 959-966.
59. F. Sebastiani, "Machine learning in automated text categorization," ACM Comput. Surv., vol. 34, no. 1, pp. 1-47, 2002.
60. S. Boutemedjet, D. Ziou, and N. Bouguila, "Unsupervised feature selection for accurate recommendation of high-dimensional image data," in Advances in Neural Information Processing Systems (NIPS). Cambridge, MA: MIT Press, 2007, pp. 177-184.
61. E. B. Sudderth, A. B. Torralba, W. T. Freeman, and A. S. Willsky, "Describing visual scenes using transformed Dirichlet processes," in Advances in Neural Information Processing Systems (NIPS). Cambridge, MA: MIT Press, 2006, pp. 1297-1304.
62. L. Fei-Fei and P. Perona, "A Bayesian hierarchical model for learning natural scene categories," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., 2005, pp. 524-531.
63. D. G. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, 2004.
64. S. Lazebnik, C. Schmid, and J. Ponce, "Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., 2006, pp. 2169-2178.
65. J. J. Kivinen, E. B. Sudderth, and M. I. Jordan, "Learning multiscale representations of natural scenes using Dirichlet processes," in Proc. IEEE 11th Int. Conf. Comput. Vis., 2007, pp. 1-8.
66. A. Bosch, A. Zisserman, and X. Munoz, "Scene classification via pLSA," in Proc. 9th Eur. Conf. Comput. Vis., 2006, pp. 517-530.
67. N. Bouguila and D. Ziou, "Unsupervised selection of a finite Dirichlet mixture model: An MML-based approach," IEEE Trans. Knowl. Data Eng., vol. 18, no. 8, pp. 993-1009, Aug. 2006.
68. J. Yang, Y.-G. Jiang, A. G. Hauptmann, and C.-W. Ngo, "Evaluating bag-of-visual-words representations in scene classification," in Proc. Int. Workshop Multimedia Inf. Retrieval, 2007, pp. 197-206.
69. D. M. Chickering and D. Heckerman, "Efficient approximations for the marginal likelihood of Bayesian networks with hidden variables," Mach. Learn., vol. 29, pp. 181-212, 1997.
70. A. E. Raftery, "Hypothesis testing and model selection," in Markov Chain Monte Carlo in Practice, D. J. Spiegelhalter, W. R. Gilks, and S. Richardson, Eds. London, U.K.: Chapman & Hall, 1996, pp. 163-188.
71. G. Schwarz, "Estimating the dimension of a model," Ann. Statist., vol. 6, pp. 461-464, 1978.