Mixture-model cluster analysis using information theoretical criteria

Intelligent Data Analysis

Volumn 11, Issue 2, 2007, Pages 155-173

  Fonseca, Jaime R S ^a   Cardoso, Margarida G M S ^b  

^a Instituto Superior de Ciências Sociais e Políticas ISCSP Technical University of Lisbon   (Portugal)

^b UNIVERSITY INSTITUTE OF LISBON   (Portugal)

Author keywords

Cluster analysis; Finite mixture models; Information theoretical criteria; Model selection

Indexed keywords

EID: 41849101656     PISSN: 1088467X     EISSN: 15714128     Source Type: Journal    
DOI: 10.3233/ida-2007-11204     Document Type: Article

References (68)

1. H. Akaike, Information Theory and an Extension of Maximum Likelihood Principle, in: Selected Papers of Hirotugu Akaike, in Proceedings of the Second International Symposium on Information Theory, B.N. Petrov and F. caski, eds, Akademiai Kiado, Budapest, 1973, pp. 267-281, K.T. Emanuel Parzen and Genshiro Kitagawa, eds, Springer-Verlag New York, Inc, Texas, 1973, p. 434.
2. H. Akaike, Maximum likelihood identification of Gaussian autorregressive moving average models, Biometrika 60 (1973), 255-265.
3. R.L. Andrews and I.S. Currim, A comparison of segment retention for finite Mixture Logit Models, Journal of Marketing Research XI (2003a), 235-243.
4. S. Bandyopadhyay and U. Maulik, Nonparametric Genetic Clustering: Comparison of Validity Indices, IEEE Transactions on Systems, Man, and Cybernetics-Part C: Applications and Reviews 31 (2001), 120-125.
5. J.D. Banfield and A.E. Raftery, Model-Based Gaussian and Non-Gaussian Clustering, Biometrics 49 (1993), 803-821.
6. H. Bensmail and J.J. Meulman, Model-based Clustering with Noise: Bayesian Inference and Estimation, Journal of Classification 20 (2003), 49-76.
7. J.C. Bezdek, W.Q. Li, Y. Attikiouzel and M. Windham, A geometric approach to cluster validity for normal mixtures, Soft Computing 1 (1997), 166-179.
8. A. Bhatnagar and S. Ghose, A latent class segmentation analysis of e-shoppers, Journal of Business Research 57 (2004), 758-767.
9. C. Biernacki, Choix de modèles en Classification-Ph.D. Tesis., 1997.
10. C. Biernacki, G. Celeux and G. Govaert, Assessing a Mixture model for Clustering with the integrated Completed Likelihood, IEEE Transactions on Pattern analysis and Machine Intelligence 22 (2000), 719-725.
11. C. Biernacki, G. Celeux and G. Govaert, Choosing starting values for the EM algorithm for getting the highest likelihood in multivariate Gaussian mixture models, Computational Statistics & Data Analysis 41 (2003), 561-575.
12. C. Biernacki, G. Celeux and G. Govaert, An improvement of the NEC criterion for assessing the number of clusters in mixture model, Pattern Recognition Letters 20 (1999), 267-272.
13. C Biernacki and G. Govaert, Using the classification likelihood to choose the number of clusters, Computing Science and Statistics 29 (1997), 451-457.
14. S. Boucheron and E. Gassiat, Order Estimation and Model Selection, in: Inference in Hidden markov Models, O.C.a.T. Rydden, ed., 2002, p. 25.
15. H. Bozdogan, ICOMP: A New Model Selection criterion, in: Classification and Related Methods of Data Analysis, Hans H. Bock (ed.), North-Holland, Amesterdam, 1988, pp. 599-608.
16. H. Bozdogan, Mixture-Model Cluster Analysis using Model Selection criteria and a new Informational Measure of Complexity, in: Proceedings of the First US/Japan Conference on the Frontiers of Statistical Modeling: An Approach, 69-113, H. Bozdogan, ed., Kluwer Academic Publishers, 1994, pp. 69-113.
17. H. Bozdogan, Model Selection and Akaikes's Information Criterion (AIC): The General Theory and its Analytical Extensions, Psycometrika 52 (1987), 345-370.
18. H. Bozdogan and P. Bearse, Information complexity criteria for detecting influential observations in dynamic multivariate linear models using the genetic algorithm, Journal of Statistical Planning and Inference 114 (2003), 31-44.
19. K.P. Burnham and D.R. Anderson, Multimodel Inference. Understanding AIC and BIC in Model Selection, Sociological Methods & Research 33 (2004), 261-304.
20. G. Celeux and G. Soromenho, An entropy criterion for acessing the number of clusters in a mixture model, Journal of Classification 13(1996), 195-212.
21. A. Cutler and M.P. Windham, Information-Based Validity Functionals for Mixture Analysis, in: First US/Japan Conference on the Frontiers of Statistical Modeling: An Informational Approach, H. Bozdogan, ed., Kluwer Academic Publishers, 1994.
22. N.E. Day, Estimating the Components of a mixture of normal Distributions, Biometrika 56 (1969), 463-474.
23. A.P. Dempster, N.M. Laird and D.B. Rubin, Maximum Likelihood from ioncomplete Data via EM algorithm. Journal of the Royal Statistics Society, B 39 (1977), 1-38.
24. J.G. Dias, Finite Mixture Models; Review, Applications, and Computer-intensive Methods, Econimics, Groningen University, PhD Thesis, Groningen, 2004, 199.
25. W.R. Dillon and A. Kumar, Latent structure and other mixture models in marketing: An integrative survey and overview, chapter 9, in: Advanced methods of Marketing Research, R.P. Bagozi, ed., Cambridge: blackwell Publishers, 1994, pp. 352-388.
26. A.F. Famili, G. Liu and Z. Liu, Evaluation and optimization of clustering in gene expression data analysis, Bioinfonnatics 20 (2004), 1535-1545.
27. M.A.T. Figueiredo and A.K. Jain, Unsupervised Learning of Finite Mixture Models, IEEE Transactions on pattern analysis and Machine Intelligence 24 (2002), 1-16.
28. R.S. Fonseca, Jaime and G.M.S. Cardoso, Margarida, Retail Clients Latent Segments. Proceedings of the 12th Portuguese Conference on Artificial Intelligence, EPIA 2005, December 5-8, in: Progress in Artificial Intelligence, C. Bento, A. Cardoso and G. Dias, eds, Springer-Verlag, Lecture Notes in Computing Science, Vol. 3808, 2005, pp. 348-358.
29. J.G. Fryer and C.A. Robertson, A Comparision of Some methods for Estimating Mixed Normal Distributions, Biometrika 59 (1972), 639-648.
30. M. Halkidi, Y. Batistakis and M. Vazirgiannis, On Clustering Validation Techniques, Journal of Intelligent Information Systems 17 (2001), 107-145.
31. P. Hall and D.M. Titterington, Efficient Nonparametric Estimation of Mixture Proportions, Journal of the Royal Statistical Society, Series B 46 (1984), 465-473.
32. R.J. Hataway, A Constrained Formulation of Maximum-Likelihood Estimation for Normal Mixture Distributions, The Annals of Statistics 13 (1985), 795-800.
33. E.R. Hruschka and N.F.F. Ebecken, A genetic algorithm for cluster analysis, Intelligent Data Analysis 7 (2003), 15-25.
34. L. Hunt and M. Jorgensen, Mixture model clustering for mixed data with missing information, Computational Statistics & Data Analysis 41 (2003), 429-440.
35. L.A. Hunt and K.E. Basford, Fitting a Mixture Model to Three-Mode Trhee-Way Data with Categorical and Continuous Variables, Journal of Classification 16 (1999), 283-296.
36. C.M. Hurvich and C.-L. Tsai, regression and Time Series Model Selection in Small Samples, Biometrika 76 (1989), 297-307.
37. L.F. James, C.E. Priebe and D.J. Marchette, Consistency Estimation of Mixture Complexity, The Annals of Statistics 29 (2001), 1281-1296.
38. K. Jedidi, V. Ramaswamy and W.S. DeSarbo, A Maximum Likelihood Method for Latent Class Regression involving a Censored dependent Variable, Psycometrika 58 (1993), 375-394.
39. A.B.M.L. Kabir, Estimation of Parameters of a Finite Mixture of Distributions, Journal of the Royal Statistical Society, Series B (Methodological) 30 (1968), 472-482.
40. C. Keribin, Estimation consistante de Forde de modèes de mélange, Comptes Rendues de l'Academie des Sciences, Paris, t. 326 Séie I (1998), 243-248.
41. Y. Kim, W.N. Street and F. Menezer, Evolutionary model selection in unsupervised learning, Intelligent Data Analysis 6 (2002), 531-556.
42. B.G. Leroux, Consistent Estimation of a Mixing Distribution, The Annals of Statistics 20 (1992), 1350-1360.
43. B.G. Leroux and M.L. Puterman, Maximum-Penalized-Likelihood Estimation for Independent and Markov-Dependent Mixture Models, Biometrics 48 (1992), 545-558.
44. R. Maronna and P.M. Jacovkis, Multivariate Clustering Procedures with variable Metrics, Biometrics 30 (1974), 499-505.
45. G. McLachlan and T. Krishnan, The EM Algorithm and Extensions, John Wiley & Sons, New York, 1997.
46. G.F. McLachlan and N. Khan, On a resampling approach for tests on the number of clusters with mixture model-based clustering of tissue samples, Journal of multivariate Analysis 90 (2004), 90-105.
47. G.F. McLachlan and D. Peel, Finite Mixture Models, John Wiley & Sons, Inc., 2000.
48. G.J. McLachlan, On Bootstrapping the Likelihood Ratio Test Statistic for the Number of components in a Normal Mixture, Appllied Statistics 36 (1987), 318-324.
49. G.J. McLachlan and K.E. Basford, Mixture Models: Inference and Applications to Clustering., Marcel Deckker, Inc., New York, 1988.
50. A. McQuarrie, R. Shumway and C.-L. Tsai, The model selection criterion AICu, Statistics & Probability Letters 34 (1997), 285-292.
51. I. Moustaki and I. Papageorgiou, Latent class models for mixed variables with applications in Archaeometry, Computational Statistics & Data Analysis, In Press (2004).
52. S. Newcomb, A Generalized Theory of the Combination of Observations so as to Obtain the Best Result, American journal of Mathematics 8 (1886), 343-366.
53. M.K. Pakhira, S. Bandyopadhyay and U. Maulik, Validity index for crisp anf fuzzy clusters, Pattern Recognition 37 (2004), 487-501.
54. K. Pearson, On the probability that two independent Distributions of frequency are really samples of the same population with special reference to recent work on the identity of trypanosome strains, Biometrika 10 (1914), 85-143.
55. Y. Qu and S. Xu, Supervised cluster analysis for microarray data based on multivariate Gaussian mixture, Bioinformatics 20 (2004), 1905-1913.
56. J. Rissanen, Modeling by shortest data description, Automatica 14 (1978), 465-471.
57. K. Roeder, A graphical Technique for Determining the Number of components in a Mixture of Normals, Journal of the American Statistical Association 89 (1994), 487-495.
58. G. Schwarz, Estimating the Dimenson of a Model, The Annals of Statistics 6 (1978), 461-464.
59. J. Seo, M. Bakay, Y.-W. Chen, S. Hilmer, B. Shneiderman and E.P. Hoffman, Interactively optimizing signal-to-noise ratios in expression profiling: project-specific algorithm selection and detection p-value weighting in Affymetrix mocroarrays, Bioinformatics 20 (2004), 2534-2544.
60. M. Shaked, On Mixtures from Exponential Families, Journal of the Royal Statistical Society, Series B (Methodological) 42 (1980), 192-198.
61. R. Shibata, Information Criteria for Statistical Model Selection, Electronics and Communications in Japan, Part 3 85 (2002), 605-611.
62. J. Tao, N.-Z. Shi and S.-Y. Lee, Drug risk assessment with determining the number of sub-populations under finite mixture normal models, Computational Statistics & Data Analysis 46 (2004), 661-676.
63. J.K. Vermunt and J. Magidson, Latent class cluster analysis., in: Applied Latent Class Analysis, J.A. Hagenaars and A.L. McCutcheon, eds, Cambridge University Press, 2002, 89-106.
64. M. Vriens, Market Segmentation. Analytical Developments and Applications Guidlines, Technical Overview Series, Millward Brown IntelliQuest, 2001, 1-42.
65. H.x. Wang, Q.b. Zhang, B. Luo and S. Wei, Robust mixture modelling using multivariate t-distribution with missing information, Pattern Recognition Letters 25 (2004), 701-710.
66. R. Wehrens, L.M.C Buydens, C. Fraley and A.E. Raftery, Model-Based Clustering for Image Segmentation and Large Datasets via Sampling, Journal of Classification 21 (2004), 231-253.
67. X.L. Xie and G. Beni, A validity measure for Fuzzy Clustering, IEEE Transactions on pattern analysis and Machine Intelligence 13 (1991), 841-847.
68. X. Yang and S.M. Krishnan, Image segmentation using finite mixtures and spatial information, Image and Vision Computing 22 (2004), 735-745.