Determining the number of clusters using information entropy for mixed data

Pattern Recognition

Volumn 45, Issue 6, 2012, Pages 2251-2265

  Liang, Jiye ^a   Zhao, Xingwang ^a   Li, Deyu ^a   Cao, Fuyuan ^a   Dang, Chuangyin ^b  

^a SHANXI UNIVERSITY   (China)

^b CITY UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Cluster validity index; Clustering; Information entropy; k Prototypes algorithm; Mixed data; Number of clusters

Indexed keywords

CLUSTER VALIDITY INDICES; CLUSTERING; INFORMATION ENTROPY; MIXED DATA; NUMBER OF CLUSTERS;

CLUSTER ANALYSIS; FUZZY CLUSTERING;

CLUSTERING ALGORITHMS;

EID: 84857042237     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2011.12.017     Document Type: Article

References (64)

1. J.W. Han, and M. Kamber Data Mining Concepts and Techniques 2001 Morgan Kaufmann San Francisco
2. A.K. Jain, M.N. Murty, and P.J. Flynn Data clustering: a review ACM Computing Surveys 31 3 1999 264 323
3. R. Xu, and D. Wunsch II Survey of clustering algorithms IEEE Transactions on Neural Networks 16 3 2005 645 678 (Pubitemid 40718010)
4. A.K. Jain Data clustering: 50 years beyond k-means Pattern Recognition Letters 31 8 2010 651 666
5. H.P. Kriegel, P. Kröger, and A. Zimek Clustering high-dimensional data: a survey on subspace clustering, pattern-based clustering and correlation clustering ACM Transactions on Knowledge Discovery from Data 3 1 2009 1 58
6. L. Bai, J.Y. Liang, C.Y. Dang, and F.Y. Cao A novel attribute weighting algorithm for clustering high-dimensional categorical data Pattern Recognition 44 12 2011 2843 2861
7. T.W. Liao Clustering of time series data survey Pattern Recognition 38 11 2005 1857 1874
8. C.C. Aggarwal, J.W. Han, J.Y. Wang, P.S. Yu, A framework for clustering evolving data streams, in: Proceedings of the 29th VLDB Conference, Berlin, Germany, 2003.
9. F.Y. Cao, J.Y. Liang, L. Bai, X.W. Zhao, and C.Y. Dang A framework for clustering categorical time-evolving data IEEE Transactions on Fuzzy Systems 18 5 2010 872 882
10. L. Hunt, and M. Jorgensen Clustering mixed data WIREs Data Mining and Knowledge Discovery 1 4 2011 352 361
11. V. Estivill-Castro, J. Yang, A fast and robust general purpose clustering algorithm, in: Proceeding of 6th Pacific Rim International Conference Artificial Intelligence, Melbourne, Australia, 2000, pp. 208218.
12. A. Ahmad, and L. Dey A k-mean clustering algorithm for mixed numeric and categorical data Data & Knowledge Engineering 63 2 2007 503 527 (Pubitemid 47053940)
13. Z.X. Huang Extensions to the k-means algorithm for clustering large data sets with categorical values Data Mining and Knowledge Discovery 2 3 1998 283 304
14. J.B. MacQueen, Some methods for classification and analysis of multivariate observations, in: Proceeding of the 5th Berkeley Symposium on Mathematical Statistics and Probability, 1967, pp. 281297.
15. F. Höppner, F. Klawonn, and R. Kruse Fuzzy Cluster Analysis: Methods for Classification, Data Analysis, and Image Recognition 1999 Wiley New York
16. Z.X. Huang, A fast clustering algorithm to cluster very large categorical data sets in data mining, in: Proceeding of the SIGMOD Workshop on Research Issues on Data Mining and Knowledge Discovery, 1997, pp. 18.
17. L. Kaufman, and P. Rousseeuw Finding Groups in Data: An Introduction to Cluster Analysis 1990 Wiley
18. T.K. Xiong, S.R. Wang, A. Mayers, E. Monga, DHCC: Divisive hierarchical clustering of categorical data, Data Mining and Knowledge Discovery, doi: 10.1007/s10618-011-0221-2 2011.
19. C. Li, and G. Biswas Unsupervised learning with mixed numeric and nominal data IEEE Transactions on Knowledge and Data Engineering 14 4 2002 673 690
20. C.C. Hsu, C.L. Chen, and Y.W. Su Hierarchical clustering of mixed data based on distance hierarchy Information Sciences 177 20 2007 4474 4492
21. S. Guha, R. Rastogi, K. Shim, CURE: An efficient clustering algorithm for large databases, in: Proceeding of ACM SIGMOD International Conference Management of Data, 1998, pp. 7384.
22. G. Karypis, E. Han, and V. Kumar Chameleon: hierarchical clustering using dynamic modeling IEEE Computer 32 8 1999 68 75
23. T. Zhang, R. Ramakrishnan, M. Livny, BIRCH: an efficient data clustering method for very large databases, in: Proceeding of ACM SIGMOD International Conference Management of Data, 1996, pp. 103114.
24. S. Guha, R. Rastogi, and K. Shim ROCK: a robust clustering algorithm for categorical attributes Information Systems 25 5 2000 345 366
25. B. Mirkin Choosing the number of clusters WIREs Data Mining and Knowledge Discovery 1 3 2011 252 260
26. H.J. Sun, S.R. Wang, and Q.S. Jiang FCM-based model selection algorithms for determining the number of clusters Pattern Recognition 37 10 2004 2027 2037
27. R. Kothari, and D. Pitts On finding the number of clusters Pattern Recognition Letters 20 4 1999 405 416 (Pubitemid 29377623)
28. M.J. Li, M.K. Ng, Y. Cheung, and Z.X. Huang Agglomerative fuzzy k-means clustering algorithm with selection of number of clusters IEEE Transactions on Knowledge and Data Engineering 20 11 2008 1519 1534
29. Y. Leung, J.S. Zhang, and Z.B. Xu Clustering by scale-space filtering IEEE Transactions on Pattern Analysis and Machine Intelligence 22 12 2000 1394 1410
30. S. Bandyopadhyay, and U. Maulik Genetic clustering for automatic evolution of clusters and application to image classification Pattern Recognition 35 6 2002 1197 1208 (Pubitemid 34262330)
31. S. Bandyopadhyay, and S. Saha A point symmetry-based clustering technique for automatic evolution of clusters IEEE Transactions on Knowledge and Data Engineering 20 11 2008 1441 1457
32. S. Bandyopadhyay Genetic algorithms for clustering and fuzzy clustering WIREs Data Mining and Knowledge Discovery 1 6 2011 524 531
33. C.A. Sugar, and G.M. James Finding the number of clusters in a data set: an information theoretic approach Journal of the American Statistical Association 98 463 2003 750 763 (Pubitemid 37395947)
34. M. Aghagolzadeh, H. Soltanian-Zadeh, B.N. Araabi, A. Aghagolzadeh, Finding the number of clusters in a dataset using an information theoretic hierarchical algorithm, in: Proceedings of the 13th IEEE International Conference on Electronics, Circuits and Systems, 2006, pp. 13361339.
35. L. Bai, J.Y. Liang, and C.Y. Dang An initialization method to simultaneously find initial cluster centers and the number of clusters for clustering categorical data Knowledge-Based Systems 24 6 2011 785 795
36. K.K. Chen, L. Liu, The best k for entropy-based categorical clustering, in: Proceeding of the 17th International Conference on Scientific and Statistical Database Management, 2005.
37. H. Yan, K.K. Chen, L. Liu, and J. Bae Determining the best k for clustering transactional datasets: a coverage density-based approach Data & Knowledge Engineering 68 1 2009 28 48
38. R. Jensen, Q. Shen, Fuzzy-rough sets for descriptive dimensionality reduction, in: Proceeding of the 2002 IEEE International Conference on Fuzzy Systems, 2002, pp. 2934.
39. C.E. Shannon A mathematical theory of communication Bell Systems Technical Journal 27 3-4 1948 379 423
40. D. Barbara, Y. Li, J. Couto, Coolcat: an entropy-based algorithm for categorical clustering, in: Proceeding of the 2002 ACM CIKM International Conference on Information and Knowledge Management, 2002, pp. 582589.
41. Z.Y. He, S.C. Deng, and X.F. Xu An optimization model for outlier detection in categorical data Lecture Notes in Computer Science vol. 3644 2005 400 409
42. I. Düntsch, and G. Gediga Uncertainty measures of rough set prediction Artificial Intelligence 106 1 1998 109 137 (Pubitemid 128402352)
43. A. Renyi, On measures of entropy and information, in: Proceeding of the 4th Berkeley Symposium on Mathematics of Statistics and Probability, 1961, pp. 547561.
44. E. Parzen On the estimation of a probability density function and the mode Annals of Mathematical Statistics 33 3 1962 1065 1076
45. R. Jenssen, T. Eltoft, D. Erdogmus, and J.C. Principe Some equivalences between kernel methods and information theoretic methods Journal of VLSI Signal Processing Systems 49 12 2006 49 65 (Pubitemid 46132343)
46. E. Gokcay, and J.C. Principe Information theoretic clustering IEEE Transactions on Pattern Analysis and Machine Intelligence 24 2 2002 158 171 (Pubitemid 34198206)
47. J.Y. Liang, K.S. Chin, C.Y. Dang, and C.M. Yam Richard A new method for measuring uncertainly and fuzziness in rough set theory International Journal of General Systems 31 4 2002 331 342 (Pubitemid 41623247)
48. Y.H. Qian, J.Y. Liang, W. Pedrycz, and C.Y. Dang Positive approximation: an accelerator for attribute reduction in rough set theory Artificial Intelligence 174 910 2010 597 618
49. Y.H. Qian, J.Y. Liang, D.Y. Li, H.Y. Zhang, and C. Y Dang Measures for evaluating the decision performance of a decision table in rough set theory Information Sciences 178 1 2008 181 202
50. J.Y. Liang, and D.Y. Li Uncertainty and Knowledge Acquisition in Information Systems 2005 Science Press Beijing, China
51. J.Y. Liang, Z.Z. Shi, D.Y. Li, and M.J. Wierman The information entropy, rough entropy and knowledge granulation in incomplete information system International Journal of General Systems 35 6 2006 641 654 (Pubitemid 44728618)
52. M. Halkidi, and M. Vazirgiannis A density-based cluster validity approach using multi-representatives Pattern Recognition Letters 29 6 2008 773 786
53. M. Rezaee, B. Lelieveldt, and J. Reiber A new cluster validity index for the fuzzy c-mean Pattern Recognition Letters 19 3-4 1998 237 346
54. W.N. Wang, and Y.J. Zhang On fuzzy cluster validity indices Fuzzy Sets and Systems 158 19 2007 2095 2117
55. M.A. Gluck, and J.E. Corter Information, uncertainty, and the utility of categories Proceeding of the 7th Annual Conference of the Cognitive Science Society 1985 Lawrence Erlbaum Associates Irvine, CA 283 287
56. D.H. Fisher Knowledge acquisition via incremental conceptual clustering Machine Learning 2 2 1987 139 172
57. K. McKusick, K. Thompson, COBWEB/3: A Portable Implementation, Technical Report FIA-90-6-18-2, NASA Ames Research Center, 1990.
58. B. Mirkin Reinterpreting the category utility function Machine Learning 45 2 2001 219 228 (Pubitemid 33635986)
59. J. Al-Shaqsi, W.J. Wang, A clustering ensemble method for clustering mixed data, in: The 2010 International Joint Conference on Neural Networks, 2010.
60. W.D. Zhao, W.H. Dai, C.B. Tang, K-centers algorithm for clustering mixed type data, in: Proceedings of the 11th Pacific-Asia Conference on Knowledge Discovery and Data Mining, 2007, pp. 11401147.
61. J.C. Bezdek Pattern Recognition in Handbook of Fuzzy Computation 1998 IOP Publishing Limited Boston, New York (Chapter F6)
62. L. Hubert, and P. Arabie Comparing partitions Journal of Classification 2 1 1985 193 218
63. M. Bohanec, V. Rajkovic, Knowledge acquisition and explanation for multi-attribute decision making, in: Proceeding of the 8th International Workshop on Expert Systems and Their Applications, Avignon, France, 1988, pp. 5978.
64. UCI Machine Learning Repository 〈 http://www.ics.uci.edu/mlearn/ MLRepository.html 〉, 2011.