A comparative study of efficient initialization methods for the k-means clustering algorithm

Expert Systems with Applications

Volumn 40, Issue 1, 2013, Pages 200-210

  Celebi, M Emre ^a   Kingravi, Hassan A ^b   Vela, Patricio A ^b  

^a Louisiana State University   (United States)

^b GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Cluster center initialization; k means; Partitional clustering; Sum of squared error criterion

Indexed keywords

CLUSTER CENTERS; COMPARATIVE STUDIES; DATA SETS; GRADIENT DESCENT; INITIALIZATION METHODS; K-MEANS; K-MEANS CLUSTERING ALGORITHM; LINEAR TIME COMPLEXITY; NON-PARAMETRIC STATISTICAL TESTS; PARTITIONAL CLUSTERING; PARTITIONAL CLUSTERING ALGORITHM; PERFORMANCE CRITERION; SUM OF SQUARED ERRORS;

STATISTICAL TESTS;

CLUSTERING ALGORITHMS;

EID: 84866127615     PISSN: 09574174     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.eswa.2012.07.021     Document Type: Article

References (76)

1. Al-Daoud, M. (2005). A new algorithm for cluster initialization, In: Proc. of the 2nd World Enformatika conf. (pp. 74-76).
2. M. Al-Daoud, and S. Roberts New methods for the initialisation of clusters Pattern Recognition Letters 17 5 1996 451 455
3. M. Al Hasan, V. Chaoji, S. Salem, and M. Zaki Robust partitional clustering by outlier and density insensitive seeding Pattern Recognition Letters 30 11 2009 994 1002
4. D. Aloise, A. Deshpande, P. Hansen, and P. Popat NP-hardness of euclidean sum-of-squares clustering Machine Learning 75 2 2009 245 248
5. D. Aloise, P. Hansen, and L. Liberti An improved column generation algorithm for minimum sum-of-squares clustering Mathematical Programming 2010 1 26
6. M.R. Anderberg Cluster analysis for applications 1973 Academic Press
7. Arthur, D., & Vassilvitskii, S. (2007). k-means++: The advantages of careful seeding, In: Proc. of the 18th annual ACM-SIAM symposium on discrete algorithms (pp. 1027-1035).
8. Astrahan, M. M. (1970). Speech analysis by clustering, or the hyperphoneme method, Tech. Rep. AIM-124, Stanford University.
9. G.P. Babu, and M.N. Murty A near-optimal initial seed value selection in k-means algorithm using a genetic algorithm Pattern Recognition Letters 14 10 1993 763 769
10. G. Babu, and M. Murty Simulated annealing for selecting optimal initial seeds in the k-means algorithm Indian Journal of Pure and Applied Mathematics 25 1-2 1994 85 94
11. G.H. Ball, and D.J. Hall A clustering technique for summarizing multivariate data Behavioral Science 12 2 1967 153 155
12. C.D. Bei, and R.M. Gray An improvement of the minimum distortion encoding algorithm for vector quantization IEEE Transactions on Communications 33 10 1985 1132 1133
13. B. Bergmann, and G. Hommel Multiple hypotheses testing Ch. improvements of general multiple test procedures for redundant systems of hypotheses 1988 Springer pp. 100-115
14. L. Bottou, and Y. Bengio Advances in neural information processing systems Ch. convergence properties of the k-means algorithms 1995 MIT Press pp. 585-592
15. Bradley, P. S., & Fayyad, U. (1998). Refining initial points for k-means clustering, In: Proc. of the 15th int. conf. on machine learning (pp. 91-99).
16. M.M. Breunig, H.P. Kriegel, R.T. Ng, and J. Sander LOF: Identifying density-based local outliers ACM SIGMOD Record 29 2 2000 93 104
17. F. Cao, J. Liang, and G. Jiang An initialization method for the k-means algorithm using neighborhood model Computers and Mathematics with Applications 58 3 2009 474 483
18. M.E. Celebi Improving the performance of k-means for color quantization Image and Vision Computing 29 4 2011 260 271
19. T.W. Chen, and S.Y. Chien Bandwidth adaptive hardware architecture of k-means clustering for video analysis IEEE Transactions on Very Large Scale Integration (VLSI) Systems 18 6 2010 957 966
20. W.Y. Chen, Y. Song, H. Bai, C.J. Lin, and E.Y. Chang Parallel spectral clustering in distributed systems IEEE Transactions on Pattern Analysis and Machine Intelligence 33 3 2011 568 586
21. Cook, W. (2011). World TSP, Georgia Institute of Technology, .
22. W.W. Daniel Applied nonparametric statistics 2000 Duxbury Press
23. Dash, M., Liu, H., & Xu, X. (2001). '1+1 > 2': Merging distance and density based clustering, In: Proc. of the 7th int. conf. on database systems for advanced applications (pp. 32-39).
24. E. Forgy Cluster analysis of multivariate data: Efficiency vs. interpretability of classification Biometrics 21 1965 768
25. Frank, A., & Asuncion, A. (2011). UCI machine learning repository, University of California, Irvine, School of Information and Computer Sciences, .
26. M. Friedman The use of ranks to avoid the assumption of normality implicit in the analysis of variance Journal of the American Statistical Association 32 200 1937 675 701
27. S. Garcia, A. Fernandez, J. Luengo, and F. Herrera A study of statistical techniques and performance measures for genetics-based machine learning: Accuracy and interpretability Soft Computing 13 10 2009 959 977
28. S. Garcia, and F. Herrera An extension on "statistical comparisons of classifiers over multiple data sets" for all pairwise comparisons Journal of Machine Learning Research 9 2008 2677 2694
29. T. Gonzalez Clustering to minimize the maximum intercluster distance Theoretical Computer Science 38 2-3 1985 293 306
30. Hamerly, G. (2010). Making k-means even faster, in: Proc. of the 2010 SIAM int. conf. on data mining (pp. 130-140).
31. J.A. Hartigan, and M.A. Wong Algorithm AS 136: A k-means clustering algorithm Journal of the Royal Statistical Society C 28 1 1979 100 108
32. He, J., Lan, M., Tan, C. L., Sung, S. Y., & Low, H. B. (2004). Initialization of cluster refinement algorithms: A review and comparative study, In: Proc. of the 2004 IEEE int. joint conf. on neural networks (pp. 297-302).
33. H. Hotelling Simplified calculation of principal components Psychometrika 1 1 1936 27 35
34. S.H. Huang, and S.H. Chen Fast encoding algorithm for VQ-based image coding Electronics Letters 26 19 1990 1618 1619
35. C.M. Huang, and R.W. Harris A comparison of several vector quantization codebook generation approaches IEEE Transactions on Image Processing 2 1 1993 108 112
36. L. Hubert, and P. Arabie Comparing partitions Journal of Classification 2 1 1985 193 218
37. A. Hyvärinen Fast and robust fixed-point algorithms for independent component analysis IEEE Transactions on Neural Networks 10 3 1999 626 634
38. R.L. Iman, and J.M. Davenport Approximations of the critical region of the friedman statistic Communications in Statistics - Theory and Methods 9 6 1980 571 595
39. A.K. Jain Data clustering: 50 years beyond k-means Pattern Recognition Letters 31 8 2010 651 666
40. A.K. Jain, M.N. Murty, and P.J. Flynn Data clustering: A review ACM Computing Surveys 31 3 1999 264 323
41. R.C. Jancey Multidimensional group analysis Australian Journal of Botany 14 1 1966 127 130
42. T. Kanungo, D. Mount, N. Netanyahu, C. Piatko, R. Silverman, and A. Wu An efficient k-means clustering algorithm: analysis and implementation IEEE Transactions on Pattern Analysis and Machine Intelligence 24 7 2002 881 892
43. I. Katsavounidis, C.-C.J. Kuo, and Z. Zhang A new initialization technique for generalized Lloyd iteration IEEE Signal Processing Letters 1 10 1994 144 146
44. L. Kaufman, and P. Rousseeuw Finding groups in data: An introduction to cluster analysis 1990 Wiley Interscience
45. G.N. Lance, and W.T. Williams A general theory of classificatory sorting strategies - II. Clustering systems The Computer Journal 10 3 1967 271 277
46. A. Likas, N. Vlassis, and J. Verbeek The global k-means clustering algorithm Pattern Recognition 36 2 2003 451 461
47. Y. Linde, A. Buzo, and R. Gray An algorithm for vector quantizer design IEEE Transactions on Communications 28 1 1980 84 95
48. S. Lloyd Least squares quantization in PCM IEEE Transacations on Information Theory 28 2 1982 129 136
49. J. Luengo, S. Garcia, and F. Herrera A study on the use of statistical tests for experimentation with neural networks: Analysis of parametric test conditions and non-parametric tests Expert Systems with Applications 36 4 2009 7798 7808
50. J.F. Lu, J.B. Tang, Z.M. Tang, and J.Y. Yang Hierarchical initialization approach for k-means clustering Pattern Recognition Letters 29 6 2008 787 795
51. MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations, In: Proc. of the 5th Berkeley symposium on mathematical statistics and probability (pp. 281-297).
52. Mahajan, M., Nimbhorkar, P., & Varadarajan, K. (2012). The planar k-means problem is NP-hard. Theoretical Computer Science, 442, 13-21.
53. R. Maitra, and V. Melnykov Simulating data to study performance of finite mixture modeling and clustering algorithms Journal of Computational and Graphical Statistics 19 2 2010 354 376
54. J. Mao, and A.K. Jain A self-organizing network for hyperellipsoidal clustering (HEC) IEEE Transacations on Neural Networks 7 1 1996 16 29
55. M. Matsumoto, and T. Nishimura Mersenne twister: A 623-dimensionally equidistributed uniform pseudo-random number generator ACM Transactions on Modeling and Computer Simulation 8 1 1998 3 30
56. M. Meila Comparing clusterings - An information based distance Journal of Multivariate Analysis 98 5 2007 873 895
57. G.W. Milligan An examination of the effect of six types of error perturbation on fifteen clustering algorithms Psychometrika 45 3 1980 325 342
58. G. Milligan, and M.C. Cooper A study of standardization of variables in cluster analysis Journal of Classification 5 2 1988 181 204
59. M.J. Norušis IBM SPSS statistics 19 statistical procedures companion 2011 Addison Wesley
60. T. Onoda, M. Sakai, and S. Yamada Careful seeding method based on independent components analysis for k-means clustering Journal of Emerging Technologies in Web Intelligence 4 1 2012 51 59
61. C. Ordonez, and E. Omiecinski Efficient disk-based k-means clustering for relational databases IEEE Transactions on Knowledge and Data Engineering 16 8 2004 909 921
62. S.K. Pal, and D.D. Majumder Fuzzy sets and decision making approaches in vowel and speaker recognition IEEE Transacations on Systems, Man, and Cybernetics 7 8 1977 625 629
63. J.M. Pena, J.A. Lozano, and P. Larranaga An empirical comparison of four initialization methods for the k-means algorithm Pattern Recognition Letters 20 10 1999 1027 1040
64. Pizzuti, C., Talia, D., & Vonella, G. (1999). A divisive initialisation method for clustering algorithms, In: Proc. of the 3rd European conf. on principles and practice of knowledge discovery in databases (pp. 484-491).
65. S.J. Redmond, and C. Heneghan A method for initialising the k-means clustering algorithm using kd-trees Pattern Recognition Letters 28 8 2007 965 973
66. SAS Institute Inc., SAS/STAT 9.2 User's Guide, SAS Publishing, 2009.
67. S.Z. Selim, and M.A. Ismail K-means-type algorithms: A generalized convergence theorem and characterization of local optimality IEEE Transactions on Pattern Analysis and Machine Intelligence 6 1 1984 81 87
68. H. Späth Computational experiences with the exchange method: Applied to four commonly used partitioning cluster analysis criteria European Journal of Operational Research 1 1 1977 23 31
69. T. Su, and J.G. Dy In search of deterministic methods for initializing k-means and Gaussian mixture clustering Intelligent Data Analysis 11 4 2007 319 338
70. A. Tarsitano A computational study of several relocation methods for k-means algorithms Pattern Recognition 36 12 2003 2955 2966
71. J.T. Tou, and R.C. Gonzales Pattern recognition principles 1974 Addison-Wesley
72. van Dongen, S. (2000). Performance criteria for graph clustering and Markov cluster experiments, Tech. Rep. INS-R0012, Centrum voor Wiskunde en Informatica.
73. Voss, T., Hansen, N., & Igel, C. (2010). Improved step size adaptation for the MO-CMA-ES, In: Proc. of the 12th annual conf. on genetic and evolutionary computation (pp. 487-494).
74. J. Wu, J. Chen, H. Xiong, and M. Xie External validation measures for k-means clustering: A data distribution perspective Expert Systems with Applications 36 3 2009 6050 6061
75. Wu, J., Xiong, H., & Chen, J. (2009b). Adapting the right measures for k-means clustering, In: Proc. of the 15th ACM SIGKDD int. conf. on knowledge discovery and data mining (pp. 877-885).
76. J.S. Zhang, and Y.-W. Leung Robust clustering by pruning outliers IEEE Transactions on Systems, Man, and Cybernetics - Part B 33 6 2003 983 999