What to do when K-means clustering fails: A simple yet principled alternative algorithm

PLoS ONE

Volumn 11, Issue 9, 2016, Pages

  Raykov, Yordan P ^a   Boukouvalas, Alexis ^b   Baig, Fahd ^c   Little, Max A ^a,d  

^a ASTON UNIVERSITY   (United Kingdom)

^b UNIVERSITY OF MANCHESTER   (United Kingdom)

^c UNIVERSITY OF OXFORD   (United Kingdom)

^d MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; BAYES THEOREM; CLASSIFICATION ALGORITHM; CLUSTER ANALYSIS; CONTROLLED STUDY; DATA ANALYSIS; HUMAN; K MEANS ALGORITHM; MAJOR CLINICAL STUDY; MAXIMUM A POSTERIORI DIRICHLET PROCESS MIXTURE ALGORITHM; MEDICAL INFORMATICS; PARKINSON DISEASE; STATISTICAL MODEL;

EID: 84992108763     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0162259     Document Type: Article

References (50)

1. Jain AK, Murty MN, Flynn PJ. Data clustering: a review. ACM Computing Surveys. 1999;31(3):264-323. doi: 10.1145/331499.331504
2. Celebi ME, Kingravi HA, Vela PA. A comparative study of efficient initialization methods for the K-means clustering algorithm. Expert Systems with Applications. 2013;40(1):200-210. doi: 10.1016/j. eswa.2012.07.021
3. Bordogna G, Pasi G. A quality driven hierarchical data divisive soft clustering for information retrieval. Knowledge-Based Systems. 2012;26:9-19. doi: 10.1016/j.knosys.2011.06.012
4. Luo C, Pang W, Wang Z. Semi-supervised clustering on heterogeneous information networks. In: Advances in Knowledge Discovery and Data Mining; 2014. p. 548-559.
5. Yang HJ, Kim YE, Yun JY, Kim HJ, Jeon BS. Identifying the clusters within nonmotor manifestations in early Parkinson's disease by using unsupervised cluster analysis. PLoS One. 2014;9(3):e91906. doi: 10.1371/journal.pone.0091906 PMID: 24643014
6. Bogner C, Trancon Y Widemann B, Lange H. Characterising flow patterns in soils by feature extraction and multiple consensus clustering. Ecological Informatics. 2013;15:44-52. doi: 10.1016/j.ecoinf.2013. 03.001
7. Zhang W, Yoshida T, Tang X, Wang Q. Text clustering using frequent itemsets. Knowledge-Based Systems. 2010;23(5):379-388. doi: 10.1016/j.knosys.2010.01.011
8. Saeed F, Salim N, Abdo A. Information theory and voting based consensus clustering for combining multiple clusterings of chemical structures. Molecular Informatics. 2013;32(7):591-598. doi: 10.1002/minf.201300004 PMID: 27481767
9. Berkhin P. A survey of clustering data mining techniques. In: Grouping Multidimensional Data. Springer-Verlag, Heidelberg; 2006. p. 25-71.
10. Lloyd SP. Least squares quantization in PCM. IEEE Transactions on Information Theory. 1982;28(2):129-137. doi: 10.1109/TIT.1982.1056489
11. Van Rooden SM, Heiser WJ, Kok JN, Verbaan D, Van Hilten JJ, Marinus J. The identification of Parkinson's disease subtypes using cluster analysis: a systematic review. Movement Disorders. 2010;25(8):969-978. doi: 10.1002/mds.23116 PMID: 20535823
12. Fei-Fei L, Perona P. A Bayesian hierarchical model for learning natural scene categories. In: IEEE Computer Society Conference Computer Vision and Pattern Recognition, CVPR 2005. vol. 2; 2005. p. 524-531.
13. Sung KK, Poggio T. Example-based learning for view-based human face detection. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1998;20(1):39-51. doi: 10.1109/34.655648
14. Hjort NL, Holmes C, Muller P, Walker SG. Bayesian nonparametrics. vol. 28. Cambridge University Press; 2010.
15. Blei DM, Jordan MI. Variational methods for the Dirichlet process. In: The 21st International Conference on Machine Learning, ICML 2004. ACM; 2004. p. 12.
16. Neal RM. Markov chain sampling methods for Dirichlet process mixture models. Journal of Computational and Graphical Statistics. 2000;9(2):249-265. doi: 10.2307/1390653
17. Neal RM. Slice sampling. Annals of Statistics. 2003;31(3):705-767. doi: 10.1214/aos/1056562461
18. Bishop CM. Pattern Recognition and Machine Learning. Springer-Verlag, New York; 2006.
19. Dempster AP, Laird NM, Rubin DB. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B. 1977;39:1-38.
20. Jiang K, Kulis B, Jordan MI. Small-variance asymptotics for exponential family Dirichlet process mixture models. In: Advances in Neural Information Processing Systems; 2012. p. 3158-3166.
21. Pelleg D, Moore AW. X-means: extending K-means with efficient estimation of the number of clusters. In: ICML 2000;2000. p. 727-734.
22. Bischof H, Leonardis A, Selb A. MDL principle for robust vector quantisation. Pattern Analysis & Applications. 1999;2(1):59-72. doi: 10.1007/s100440050015
23. Hamerly G, Elkan C. Learning the K in K-means. In: Advances in Neural Information Processing Systems; 2003.
24. Gao H, Bryc K, Bustamante CD. On identifying the optimal number of population clusters via the deviance information criterion. PLoS One. 2011;6(6):e21014. doi: 10.1371/journal.pone.0021014 PMID: 21738600
25. Welling M, Kurihara K. Bayesian K-means as a "Maximization-Expectation" Algorithm. In: SDM. SIAM; 2006. p. 474-478.
26. Teh YW. Dirichlet process. In: Encyclopedia of Machine Learning. Springer, US; 2010. p. 280-287.
27. MacEachern SN, Muller P. Estimating mixture of Dirichlet process models. Journal of Computational and Graphical Statistics. 1998;7(2):223-238. doi: 10.1080/10618600.1998.10474772
28. Jain S, Neal RM. Splitting and merging components of a nonconjugate Dirichlet process mixture model. Bayesian Analysis. 2007;2(3):445-472. doi: 10.1214/07-BA219
29. Casella G, Robert CP. Rao-Blackwellisation of sampling schemes. Biometrika. 1996;83(1):81-94. doi: 10.1093/biomet/83.1.81
30. Blackwell D. Conditional expectation and unbiased sequential estimation. The Annals of Mathematical Statistics. 1947; p. 105-110. doi: 10.1214/aoms/1177730497
31. Gershman SJ, Blei DM. A tutorial on Bayesian nonparametric models. Journal of Mathematical Psychology. 2012;56(1):1-12. doi: 10.1016/j.jmp. 2011.08.004
32. Arthur D, Vassilvitskii S. k++-means The advantages of careful seeding. In: Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms. Society for Industrial and Applied Mathematics; 2007. p. 1027-1035.
33. Kurlan R, Murphy D. Parkinson's disease data and organizing center. Movement Disorders. 2007;22(6):904. doi: 10.1002/mds.21415 PMID: 17343272
34. Reijnders JSAM, Ehrt U, Lousberg R, Aarsland D, Leentjens AFG. The association between motor subtypes and psychopathology in Parkinson's disease. Parkinsonism & Related Disorders. 2009;15(5):379-382. doi: 10.1016/j.parkreldis.2008.09.003
35. Lewis SJG, Foltynie T, Blackwell AD, Robbins TW, Owen AM, Barker RA. Heterogeneity of Parkinson's disease in the early clinical stages using a data driven approach. Journal of Neurology, Neurosurgery & Psychiatry. 2005;76(3):343-348. doi: 10.1136/jnnp. 2003.033530
36. Liu P, Feng T, Wang YJ, Zhang X, Chen B. Clinical heterogeneity in patients with early-stage Parkinson's disease: a cluster analysis. Journal of Zhejiang University Science B. 2011;12(9):694-703. doi: 10.1631/jzus. B1100069 PMID: 21887844
37. Gasparoli E, Delibori D, Polesello G, Santelli L, Ermani M, Battistin L, et al. Clinical predictors in Parkinson's disease. Neurological Sciences. 2002;23(2):s77-s78. doi: 10.1007/s100720200078 PMID: 12548352
38. Hoehn MM, Yahr MD. Parkinsonism: onset, progression and mortality. Neurology. 1967;50(2):318-318.
39. Bouveyron C, Girard S, Schmid C. High-dimensional data clustering. Computational Statistics & Data Analysis. 2007;52(1):502-519. doi: 10.1016/j.csda.2007.02.009
40. Rabiner L, Juang B. An introduction to hidden Markov models. ieee assp magazine. 1986;3(1):4-16. doi: 10.1109/MASSP.1986.1165342
41. Raykov, YP, Boukouvalas, A, Little, MA. Iterative collapsed MAP inference for Bayesian nonparametrics;.
42. Pitman J, Yor M. The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator. The Annals of Probability. 1997; p. 855-900.
43. Teh YW, Jordan MI, Beal MJ, Blei DM. Hierarchical dirichlet processes. Journal of the American Statistical Association. 2006;101 (476). doi: 10.1198/016214506000000302
44. Beal MJ, Ghahramani Z, Rasmussen CE. The infinite hidden Markov model. In: Advances in Neural Information Processing Systems. vol. 14. MIT Press; 2002.
45. Griffiths TL, Ghahramani Z. The Indian buffet process: an introduction and review. The Journal of Machine Learning Research. 2011;12:1185-1224.
46. Rubin DB. Inference and missing data. Biometrika. 1976;63(3):581-592. doi: 10.1093/biomet/63.3. 581
47. Molenberghs G, Beunckens C, Sotto C, Kenward MG. Every missingness not at random model has a missingness at random counterpart with equal fit. Journal of the Royal Statistical Society: Series B. 2008;70(2):371-388. doi: 10.1111/j.1467-9868.2007.00640.x
48. Morris CN. Parametric empirical Bayes inference: theory and applications. Journal of the American Statistical Association. 1983;78(381):47-55. doi: 10.1080/01621459.1983.10477920
49. Snoek J, Larochelle H, Adams RP. Practical Bayesian optimization of machine learning algorithms. In: Advances in Neural Information Processing Systems; 2012. p. 2951-2959.
50. Rasmussen CE. The infinite Gaussian mixture model. In: Advances in Neural Information Processing Systems. vol. 12; 1999. p. 554-560.