Approximating K-means-type clustering via semidefinite programming

SIAM Journal on Optimization

Volumn 18, Issue 1, 2007, Pages 186-205

  Peng, Jiming ^a   Wei, Yu ^a  

^a MCMASTER UNIVERSITY   (Canada)

Author keywords

0 1 SDP; Approximation; Computational complexity; K means clustering; Principal component analysis; Relaxation

Indexed keywords

APPROXIMATION THEORY; COMPUTATIONAL COMPLEXITY; LINEAR PROGRAMMING; NUMERICAL METHODS; PRINCIPAL COMPONENT ANALYSIS; PROBLEM SOLVING; RELAXATION PROCESSES;

SEMIDEFINITE PROGRAMMING (SDP); SPECTRAL CLUSTERING;

CLUSTERING ALGORITHMS;

EID: 39449098473     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/050641983     Document Type: Article

References (45)

1. P. K. AGARWAL AND PROCOPIUC, Exact and approximation algorithms for clustering. Algorithms, 33 (2002), pp. 201-226.
2. F. R. BACH AND M. I. JORDAN, Learning Spectral Clustering, Adv. Neural Inf. Process. Syst., 16, MIT Press, Cambridge, MA, 2004.
3. P. BRADLEY, K. BENNET, AND A. DEMIRIZ, Constrained K-means Clustering, Tech. report MSR-TR-2000-65, Microsoft Research, 2000.
4. P. S. BRADLEY, U. M. FAYYAD, AND O. L. MANGASARIAN, Mathematical programming for data mining: Formulations and challenges, INFORMS J. Comput., 11 (1999), pp. 217-238.
5. C. DING AND X. HE, K-means clustering via principal component analysis, in Proceedings of the 21st Annual International Conference on Machine Learning, Banff, Canada, 2004.
6. P. DRINEAS, Private communication, 2005.
7. P. DRINEAS, A. FRIEZE, R. KANNAN, R. VEMPALA, AND V. VINAY, Clustering large graphs via singular value decomposition, Machine Learning, 56 (2004), pp. 9-33.
8. O. DU MERLE, P. HANSEN, B. JAUMARD, AND N. MLADENOVIĆ, An interior-point algorithm for minimum sum-of-squares clustering, SIAM J. Sci. Comput., 21 (2000), pp. 1485-1505.
9. J. GHOSH, Scalable Clustering, in The Handbook of Data Mining, N. Ye, ed., Lawrence Erlbaum Associate, Inc., 2003, pp. 247-277.
10. M. X. GOEMANS, Semidefinite programming in combinatorial optimization, Math. Programming, 79 (1997), pp. 143-161.
11. G. GOLUB AND C. V. LOAN, Matrix Computations, John Hopkins University Press, Baltimore, MD, 1996.
12. A. D. GORDON AND J. T. HENDERSON, An algorithm for Euclidean sum of squares classification, Biometrics, 33 (1977), pp. 355-362.
13. M. GU, H. ZHA, C. DING, X. HE, AND H. SIMON, Spectral Relaxation Models and Structure Analysis for k-way Graph Clustering and Bi-clustering, Penn State University Technical Report, State College, PA, 2001.
14. P. HANSEN AND B. JAUMARD, Cluster analysis and mathematical programming, Math. Programming, 79(B) (1997), pp. 191-215.
15. P. HANSEN, B. JAUMARD, AND N. MLADENOVIĆ , Minimum sum of squares clustering in a low dimensional space, J. Classification, 15 (1998), pp. 37-55.
16. S. HASEGAWA, H. IMAI, M. INABA, N. KATOH, AND J. NAKANO, Efficient algorithms for variance-based k-clustering, in Proceedings of the First Annual Pacific Conference on Computer Graphics and Applications (Seoul, Korea), Vol. 1, World Scientific, Singapore, 1993, pp. 75-89.
17. H. HOWARD, Classifying a population into homogeneous groups, in Operational Research in Social Science, J. R. Lawrence, ed., Tavistock Publications, London, 1966.
18. M. INABA, N. KATOH, AND H. IMAI, Applications of weighted Voronoi diagrams and randomization to variance-based k-clustering: Extended abstract, in Proceedings of the Tenth Annual Symposium on Computational Geometry, ACM, New York, 1994, pp. 332-339.
19. A. K. JAIN AND R. C. DUBES, Algorithms for Clustering Data, Prentice-Hall, Englewood Cliffs, NJ, 1988.
20. A. K. JAIN, M. N. MURTY, AND P. J. FLYNN, Data clustering: A review, ACM Computing Surveys, 31 (1999), pp. 264-323.
21. R. C. JANCEY, Multidimensional group analysis, Australian J. Botany, 14 (1966), pp. 127-130.
22. I. JOLLIFFE, Principal Component Analysis, 2nd ed., Springer-Verlag, New York, 2002.
23. L. KAUFMAN AND P. P. ROUSSEEUW, Finding Groups in Data: An Introduction to Cluster Analysis, John Wiley, New York, 1990.
24. S. E. KARISCH AND F. RENDL, Semidefinite programming and graph equipartition, Fields Inst. Commun., 18 (1998), pp. 77-95.
25. A. KUMAR, Y. SABHARWAL, AND S. SEN, A simple linear time (1 + ε)-approximation algorithm for k-means clustering in any dimensions, in Proceedings of the 45th Annual IEEE Symposium on Foundations of Computer Science (FOCS'04), IEEE Computer Society, Los Alamitos, CA, 2004, pp. 454-462.
26. A. LEISSER AND F. RENDL, Graph partitioning using linear and semidefinite programming, Math. Program. (B), 95 (2003), pp. 91-101.
27. A. MCCALLUM, Bow: A Toolkit for Statistical Language Modeling, Text Retrieval, Classification, and Clustering, available online at http://www.cs.umass.edu/~mccallum/bow/.
28. J. MCQUEEN, Some methods for classification and analysis of multivariate observations, Computer and Chemistry, 4 (1967), pp. 257-272.
29. O. L. MANGASARIAN, Nonlinear Programming, Classics Appl. Math. 10, SIAM, Philadelphia, 1994.
30. O. L. MANGASARIAN, Mathematical programming in data mining, Data Min. Knowl. Discov., 1 (1997), pp. 183-201.
31. M. MEILA AND J. SHI, A random walks view of spectral segmentation, International Workshop on AI and Stat., (2001).
32. R. S. MICHALSKI AND R. L. CHILAUSKY, Learning by being told and learning from examples: An experimental comparison of the two methods of knowledge acquisition in the context of developing an expert system for soybean disease diagnosis, International Journal of Policy Analysis and Information Systems, 4 (1980), pp. 125-161.
33. J. MATOUSEK, On approximate geometric k-clustering, Discrete Comput. Geom., 24 (2000), pp. 61-84.
34. A. Y. NG, M. I. JORDAN, AND Y. WEISS, On spectral clustering: Analysis and an algorithm, Proc. Neural Info. Processing Systems, NIPS, 14 (2001).
35. M. L. OVERTON AND R. S. WOMERSLEY, Optimality conditions and duality theory for minimizing sums of the largest eigenvalues of symmetric matrices, Math. Programming, 62 (1993), pp. 321-357.
36. J. M. PENG AND Y. XIA, A new theoretical framework for K-means clustering, in Foundation and Recent Advances in Data Mining, W. Chu and T. Lin, eds., Springer-Verlag, New York, 2005, pp. 79-98.
37. J. M. PENG AND Y. XIA, A cutting plane method for the minimum-sum-of-squared error clustering, in Proceedings of the SIAM International Conference on Data Mining, Newport Beach, CA, 2005.
38. G. MA, J. PENG, AND Y. WEI, On approximate balanced bi-clustering, in Computing and Combinatorics Conference, Lecture Notes in Comput. Sci. 3595, Springer-Verlag, Berlin, 2005, pp. 661-670.
39. E. H. RUSPINI, Numerical methods for fuzzy clustering, Inform. Sci., 2 (1970), pp. 319-350.
40. J. SHI AND J. MALIK, Normalized cuts and image segmentation, IEEE. Trans. Pattern Analysis and Machine Intelligence, 22 (2000), pp. 888-905.
41. H. SPÄTH, Cluster Analysis Algorithms for Data Reduction and Classification of Objects, Ellis Horwood Ltd., West Sussex, UK, 1980.
42. J. H. WARD, Hierarchical grouping to optimize an objective function, J. Amer. Statist. Assoc., 58 (1963), pp. 236-244.
43. Y. WEISS, Segmentation using eigenvectors: A unifying view, in Proceedings of the IEEE International Conference on Computer Vision, IEEE Computer Society, Los Alamitos, CA, 1999, pp. 975-982.
44. E. P. XING AND M. I. JORDAN, On Semidefinite Relaxation for Normalized k-cut and Connections to Spectral Clustering, Tech. report CSD-03-1265, UC Berkeley, Berkeley, CA, 2003.
45. H. ZHA, C. DING, M. GU, X. HE, AND H. SIMON, Spectral relaxation for K-means clustering, in Advances in Neural Information Processing Systems 14, T. Dietterich, S. Becker, and Z. Ghahramani, eds., MIT Press, Cambridge, MA, 2002, pp. 1057-1064.