Dimensionality reduction for k-means clustering and low rank approximation

Proceedings of the Annual ACM Symposium on Theory of Computing

Volumn 14-17-June-2015, Issue , 2015, Pages 163-172

  Cohen, Michael B ^a   Elder, Sam ^a   Musco, Cameron ^a   Musco, Christopher ^a   Persu, Madalina ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION THEORY; COMPUTATION THEORY; HEURISTIC ALGORITHMS; MATRIX ALGEBRA; PRINCIPAL COMPONENT ANALYSIS; REDUCTION;

APPROXIMATION PROBLEMS; COLUMN SELECTIONS; DIMENSIONALITY REDUCTION; JOHNSON LINDENSTRAUSS; K-MEANS CLUSTERING; LOW RANK APPROXIMATIONS; RELATIVE ERRORS; ROW PROJECTIONS;

DATA REDUCTION;

EID: 84958764795     PISSN: 07378017     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2746539.2746569     Document Type: Conference Paper

References (46)

1. D. Achlioptas. Database-friendly random projections: Johnson-Lindenstrauss with binary coins. J. Comput. Syst. Sci., 66(4), 2003.
2. D. Aloise, A. Deshpande, P. Hansen, and P. Popat. NP-hardness of Euclidean sum-of-squares clustering. Machine Learning, 75(2):245-248, 2009.
3. D. Arthur and S. Vassilvitskii. K-means++: The advantages of careful seeding. In Symposium on Discrete Algorithms (SODA), pages 1027-1035, 2007.
4. M. Asteris, D. Papailiopoulos, and A. Dimakis. Nonnegative sparse PCA with provable guarantees. In International Conference on Machine Learning (ICML), pages 1728-1736, 2014.
5. M.-F. Balcan, S. Ehrlich, and Y. Liang. Distributed k-means and k-median clustering on general topologies. In Advances in Neural Information Processing Systems (NIPS), pages 1995-2003, 2013.
6. M.-F. Balcan, V. Kanchanapally, Y. Liang, and D. P. Woodruff. Improved distributed principal component analysis. In Advances in Neural Information Processing Systems (NIPS), pages 3113-3121, 2014.
7. J. Batson, D. A. Spielman, and N. Srivastava. Twice-Ramanujan sparsifiers. SIAM Journal on Computing, 41(6):1704-1721, 2012.
8. C. Boutsidis, P. Drineas, and M. Magdon-Ismail. Near-optimal column-based matrix reconstruction. SIAM Journal on Computing, 43(2):687-717, 2014.
9. C. Boutsidis and M. Magdon-Ismail. Deterministic feature selection for k-means clustering. IEEE Transactions on Information Theory, 59(9), 2013.
10. C. Boutsidis, M. W. Mahoney, and P. Drineas. Unsupervised feature selection for the k-means clustering problem. In Advances in Neural Information Processing Systems (NIPS), pages 153-161, 2009.
11. C. Boutsidis and D. P. Woodruff. Optimal CUR matrix decompositions. In Symposium on Theory of Computing (STOC), pages 291-300, 2014.
12. C. Boutsidis, A. Zouzias, and P. Drineas. Random projections for k-means clustering. In Advances in Neural Information Processing Systems (NIPS), 2010.
13. C. Boutsidis, A. Zouzias, M. W. Mahoney, and P. Drineas. Randomized dimensionality reduction for k-means clustering. IEEE Transactions on Information Theory, 61(2):1045-1062, Feb 2015.
14. K. Clarkson and D. P. Woodruff. Numerical linear algebra in the streaming model. In Symposium on Theory of Computing (STOC), pages 205-214, 2009.
15. K. L. Clarkson and D. P. Woodruff. Low rank approximation and regression in input sparsity time. In Symposium on Theory of Computing (STOC), 2013.
16. M. B. Cohen, S. Elder, C. Musco, C. Musco, and M. Persu. Dimensionality reduction for k-means clustering and low rank approximation. Computing Research Repository (CoRR), abs/1410.6801, 2014.
17. M. B. Cohen, Y. T. Lee, C. Musco, C. Musco, R. Peng, and A. Sidford. Uniform sampling for matrix approximation. In Innovations in Theoretical Computer Science (ITCS), pages 181-190, 2015.
18. M. B. Cohen, J. Nelson, and D. P. Woodruff. Optimal approximate matrix product in terms of stable rank. Manuscript, 2014.
19. A. Deshpande, L. Rademacher, S. Vempala, and G. Wang. Matrix approximation and projective clustering via volume sampling. Theory of Computing, 2(1):225-247, 2006.
20. P. Drineas, A. Frieze, R. Kannan, S. Vempala, and V. Vinay. Clustering large graphs via the singular value decomposition. Machine Learning, 56(1-3), 2004.
21. D. Feldman, M. Schmidt, and C. Sohler. Turning big data into tiny data: Constant-size coresets for k-means, PCA, and projective clustering. In Symposium on Discrete Algorithms (SODA), 2013.
22. M. Ghashami and J. M. Phillips. Relative errors for deterministic low-rank matrix approximations. In Symposium on Discrete Algorithms (SODA), 2014.
23. V. Guruswami and A. K. Sinop. Optimal column-based low-rank matrix reconstruction. In Symposium on Discrete Algorithms (SODA), 2012.
24. N. Halko, P.-G. Martinsson, and J. A. Tropp. Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM Review, 53(2):217-288, 2011.
25. S. Har-Peled and A. Kushal. Smaller coresets for k-median and k-means clustering. Discrete and Computational Geometry, 37(1):3-19, 2007.
26. S. Har-Peled and S. Mazumdar. On coresets for k-means and k-median clustering. In Symposium on Theory of Computing (STOC), pages 291-300, 2004.
27. D. Hsu, S. Kakade, and T. Zhang. Tail inequalities for sums of random matrices that depend on the intrinsic dimension. Electron. Commun. Probab., 17:1-13, 2012.
28. M. Inaba, N. Katoh, and H. Imai. Applications of weighted Voronoi diagrams and randomization to variance-based k-clustering. In Symposium on Computational Geometry (SCG), pages 332-339, 1994.
29. D. Kane and J. Nelson. Sparser Johnson-Lindenstrauss transforms. Journal of the ACM, 61(1):4, 2014.
30. R. Kannan, S. S. Vempala, and D. P. Woodruff. Principal component analysis and higher correlations for distributed data. In Conference on Computational Learning Theory (COLT), pages 1040-1057, 2014.
31. T. Kanungo, D. M. Mount, N. S. Netanyahu, C. D. Piatko, R. Silverman, and A. Y. Wu. A local search approximation algorithm for k-means clustering. In Symposium on Computational Geometry (SCG), 2002.
32. A. Kumar, Y. Sabharwal, and S. Sen. A simple linear time (1 + e)-approximation algorithm for fc-means clustering in any dimensions. In Foundations of Computer Science (FOCS), pages 454-462, 2004.
33. M. Li, G. L. Miller, and R. Peng. Iterative row sampling. In Foundations of Computer Science (FOCS), pages 127-136, 2013.
34. Y. Liang, M.-F. Balcan, and V. Kanchanapally. Distributed PCA and fc-means clustering. In The Big Learning Workshop at Advances in Neural Information Processing Systems (NIPS), 2013.
35. E. Liberty. Simple and deterministic matrix sketching. In Knowledge Discovery and Data Mining (KDD), pages 581-588, 2013.
36. S. Lloyd. Least squares quantization in PCM. IEEE Transactions on Information Theory, 28(2), 1982.
37. M. Mahajan, P. Nimbhorkar, and K. Varadarajan. The planar fc-means problem is NP-hard. In International Workshop on Algorithms and Computation (WALCOM), pages 274-285, 2009.
38. M. W. Mahoney. Randomized algorithms for matrices and data. Foundations and Trends in Machine Learning, 3(2):123-224, 2011.
39. M. W. Mahoney and X. Meng. Low-distortion subspace embeddings in input-sparsity time and applications to robust linear regression. In Symposium on Theory of Computing (STOC), pages 91-100, 2013.
40. L. Mirsky. Symmetric gauge functions and unitarily invariant norms. The Quarterly Journal of Mathematics, 11:50-59, 1960.
41. J. Nelson and H. L. Nguyen. OSNAP: Faster numerical linear algebra algorithms via sparser subspace embeddings. In Foundations of Computer Science (FOCS), pages 117-126, 2013.
42. D. Papailiopoulos, A. Dimakis, and S. Korokythakis. Sparse PCA through low-rank approximations. In International Conference on Machine Learning (ICML), pages 747-755, 2013.
43. T. Sarlós. Improved approximation algorithms for large matrices via random projections. In Foundations of Computer Science (FOCS), pages 143-152, 2006.
44. D. P. Woodruff. Low rank approximation lower bounds in row-update streams. In Advances in Neural Information Processing Systems (NIPS), 2014.
45. D. P. Woodruff. Sketching as a tool for numerical linear algebra. Foundations and Trends in Theoretical Computer Science, 10(1-2):1-157, 2014.
46. X.-T. Yuan and T. Zhang. Truncated power method for sparse eigenvalue problems. The Journal of Machine Learning Research, 14(1):899-925, 2013.