Randomized dimensionality reduction for κ-means clustering

IEEE Transactions on Information Theory

Volumn 61, Issue 2, 2015, Pages 1045-1062

  Boutsidis, Christos ^a   Zouzias, Anastasios ^b,c   Mahoney, Michael W ^d   Drineas, Petros ^e  

^a YAHOO RESEARCH   (United States)

^b UNIVERSITY OF TORONTO   (Canada)

^c IBM RESEARCH ZURICH   (Switzerland)

^d UNIVERSITY OF CALIFORNIA   (United States)

^e Rensselaer Polytechnic Institute   (United States)

Author keywords

clustering; dimensionality reduction; randomized algorithms

Indexed keywords

APPROXIMATION ALGORITHMS; EXTRACTION; FEATURE EXTRACTION; HEURISTIC METHODS; REDUCTION; SINGULAR VALUE DECOMPOSITION;

CLUSTERING; CONSTANT FACTOR APPROXIMATION; DIMENSIONALITY REDUCTION; FEATURE EXTRACTION METHODS; FEATURE SELECTION METHODS; MEANS CLUSTERING; RANDOM PROJECTIONS; RANDOMIZED ALGORITHMS;

CLUSTERING ALGORITHMS;

EID: 84921498474     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2014.2375327     Document Type: Article

References (35)

1. J. A. Hartigan, Clustering Algorithms. New York, NY, USA: Wiley, 1975.
2. S. Lloyd, "Least squares quantization in PCM," IEEE Trans. Inf. Theory, vol. 28, no. 2, pp. 129-137, Mar. 1982.
3. R. Ostrovsky, Y. Rabani, L. J. Schulman, and C. Swamy, "The effectiveness of Lloyd-type methods for the κ-means problem," in Proc. 47th Annu. IEEE Symp. Found. Comput. Sci. (FOCS), Oct. 2006, pp. 165-176.
4. X. Wu et al., "Top 10 algorithms in data mining," Knowl. Inf. Syst., vol. 14, no. 1, pp. 1-37, 2008.
5. I. Guyon, S. Gunn, A. Ben-Hur, and G. Dror, "Result analysis of the NIPS 2003 feature selection challenge," in Neural Information Processing Systems. Red Hook, NY, USA: Curran & Associates Inc., 2005.
6. R. Ostrovsky and Y. Rabani, "Polynomial time approximation schemes for geometric k-clustering," in Proc. 41st Annu. IEEE Symp. Found. Comput. Sci. (FOCS), 2000, pp. 349-358.
7. A. Kumar, Y. Sabharwal, and S. Sen, "A simple linear time (1 +∈)- approximation algorithm for κ-means clustering in any dimensions," in Proc. 45th Annu. IEEE Symp. Found. Comput. Sci. (FOCS), 2004, pp. 454-462.
8. S. Har-Peled and S. Mazumdar, "On coresets for κ-means and κ-median clustering," in Proc. 36th Annu. ACM Symp. Theory Comput. (STOC), 2004, pp. 291-300.
9. S. Har-Peled and A. Kushal, "Smaller coresets for κ-median and κ-means clustering," in Proc. 21st Annu. Symp. Comput. Geometry (SoCG), 2005, pp. 126-134.
10. G. Frahling and C. Sohler, "A fast κ-means implementation using coresets," in Proc. 22nd Annu. Symp. Comput. Geometry (SoCG), 2006, pp. 135-143.
11. D. Arthur and S. Vassilvitskii, "κ-means++: The advantages of careful seeding," in Proc. 18th Annu. ACM-SIAM Symp. Discrete Algorithms (SODA), 2007, pp. 1027-1035.
12. P. Drineas, A. Frieze, R. Kannan, S. Vempala, and V. Vinay, "Clustering in large graphs and matrices," in Proc. 10th Annu. ACM-SIAM Symp. Discrete Algorithms (SODA), 1999, pp. 291-299.
13. D. Feldman, M. Schmidt, and C. Sohler, "Turning big data into tiny data: Constant-size coresets for κ-means, PCA and projective clustering," in Proc. 24th Annu. ACM-SIAM SODA, 2013, pp. 1434-1453.
14. I. Guyon and A. Elisseeff, "An introduction to variable and feature selection," J. Mach. Learn. Res., vol. 3, pp. 1157-1182, Mar. 2003.
15. W. B. Johnson and J. Lindenstrauss, "Extensions of Lipschitz mappings into a Hilbert space," Contemp. Math., vol. 26, pp. 189-206, 1984.
16. C. Boutsidis, M. W. Mahoney, and P. Drineas, "Unsupervised feature selection for the κ-means clustering problem," in Neural Information Processing Systems. Red Hook, NY, USA: Curran & Associates Inc., 2009.
17. M. Rudelson and R. Vershynin, "Sampling from large matrices: An approach through geometric functional analysis," J. ACM, vol. 54, no. 4, 2007, Art. ID 21.
18. T. Sarlos, "Improved approximation algorithms for large matrices via random projections," in Proc. 47th Annu. IEEE Symp. Found. Comput. Sci. (FOCS), Oct. 2006, pp. 143-152.
19. C. Boutsidis, P. Drineas, and M. Magdon-Ismail. (2011). "Near optimal column based matrix reconstruction." [Online]. Available: http://arxiv.org/abs/1103.0995
20. M. D. Vose, "A linear algorithm for generating random numbers with a given distribution," IEEE Trans. Softw. Eng., vol. 17, no. 9, pp. 972-975, Sep. 1991.
21. M. Magdon-Ismail. (2010). "Row sampling for matrix algorithms via a non-commutative bernstein bound." [Online]. Available: http://arxiv.org/abs/1008.0587
22. P. Indyk and R. Motwani, "Approximate nearest neighbors: Towards removing the curse of dimensionality," in Proc. 30th Annu. ACM Symp. Theory Comput. (STOC), 1998, pp. 604-613.
23. N. Ailon and B. Chazelle, "Approximate nearest neighbors and the fast Johnson-Lindenstrauss transform," in Proc. 38th Annu. ACM Symp. Theory Comput. (STOC), 2006, pp. 557-563.
24. D. Achlioptas, "Database-friendly random projections: Johnson-Lindenstrauss with binary coins," J. Comput. Syst. Sci., vol. 66, no. 4, pp. 671-687, 2003.
25. E. Liberty and S. W. Zucker, "The Mailman algorithm: A note on matrix-vector multiplication," Inf. Process. Lett., vol. 109, no. 3, pp. 179-182, 2009.
26. MATLAB, 7.13.0.564 (R2011b), MathWorks, Natick, MA, USA, 2010.
27. X. He, D. Cai, and P. Niyogi, "Laplacian score for feature selection," in Neural Information Processing Systems, Y. Weiss, B. Schölkopf, and J. Platt, Eds. Red Hook, NY, USA: Curran & Associates Inc., 2006, pp. 507-514.
28. Feature Ranking Using Laplacian Score. [Online]. Available: http://www.cad.zju.edu.cn/home/dengcai/Data/MCFS.html, accessed Jun. 4, 2013.
29. K. Bache and M. Lichman. (2013). "UCI machine learning repository," School Inf. Comput. Sci., Univ. California, Irvine, CA, USA, Tech. Rep. [Online]. Available: http://archive.ics.uci.edu/ml
30. S. A. Nene, S. K. Nayar, and H. Murase. (Feb. 1996). "Columbia University image library," Tech. Rep. CUCS-005-96. [Online]. Available: http://www.cs.columbia.edu/CAVE/software/softlib/coil-20.php
31. AT&T Lab., Cambridge, U.K. The ORL Database of Faces. [Online]. Available: http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html, accessed Jan 1, 2013.
32. Carnegie Mellon Univ. Pie Database. [Online]. Available: http://www.ri.cmu.edu/research-project-detail.html?project-id=418&menu-id=261, accessed Jan 1, 2013.
33. P. Drineas, R. Kannan, and M. Mahoney, "Fast Monte Carlo algorithms for matrices I: Approximating matrix multiplication," SIAM J. Comput., vol. 36, no. 1, pp. 132-157, 2006.
34. K. L. Clarkson, "Tighter bounds for random projections of manifolds," in Proc. 24th Annu. Symp. Comput. Geometry (SoCG), 2008, pp. 39-48.
35. S. Arora, E. Hazan, and S. Kale, "A fast random sampling algorithm for sparsifying matrices," in Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (Lecture Notes in Computer Science), vol. 4110. Berlin, Germany: Springer-Verlag, 2006, pp. 272-279. [Online]. Available: http://dx.doi.org/10.1007/11830924-26