Deterministic feature selection for k-means clustering

IEEE Transactions on Information Theory

Volumn 59, Issue 9, 2013, Pages 6099-6110

  Boutsidis, Christos ^a   Magdon Ismail, Malik ^b  

^a IBM T J WATSON RESEARCH CENTER   (United States)

^b RENSSELAER POLYTECHNIC INSTITUTE   (United States)

Author keywords

Clustering methods; dimensionality reduction; unsupervised learning

Indexed keywords

CLUSTERING METHODS; DETERMINISTIC METHODS; DIMENSIONALITY REDUCTION; EMPIRICAL PERFORMANCE; FEATURE SELECTION ALGORITHM; K-MEANS CLUSTERING; RELATIVE ERRORS;

COMPUTER APPLICATIONS; INFORMATION THEORY; UNSUPERVISED LEARNING;

ALGORITHMS;

EID: 84882757134     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2013.2255021     Document Type: Article

References (41)

1. A. Aggarwal, A. Deshpande, and R. Kannan, "Adaptive sampling for k-means clustering," in Proc. 12th Int. Workshop 13th Int. Workshop Approximation, Randomization, Combinatorial Optimization, 2009, pp. 15-28.
2. D. Arthur and S. Vassilvitskii, "k-Means++: The advantages of careful seeding," in Proc. 18th Annu. ACM-SIAM Symp. Discrete Algorithms, 2007, pp. 1027-1035.
3. J. Batson, D. Spielman, and N. Srivastava, "Twice-Ramanujan sparsifiers," in Proc. ACM Symp. Theory Comput., 2009, pp. 255-262.
4. C. Boutsidis, P. Drineas, and M. Magdon-Ismail, "Near-optimal column-based matrix reconstruction," in Proc. IEEE 52nd Annu. Symp. Foundation Comput. Sci., 2011, pp. 305-314.
5. C. Boutsidis, P. Drineas, and M. Magdon-Ismail, Near-optimal column-based matrix reconstruction Technical Report, ArXiv:1103.0995.
6. C. Boutsidis, M. W. Mahoney, and P. Drineas, "Unsupervised feature selection for the -means clustering problem," in Proc. Adv. Neural Inf. Process. Syst., 2009, pp. 153-161.
7. C. Boutsidis, A. Zouzias, and P. Drineas, "Random projections for -means clustering," presented at the Adv. Neural Inf. Process. Syst., 2010.
8. C. Boutsidis,A. Zouzias, M. W. Mahoney, and P.Drineas, "Stochastic dimensionality reduction for -means clustering," in Manuscript, Under Review. :, 2011.
9. Y. Cui and J. G. Dy, Orthogonal principal feature selection.
10. A. D. Aspremont, L. E. Ghaoui,M. Jordan, and G. Lanckriet, "A direct formulation for sparse PCA using semidefinite programming," SIAM Rev., vol. 49, no. 3, pp. 434-434, 2007.
11. P. Drineas, M. Mahoney, and S. Muthukrishnan, "Relative-error CUR matrix decompositions," SIAM J. Matrix Anal. Appl., vol. 30, pp. 844-881, 2008.
12. P. Drineas, A. Frieze, R. Kannan, S. Vempala, and V. Vinay, "Clustering in large graphs and matrices," in Proc. 10th Annu. ACM-SIAM Symp. Discrete Algorithms, 1999, pp. 291-299.
13. D. Feldman, M. Monemizadeh, and C. Sohler, "A PTAS for k-means clustering based on weak coresets," presented at the Annu. Symp. Comput. Geometry, 2007.
14. D. H. Foley and J. W. Sammon, "An optimal set of discriminant vectors," IEEE Trans. Comput., vol. C-24, no. 3, pp. 281-289, Mar. 1975.
15. G. Frahling and C. Sohler, "A fast k-means implementation using coresets," presented at the Annu. Symp. Comput. Geometry, 2006.
16. I. Guyon and A. Elisseeff, "An introduction to variable and feature selection," J. Mach. Learn. Res., vol. 3, pp. 1157-1182, 2003.
17. T.N. E. Greville, "Note on the generalized inverse of a matrix product," SIAM Rev., vol. 8, no. 4, pp. 518-521, 1966.
18. G. H. Golub and C. F. Van Loan, Matrix Computations. Baltimore, MD, USA: The Johns Hopkins Univ. Press, 1996.
19. I. Guyon, S. Gunn, A. Ben-Hur, and G. Dror, "Result analysis of the NIPS 2003 feature selection challenge," in Proc. Adv. Neural Inf. Process. Syst., 2005, pp. 545-552.
20. S. Har-Peled and A. Kushal, "Smaller coresets for -median and -means clustering," in Proc. 21st Annu. Symp. Comput. Geometry, 2005, pp. 126-134. (Pubitemid 43283453)
21. S. Har-Peled and S. Mazumdar, "On coresets for k-means and k-median clustering," in Proc. 36th Annu. ACM Symp. Theory Comput., 2004, pp. 291-300.
22. X. He, D. Cai, and P. Niyogi, "Laplacian score for feature selection," presented at the Adv. Neural Inf. Process. Syst., 2006.
23. R. Jaiswal1,A.Kumar, and S. Sen,A simple -sampling based PTAS for k-means and other Clustering problems 2012 [Online]. Available: http://arxiv.org/pdf/ 1201.4206.pdf
24. I. Jolliffe, "Discarding variables in a principal component analysis - Part I: Artificial data," Appl. Statist., vol. 21, no. 2, pp. 160-173, 1972.
25. I. Jolliffe, "Discarding variables in a principal component analysis - Part II: Real data," Appl. Statist., vol. 22, no. 1, pp. 21-31, 1973.
26. W. Krzanowski, "Selection of variables to preserve multivariate data structure, using principal components," Appl. Statist., vol. 36, no. 1, pp. 22-33, 1987.
27. A. Kumar, Y. Sabharwal, and S. Sen, "A simple linear time -approximation algorithm for k-means clustering in any dimensions," in Proc. 45th Annu. IEEE Symp. Found. Comput. Sci., 2004, pp. 454-462. (Pubitemid 40575306)
28. S. Lloyd, "Least squares quantization in PCM," IEEE Trans. Inf. Theory, vol. 28, no. 2, pp. 129-137, Mar. 1982.
29. Y. Lu, I. Cohen, X. S. Zhou, and Q. Tian, "Feature selection using principal feature analysis," in Proc. 15th Int. Conf. Multimedia, 2007, pp. 301-304.
30. M. Magdon-Ismail, Row sampling for matrix algorithms via a noncommutative Bernstein bound 2010 [Online]. Available: http://arxiv. org/abs/1008.0587
31. A. Malhi and R. Gao, "PCA-based feature selection scheme for machine defect classification," IEEE Trans. Instrum. Meas., vol. 53, no. 6, pp. 1517-1525, Dec. 2004.
32. K.Mao, "Identifying critical variables of principal components for unsupervised feature selection," IEEE Trans. Syst., Man, Cybern., vol. 35, no. 2, pp. 339-344, Apr. 2005. (Pubitemid 40535916)
33. R. Ostrovsky and Y. Rabani, "Polynomial time approximation schemes for geometric -clustering," in Proc. 41st Annu. Symp. Found. Comput. Sci., 2000, pp. 349-358. (Pubitemid 32075904)
34. R. Ostrovsky, Y. Rabani, L. J. Schulman, and C. Swamy, "The effectiveness of Lloyd-type methods for the k-means problem," in Proc. IEEE 47th Annu. Symp. Found. Comput. Sci., 2006, pp. 165-176.
35. M. Rudelson, "Random vectors in the isotropic position," J. Funct. Anal., vol. 164, no. 1, pp. 60-72, 1999.
36. M. Rudelson and R. Vershynin, "Sampling from large matrices: An approach through geometric functional analysis," J. ACM, vol. 54, no. 4, 2007.
37. A. Smola and B. Schölkopf, "Sparse greedy matrix approximation for machine learning," in Proc. 7th Int. Conf. Mach. Learn., 2000, pp. 911-918.
38. J. Vaidya and C. Clifton, "Privacy-preserving k-means clustering over vertically partitioned data," in Proc. 9th ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining, 2003, pp. 206-215.
39. A.Wang and E. A. Gehan, "Gene selection formicroarray data analysis using principal component analysis," Stat. Med., vol. 24, no. 13, pp. 2069-2087, Jul. 2005. (Pubitemid 40895560)
40. X.Wu et al., "Top 10 algorithms in data mining," Knowl. Inf. Syst., vol. 14, no. 1, pp. 1-37, 2008.
41. D. Zhang, S. Chen, and Z.-H. Zhou, "Constraint score: A new filter method for feature selection with pairwise constraints," Pattern Recognit., vol. 41, no. 5, pp. 1440-1451, 2008.