Feature selection for linear SVM with provable guarantees

Journal of Machine Learning Research

Volumn 38, Issue , 2015, Pages 735-743

  Paul, Saurabh ^a   Magdon Ismail, Malik ^a   Drineas, Petros ^a  

^a RENSSELAER POLYTECHNIC INSTITUTE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE;

FEATURE SPACE; LINEAR SVM; MINIMUM ENCLOSING BALL; REAL-WORLD DATASETS; RELATIVE ERRORS; SELECTION TECHNIQUES; STATE OF THE ART; SUPPORT VECTOR;

FEATURE EXTRACTION;

EID: 84954316756     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (29)

1. J. Batson, D. Spielman, and N. Srivastava. Twiceramanujan sparsiérs. In Proceedings of the 41st annual ACM STOC, pages 255-262, 2009.
2. C. Bhattacharyya. Second order cone programming formulations for feature selection. JMLR, 5: 1417-1433, 2004.
3. C. Boutsidis and M. Magdon-Ismail. Deterministic feature selection for k-means clustering. IEEE Transactions on Information Theory, 59 (9): 6099-6110, 2013.
4. C. Boutsidis, M. W. Mahoney, and P. Drineas. Unsupervised feature selection for the k-means clustering problem. Advances in Neural Information Processing Sys-tems (NIPS), 2009.
5. C.-C. Chang and C.-J. Lin. Libsvm: A library for support vector machines. ACM Transactions on Intelligent Sys-tems and Technology, 2: 27: 1-27: 27, 2011. Software available at http://www.csie.ntu.edu.tw/cjlin/libsvm.
6. N. Cristianini and J. Shawe-Taylor. Support Vector Ma-chines and other kernel-based learning methods. Cambridge University Press, 2000.
7. A. Dasgupta, P. Drineas, B. Harb, V. Josifovski, and M. Mahoney. Feature selection methods for text classification. In Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 230-239, 2007.
8. D. Davidov, E. Gabrilovich, and S. Markovitch. Parameterized generation of labeled datasets for text categorization based on a hierarchical directory. In Proceedings of the 27th Annual International ACM SIGIR Conference, pages 250-257, 2004. http://techtc.cs.technion.ac.il/techtc300/techtc300.html.
9. H. Do, A. Kalousis, and M. Hilario. Feature weighting using margin and radius based error bound optimization in svms. In European Conference on Machine Learning (ECML), pages 315-329, 2009b.
10. R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin. Liblinear: A library for large linear classification. JMLR, pages 1871-1874, 2008.
11. M. G. Fung and O. Mangasarian. A feature selection newton method for support vector machine classification. Comput. Optim. Appl., 28 (2): 185-202, 2004.
12. R. Gilad-Bachrach, A. Navot, and N. Tishby. Margin based feature selection-theory and algorithms. In Proceedings of the twenty-first international conference on Machine learning (ICML), page 43, 2004.
13. I. Guyon, J. Weston, S. Barnhill, and V. Vapnik. Gene selection for cancer classification using support vector machines. Machine Learning, 46 (1-3): 389-422, 2002.
14. A. Kalousis and H. Do. Convex formulations of radiusmargin based support vector machines. In Proceedings of the 30th International Conference on Machine Learning (ICML), pages 169-177, 2013.
15. D. D. Lewis, Y. Yang, T. G. Rose, and F. Li. Rcv1: A new benchmark collection for text categorization research. JMLR, pages 361-397, 2004.
16. C. Park, K.-R. Kim, R. Myung, and J.-Y. Koo. Oracle properties of scad-penalized support vector machine. Journal of Statistical Planning and Inference, 142 (8): 2257-2270, 2012.
17. S. Paul and P. Drineas. Deterministic feature selection for regularized least squares classification. In Ma-chine Learning and Knowledge Discovery in Databases (ECML-PKDD), volume 8725 of LNCS, pages 533-548, 2014.
18. S. Paul, C. Boutsidis, M. Magdon-Ismail, and P. Drineas. Random projections for support vector machines. In Sixteenth International Conference on Artificial Intelli-gence and Statistics (AISTATS), pages 498-506. JMLR W&CP 31, 2013.
19. S. Paul, C. Boutsidis, M. Magdon-Ismail, and P. Drineas. Random projections for linear support vector machines. ACM Transactions on Knowledge Discovery from Data, 8 (4): 22: 1-22: 25, 2014.
20. A. Rakotomamonjy. Variable selection using svm based criteria. JMLR, 3: 1357-1370, 2003.
21. M. Rudelson and R. Vershynin. Sampling from large matrices: An approach through geometric functional analysis. J. ACM, 54 (4), 2007.
22. M. Tan, L. Wang, and I. Tsang. Learning sparse svm for feature selection on very high dimensional datasets. In Proceedings of the 27th International Conference on Ma-chine Learning (ICML), pages 1047-1054, 2010.
23. V. Vapnik. Statistical Learning Theory. Theory of Proba-bility and its Applications, 16: 264-280, 1998.
24. V. Vapnik and A. Chervonenkis. On the Uniform Convergence of Relative Frequencies of Events to their Probabilities. Theory of Probability and its Applications, 16: 264-280, 1971.
25. L. Wang, J. Zhu, and H. Zou. The doubly regularized support vector machine. Statistica Sinica, 16 (2): 589, 2006.
26. L. Wang, J. Zhu, and H. Zou. Hybrid huberized support vector machines for microarray classification and gene selection. Bioinformatics, 24 (3): 412-419, 2008.
27. J. Weston, S. Mukherjee, O. Chapelle, M. Pontil, T. Poggio, and V. Vapnik. Feature selection for svms. In NIPS, volume 12, pages 668-674, 2000.
28. J. Weston, A. Elisseé, B. Schölkopf, and M. Tipping. Use of the zero norm with linear models and kernel methods. JMLR, 3: 1439-1461, 2003.
29. G. Ye, Y. Chen, and X. Xie. Efficient variable selection in support vector machines via the alternating direction method of multipliers. In AISTATS, pages 832-840, 2011.