Support vector machines with constraints for sparsity in the primal parameters

IEEE Transactions on Neural Networks

Volumn 22, Issue 8, 2011, Pages 1269-1283

  Gomez Verdejo, Vanessa ^a   Martinez Ramon, Manel ^a   Arenas Garcia, Jernim ^a   Lazaro Gredilla, Miguel ^b   Molina Bulla, Harol ^a  

^a UNIVERSIDAD CARLOS III DE MADRID   (Spain)

^b UNIVERSITY OF CANTABRIA   (Spain)

Author keywords

Feature group selection; feature selection; margin maximization; multiclass classification; support vector machines

Indexed keywords

COMPUTATIONAL BURDEN; FEATURE GROUPS; LINEAR FEATURE; MARGIN MAXIMIZATION; MULTI-CLASS CLASSIFICATION; SPARSE CLASSIFIERS; SPARSE SOLUTIONS; STATE OF THE ART; SUPPORT VECTOR; SYNTHETIC AND REAL DATA;

FEATURE EXTRACTION;

SUPPORT VECTOR MACHINES;

ALGORITHM; ARTICLE; AUTOMATED PATTERN RECOGNITION; COMPARATIVE STUDY; METHODOLOGY; STATISTICAL MODEL; SUPPORT VECTOR MACHINE;

ALGORITHMS; LINEAR MODELS; PATTERN RECOGNITION, AUTOMATED; SUPPORT VECTOR MACHINES;

EID: 80051545883     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2011.2148727     Document Type: Article

References (40)

1. B. Schölkopf and A. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond. Cambridge, MA: MIT Press, 2001.
2. N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge, U.K.: Cambridge Univ. Press, 2000.
3. D. J. Sebald and J. A. Bucklew, "Support vector machine techniques for nonlinear equalization," IEEE Trans. Signal Process., vol. 48, no. 11, pp. 3217-3226, Nov. 2000. (Pubitemid 32027347)
4. M. M. Ramon, N. Xu, and C. Christodoulou, "Beamforming using support vector machines," IEEE Antennas Wireless Propag. Lett., vol. 4, pp. 439-442, Dec. 2005. (Pubitemid 43131829)
5. Z. Shi and M. Han, "Support vector echo-state machine for chaotic time-series prediction," IEEE Trans. Neural Netw., vol. 18, no. 2, pp. 359-372, Mar. 2007. (Pubitemid 46522562)
6. T. Hastie, R. Tibshirani, and J. H. Friedman, The Elements of Statistical Learning. New York: Springer-Verlag, 2001.
7. Y. Lin, "Support vector machines and the Bayes rule in classification," Data Min. Knowl. Discov., vol. 6, no. 3, pp. 259-275, 2002. (Pubitemid 37113866)
8. S. Rosset, J. Zhu, and T. Hastie, "Margin maximizing loss functions," in Advances in Neural Information Processing Systems 16, S. Thrun, L. Saul, and B. Schölkopf, Eds. Cambridge, MA: MIT Press, 2004, pp. 1237-1246.
9. J. Friedman, T. Hastie, S. Rosset, R. Tibshirani, and J. Zhu, "Consistency in boosting: Discussion," Ann. Stat., vol. 32, no. 1, pp. 102-107, Feb. 2004.
10. H. Liu and H. Motoda, Feature Selection for Knowledge Discover and Data Mining. Norwell, MA: Kluwer, 1998.
11. I. Guyon and A. Elisseeff, "An introduction to variable and feature selection," J. Mach. Learn. Res., vol. 3, nos. 7-8, pp. 1157-1182, Oct.-Nov. 2003.
12. L. Yu and H. Liu, "Feature selection for high-dimensional data: A fast correlation-based filter solution," in Proc. 20th Int. Conf. Mach. Learn., Washington D.C., 2003, pp. 1-8.
13. R. Kohavi and G. John, "Wrappers for feature selection," Artif. Intell., vol. 97, nos. 1-2, pp. 273-324, 1997.
14. L. Wang, N. Zhou, and F. Chu, "A general wrapper approach to selection of class-dependent features," IEEE Trans. Neural Netw., vol. 19, no. 7, pp. 1267-1278, Jul. 2008.
15. R. Tibshirani, "Regression shrinkage and selection via the LASSO," J. Royal Stat. Soc., Ser. B, vol. 58, no. 1, pp. 267-288, 1994.
16. J. Weston, S. Mukherjee, O. Chapelle, M. Pontil, T. Poggio, and V. Vapnik, "Feature selection for SVMs," in Advances in Neural Information Processing Systems 13, T. L. T. Dietterich and V. Tresp, Eds. Cambridge, MA: MIT Press, 2000.
17. J. Bi, K. Bennett, M. Embrechts, C. Breneman, and M. Song, "Dimensionality reduction via sparse support vector machines," J. Mach. Learn. Res., vol. 3, pp. 1229-1243, Mar. 2003.
18. I. Guyon, J. Weston, S. Barnhill, and V. Vapnik, "Gene selection for cancer classification using support vector machines," Mach. Learn., vol. 46, nos. 1-3, pp. 389-422, 2002. (Pubitemid 34129977)
19. A. Rakotomamonjy, "Variable selection using SVM based criteria," J. Mach. Learn. Res., vol. 3, nos. 7-8, pp. 1357-1370, 2003.
20. J. Weston, F. Perez-Cruz, O. Bousquet, O. Chapelle, A. Elisseeff, and B. Scholkopf, "Feature selection and transduction for prediction of molecular bioactivity for drug design," Bioinformatics, vol. 19, no. 6, pp. 764-771, 2003. (Pubitemid 36511901)
21. Y. Aksu, D. J. Miller, G. Kesidis, and Q. X. Yang, "Margin- maximizing feature elimination methods for linear and nonlinear kernel-based discriminant functions," IEEE Trans. Neural Netw., vol. 21, no. 5, pp. 701-717, May 2010.
22. J. Friedman, T. Hastie, and R. Tibshirani, "Sparse inverse covariance estimation with the graphical LASSO," Biostat, vol. 9, no. 3, pp. 432-441, 2008. (Pubitemid 351882084)
23. C. Z. J. Huang and S. Ma, "Adaptive LASSO for sparse highdimensional regression models," Stat. Sinica, vol. 18, no. 374, pp. 1603-1618, 2008.
24. N. Meinshausen and B. Yu, "LASSO-type recovery of sparse representations for high-dimensional data," Ann. Stat., vol. 37, no. 1, pp. 246-270, 2009.
25. Y. Li, P. Namburi, Z. Yu, C. Guan, J. Feng, and Z. Gu, "Voxel selection in fMRI data analysis based on sparse representation," IEEE Trans. Biomed. Eng., vol. 56, no. 10, pp. 2439-2451, Oct. 2009.
26. P. Bradley and O. L. Mangasarian, "Feature selection via concave minimization and support vector machines," in Proc. 15th Int. Conf. Mach. Learn., 1998, pp. 82-90.
27. L. R. Grate, C. Bhattacharyya, M. I. Jordan, and I. S. Mian, "Simultaneous relevant feature identification and classification in high-dimensional spaces," in Algorithms in Bioinformatics (Lecture Notes in Computer Science), vol. 2452. New York: Springer-Verlag, 2002, pp. 1-9.
28. G. M. Fung and O. L. Mangasarian, "A feature selection Newton method for support vector machine classification," Comput. Optim. Appl., vol. 28, no. 2, pp. 185-202, 2004.
29. J. Zhu, S. Rosset, T. Hastie, and R. Tibshirani, "1-norm support vector machines," in Advances in Neural Information Processing Systems 16, S. Thrun, L. Saul, and B. Schölkopf, Eds. Cambridge, MA: MIT Press, 2004, pp. 49-56.
30. L. Wang, J. Zhu, and H. Zou, "The doubly regularized support vector machine," Stat. Sinica, vol. 16, no. 2, pp. 589-615, 2006. (Pubitemid 44085519)
31. J. Zhu and H. Zou, "Variable selection for the linear support vector machine," in Trends in Neural Computation (Studies in Computational Intelligence), vol. 35, K. Chen and L. Wang, Eds. Berlin, Germany: Springer-Verlag, 2007, pp. 35-59.
32. H. Zou and T. Hastie, "Regularization and variable selection via the elastic net," J. Royal Stat. Soc., vol. 67, no. 2, pp. 301-320, Apr. 2005. (Pubitemid 40465877)
33. H. Zou, "An improved 1-norm support vector machine for simultaneous classification and variable selection," in Proc. 11th Int. Conf. Artif. Intell. Stat., 2007, pp. 1-7.
34. M. Yuan and Y. Lin, "Model selection and estimation in regression with grouped variables," J. Royal Stat. Soc.: Ser. B (Stat. Methodol.), vol. 68, no. 1, pp. 49-67, Feb. 2006. (Pubitemid 43415335)
35. H. Zou and M. Yuan, "The F∞-norm support vector machine," Stat. Sinica, vol. 18, no. 1, pp. 379-398, 2008.
36. B. Schölkopf, A. J. Smola, R. C. Williamson, and P. L. Bartlett, "New support vector algorithms," Neural Comput., vol. 12, no. 5, pp. 1207-1245, May 2000.
37. J. Arenas-García and F. Pérez-Cruz, "Multi-class support vector machines: A new approach," in Proc. Int. Conf. Acoust., Speech, Signal Process., vol. 2. Hong Kong, Apr. 2003, pp. 781-784.
38. A. Frank and A. Asuncion. (2010). UCI Machine Learning Repository [Online]. Available: http://archive.ics.uci.edu/ml
39. I. Guyon, J. Weston, S. Barnhill, and V. Vapnik, "Gene selection for cancer classification using support vector machines," Mach. Learn., vol. 46, nos. 1-3, pp. 389-422, 2002. (Pubitemid 34129977)
40. I. Guyon, "Feature selection challenge," in Proc. Neural Inf. Process. Syst. Workshop Feature Extract., Dec. 2003, pp. 1-8.