Primal explicit max margin feature selection for nonlinear support vector machines

Pattern Recognition

Volumn 47, Issue 6, 2014, Pages 2153-2164

  Tayal, Aditya ^a   Coleman, Thomas F ^b   Li, Yuying ^a  

^a UNIVERSITY OF WATERLOO   (Canada)

^b UNIVERSITY OF WATERLOO   (Canada)

Author keywords

Alternating optimization; Embedded; Feature selection; Non convex optimization; Nonlinear; Support vector machine; Trust region method

Indexed keywords

EID: 84894355969     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2014.01.003     Document Type: Article

References (45)

1. C. Cortes, and V. Vapnik Support-vector networks Mach. Learn. 20 1995 273 297
2. V.N. Vapnik Statistical Learning Theory 1st ed. 1998 Wiley New York, NY, USA
3. B.E. Boser, I.M. Guyon, V.N. Vapnik, A training algorithm for optimal margin classifiers, in: Proceedings of the Fifth Annual Workshop on Computational Learning Theory, COLT'92, ACM, New York, NY, USA, 1992, pp. 144-152.
4. H. Byun, S.-W. Lee, Applications of support vector machines for pattern recognition: a survey, in: Pattern Recognition with Support Vector Machines, 2002, pp. 213-236.
5. B. Scholkopf̈, K. Tsuda, and J.P. Vert Kernel Methods in Computational Biology 2004 MIT Press Cambridge, MA, USA
6. I. Guyon, and A. Elisseeff An introduction to variable and feature selection J. Mach. Learn. Res. 3 2003 1157 1182
7. R. O'Hara, and M. Sillanpaa A review of Bayesian variable selection methods: what, how, and which Bayesian Anal. 4 2009 85 118
8. C.M. Bishop, Bayesian PCA, in: NIPS, The MIT Press, Denver, Colorado, USA, 1998, pp. 382-388.
9. S. Nakajima, M. Sugiyama, S.D. Babacan, On Bayesian PCA: automatic dimensionality selection and analytic solution, in: ICML, Omnipress, New York, NY, USA, 2011, pp. 497-504.
10. J. Li, and D. Tao On preserving original variables in Bayesian PCA with application to image analysis IEEE Trans. Image Process. 21 2012 4830 4843
11. P. Bickel, Y. Ritov, A. Tsybakov, Hierarchical selection of variables in sparse high-dimensional regression, 2008.
12. K. Kira, L. Rendell, A practical approach to feature selection, in: ML92: Proceedings of the Ninth International Workshop on Machine Learning, pp. 249-256.
13. M.R. Šikonja, and I. Kononenko Theoretical and empirical analysis of ReliefF and RReliefF Mach. Learn. 53 2003 23 69
14. E. Marchiori, Feature selection for classification with proteomic data of mixed quality, in: Proceedings of the 2005 IEEE Symposium on Computational Intelligence in Bioinformatics and Computational Biology, pp. 385-391.
15. I. Guyon, J. Weston, S. Barnhill, and V. Vapnik Gene selection for cancer classification using support vector machines Mach. Learn. 46 2002 389 422
16. P.S. Bradley, O.L. Mangasarian, Feature selection via concave minimization and support vector machines, in: Proceedings of the Fifteenth International Conference on Machine Learning (ICML '98), pp. 82-90.
17. J. Zhu, S. Rosset, T. Hastie, R. Tibshirani, 1-norm support vector machines, in: Neural Information Processing Systems, vol. 16.
18. J. Weston, A. Elisseeff, B. Scholkopf̈, and M. Tipping Use of the zero norm with linear models and kernel methods J. Mach. Learn. Res. 3 2003 1439 1461
19. G.M. Fung, and O.L. Mangasarian A feature selection Newton method for support vector machine classification Comput. Optim. Appl. 28 2004 185 202
20. A.B. Chan, N. Vasconcelos, G.R.G. Lanckriet, Direct convex relaxations of sparse SVM, in: Proceedings of the Twenty Fourth International Conference on Machine Learning, ICML'07, ACM, New York, NY, USA, 2007, pp. 145-153.
21. M. Tan, L. Wang, I.W. Tsang, Learning sparse SVM for feature selection on very high dimensional datasets, in: Proceedings of the Twenty Seventh International Conference on Machine Learning (ICML-10), pp. 1047-1054.
22. T. Zhou, D. Tao, and X. Wu Manifold elastic net: a unified framework for sparse dimension reduction Data Min. Knowl. Discov. 22 2011 340 371
23. M. Varma, B.R. Babu, More generality in efficient multiple kernel learning, in: Proceedings of the Twenty Sixth International Conference on Machine Learning (ICML '09), pp. 1065-1072.
24. J. Mercer Functions of positive and negative type and their connection with the theory of integral equations Philos. Trans. R. Soc. (A) 83 1909 69 70
25. R. Courant, D. Hilbert, Methods of Mathematical Physics, Interscience, New York, NY, USA, 1953.
26. B. Scholkopf, A.J. Smola, Learning with Kernels, MIT Press, Cambridge, MA, USA, 2002.
27. R. Tibshirani Regression shrinkage and selection via the lasso J. R. Stat. Soc. Ser. B 58 1996 267 288
28. O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukherjee Choosing multiple parameters for support vector machines Mach. Learn. 46 2002 131 159
29. J. Danskin, The Theory of Max-Min and its Application to Weapons Allocation Problems, 1967.
30. S. Rosset, J. Zhu, T. Hastie, Margin maximizing loss functions, in: Advances in Neural Information Processing Systems (NIPS 15), MIT Press, Cambridge, MA, USA, 2003.
31. X. Nguyen, M.J. Wainwright, and M.I. Jordan On surrogate loss functions and f-divergences Ann. Stat. 37 2009 876 904
32. S. Boyd, and L. Vandenberghe Convex Optimization 2004 Cambridge University Press New York, NY, USA
33. T.F. Coleman, and Y. Li An interior trust region approach for nonlinear minimization subject to bounds SIAM J. Optim. 6 1996 415 425
34. J.C. Platt Advances in Kernel Methods 1999 MIT Press Cambridge, MA, USA pp. 185-208
35. R.-E. Fan, P.-H. Chen, and C.-J. Lin Working set selection using second order information for training support vector machines J. Mach. Learn. Res. 6 2005 1889 1918
36. T. Zhou, D. Tao, X. Wu, Nesvm: a fast gradient method for support vector machines, in: ICDM, IEEE Computer Society, 2010, pp. 679-688.
37. L. Grippo, and M. Sciandrone On the convergence of the block nonlinear Gauss-Seidel method under convex constraints Oper. Res. Lett. 26 2000 127 136
38. J.C. Bezdek, R.J. Hathaway, Some notes on alternating optimization, in: Proceedings of the 2002 AFSS International Conference on Fuzzy Systems (AFSS'02), pp. 288-300.
39. M.E. Thompson, NDCC: Normally Distributed Clustered Datasets on Cubes, 2006, www.cs.wisc.edu/dmi/svm/ndcc/.
40. M. Varma, 2013 (Last Accessed).
41. C.E. Thomaz, and G.A. Giraldi A new ranking method for principal components analysis and its application to face image analysis Image Vis. Comput. 28 2010 902 913
42. A. Frank, A. Asuncion, UCI Machine Learning Repository, 2010.
43. NIPS Workshop on Feature Extraction, 2003.
44. Spider Machine Learning Toolbox, 2013 (Last Accessed).
45. M. Zaffalon, M. Hutter, Robust feature selection by mutual information distributions, in: Proceedings of the Eighteenth Conference on Uncertainty in Artificial Intelligence, UAI'02, pp. 577-584.