Learning sparse classifiers with difference of convex functions algorithms

Optimization Methods and Software

Volumn 28, Issue 4, 2013, Pages 830-854

  Ong, Cheng Soon ^a   An, Le Thi Hoai ^b  

^a ETH ZURICH   (Switzerland)

^b UNIVERSITÉ DE LORRAINE   (France)

Author keywords

binary classification; DCA; sparse features and examples

Indexed keywords

BINARY CLASSIFICATION; DCA; DIFFERENCE OF CONVEX FUNCTIONS; HIGH DIMENSIONAL DATA; MACHINE LEARNING REPOSITORY; NON-CONVEX OBJECTIVE FUNCTIONS; SPARSE FEATURES; UNIVERSITY OF CALIFORNIA;

ALGORITHMS;

FUNCTIONS;

EID: 84880232279     PISSN: 10556788     EISSN: 10294937     Source Type: Journal    
DOI: 10.1080/10556788.2011.652630     Document Type: Conference Paper

References (33)

1. A. Ben-Hur, C.S. Ong, S. Sonnenburg, B. Schölkopf, and G. Rätsch, Support vector machines and kernels for computational biology, PLoS Comput. Biol. 4(10) (2008), e1000173.
2. P.S. Bradley and O.L. Mangasarian, Feature selection via concave minimization and support vector machines, Proceedings of International Conference on Machina Learning, Madison, Wisconsin, USA, 1998.
3. R. Collobert, F. Sinz, J.Weston, and L. Bottou, Trading convexity for scalability, in Large Scale Kernel Machines, L. Bottou, O. Chapelle, D. DeCoste, and J.Weston, eds., MIT Press, USA, 2007, pp. 275-300.
4. C.B. Do, Q. Le, C.H. Teo, O. Chapelle, and A. Smola, Tighter bounds for structured estimation, in Proceedings of Neural Information Processing Systems 2008, Vancouver, British Columbia, Canada, pp. 281-288.
5. R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R.Wang, and C.-J. Lin, LIBLINEAR:A library for large linear classification, J. Mach. Learn. Res. 9 (2008), pp. 1871-1874.
6. Y. Freund, A more robust boosting algorithm, Tech. Rep. arXiv:0905.2138.v1, Computer Science and Engineering, UCSD, 2009.
7. G. Gasso, A. Rakotomamonjy, and S. Canu, Recovering sparse signals with a certain family of non-convex penalties and DC programming, IEEE Trans. Signal Process. 57(12) (2009), pp. 4686-4698.
8. I. Guyon and A. Elisseeff, An introduction to variable and feature selection, J. Mach. Learn. Res. 3 (2003), pp. 1157-1182.
9. J.-B. Hiriart-Urruty and C. Lemaréchal, Fundamentals of Convex Analysis, Springer Verlag, 2001.
10. N. Krause and Y. Singer, Leveraging the margin more carefully, Proceedings of the Twenty-first International Conference on Machine Learning, Banff, Alberta, Canada, 2004.
11. H.A. Le Thi and T. Pham Dinh, The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems, Ann. Oper. Res. 133 (2005), pp. 23-46.
12. H.A. Le Thi, T. Belghiti, and T. Pham Dinh, A new efficient algorithm based on DC programming and DCA for clustering, J. Global Optim. 37 (2006), pp. 593-608.
13. H.A. Le Thi, H. Minh Le, and T. Pham Dinh, Optimization based DC programming and DCA for hierarchical clustering, Eur. J. Oper. Res. 183 (2007), pp. 1067-1085.
14. H.A. Le Thi, H.M. Le, V.V. Nguyen, and T. Pham Dinh, A DC programming approach for feature selection in support vector machines learning, Adv. Data Anal. Classification 2(3) (2008), pp. 259-278.
15. Y. Liu and X. Shen, Multicategory ψ-learning, J. Amer. Statist. Assoc. 101 (2006), pp. 500-509.
16. Y. Liu, X. Shen, and H. Doss, Multicategory ψ-learning and support vector machine: Computational tools, J. Comput. Graph. Statist. 14 (2005), pp. 219-236.
17. P. Long and R. Servedio, Random classification noise defeats all convex potential boosters, International Conference on Machine Learning, Helsinki, Finland, 2008.
18. O.L. Mangasarian, Exact 1-norm support vector machines via unconstrained convex differentiable minimization, J. Mach. Learn. Res. 7 (2006), pp. 1517-1530.
19. N. Meinshausen and B.Yu, Lasso-type recovery of sparse representations for high-dimensional data, Ann. Statist. 37(1) (2009), pp. 246-270.
20. J. Neumann, C. Schnörr, and G. Steidl, Combined SVM-based feature selection and classification, Mach. Learn. 61(1-3) (2005), pp. 129-150.
21. D. Peleg and R. Meir, A bilinear formulation for vector sparsity optimization, Signal Process. 88 (2008), pp. 375-389.
22. T. Pham Dinh, Algorithms for solving a class of non convex optimization problems. Methods of subgradients, in Fermat Days 85. Mathematics for Optimization, J.B. Hiriart-Urruty ed., Elsevier Science Publishers, B.V. North- Holland, 1986.
23. T. Pham Dinh and H.A. Le Thi, DC optimization algorithms for solving the trust region subproblem, SIAM J. Optim. 8(2) (1998), pp. 476-505.
24. B. Schölkopf and A.J. Smola, Learning with Kernels, MIT Press, USA, 2002.
25. I. Steinwart, On the influence of the kernel on the consistency of support vector machines, J. Mach. Learn. Res. 2 (2001), pp. 67-93.
26. I. Steinwart and A. Christmann, Support Vector Machines, Information Science and Statistics, SpringerVerlag, 2008.
27. I. Steinwart, D. Hush, and C. Scovel, An oracle inequality for clipped regularized risk minimizers, In B. Schölkopf, J. Platt, and T. Hoffman, eds., Advances in Neural Information Processing Systems 19, MIT Press, Cambridge, MA, pp. 1321-1328.
28. R. Tibshirani, Regression shrinkage and selection via the LASSO, J. R. Statist. Soc. B. 58(1) (1996), pp. 267-288.
29. Y. Wu and Y. Liu, Robust truncated-hinge-loss support vector machines, J. Amer. Statist. Assoc. 102(479) (2007), pp. 974-983.
30. A.L.Yuille and A. Rangarajan, The concave-convex procedure, Neural Comput. 15(4) (2003), pp. 915-936.
31. T. Zhang, Some sharp performance bounds for least squares regression with l1 regularization, Ann. Statist. 37(5A) (2009), pp. 109-2144.
32. J. Zhu and E.P. Xing, On primal and dual sparsity of Markov networks, in International Conference on Machine Learning, Montreal, Quebec, Canada, 2009.
33. J. Zhu, S. Rosset, T. Hastie, and R. Tibshirani, 1-Norm support vector machines, in Neural Information Processing Systems, 2004.