Feature selection for linear SVMs under uncertain data: Robust optimization based on difference of convex functions algorithms

Neural Networks

Volumn 59, Issue , 2014, Pages 36-50

  Le Thi, Hoai An ^a,b   Vo, Xuan Thanh ^a   Pham Dinh, Tao ^c  

^a UNIVERSITÉ DE LORRAINE   (France)

^b UNIVERSITÉ DE LORRAINE   (France)

^c Avenue de l Université 76801 Saint Etienne du Rouvray   (France)

Author keywords

DC programming; DCA; Feature selection; Robust optimization; SVM

Indexed keywords

APPROXIMATION ALGORITHMS; FUNCTIONS; OPTIMIZATION; SUPPORT VECTOR MACHINES;

COMPUTATIONAL RESULTS; D-C PROGRAMMING; DIFFERENCE OF CONVEX FUNCTIONS; ELLIPSOIDAL MODEL; FEATURE SELECTION PROBLEM; ROBUST OPTIMIZATION; TRADITIONAL APPROACHES; UNCERTAIN DATAS;

FEATURE EXTRACTION;

ARTICLE; DATA PROCESSING; DIFFERENCE OF CONVEX FUNCTION ALGORITHM; DIFFERENCE OF CONVEX FUNCTION PROGRAMMING; LINEAR SYSTEM; MATHEMATICAL COMPUTING; MEASUREMENT ERROR; PRIORITY JOURNAL; PROCESS OPTIMIZATION; ROBUST OPTIMIZATION; STATISTICAL ANALYSIS; STATISTICAL DISTRIBUTION; STATISTICAL MODEL; STATISTICAL PARAMETERS; SUPPORT VECTOR MACHINE; UNCERTAINTY; ALGORITHM; GENETICS; HUMAN; LEUKEMIA; MICROARRAY ANALYSIS; WEATHER;

ALGORITHMS; DATA INTERPRETATION, STATISTICAL; HUMANS; LEUKEMIA; LINEAR MODELS; MICROARRAY ANALYSIS; SUPPORT VECTOR MACHINES; WEATHER;

EID: 84904859974     PISSN: 08936080     EISSN: 18792782     Source Type: Journal    
DOI: 10.1016/j.neunet.2014.06.011     Document Type: Article

References (51)

1. Ben-Tal A., El Ghaoui L., Nemirovski A. Robust optimization 2009, Princeton University Press, Princeton, NJ.
2. Bhattacharyya C., Grate L.R., Jordan M.I., Ghaoui L.E., Mian I.S. Robust sparse hyperplane classifier: application to uncertain molecular profiling data. Journal of Computational Biology 2004, 11:1073-1089.
3. Bhattacharyya C., Pannagadatta K.S., Smola A.J. A second order cone programming formulation for classifying missing data. Advances in neural information processing systems, Vol. 17 2004, MIT Press, Cambridge, MA. L.K. Saul, Y. Weiss, L. Bottou (Eds.).
4. Bi J., Zhang T. Support vector classification with input data uncertainty. Advances in neural information processing systems, Vol. 17 2004, MIT Press, Cambridge, MA. L.K. Saul, Y. Weiss, L. Bottou (Eds.).
5. Bradley P.S., Magasarian O.L. Feature selection via concave minimization and support vector machines. Machine learning proceedings of the fifteenth international conference 1998, 82-90. Morgan Kaufmann. J. Shavlik (Ed.).
6. Caramanis C., Mannor S., Xu H. Optimization for machine learning. Robust optimization in machine learning 2011, MIT Press, Cambridge, MA. S. Sra, S. Nowozin, S. Wright (Eds.).
7. Carroll R., Stefanski L.A., Ruppert D., Craincieanu C.M. Measurement error in nonlinear models 2006, Chapman and Hall.
8. Chambolle A., DeVore R.A., Lee N.Y., Lucier B.J. Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage. IEEE Transactions on Image Processing 1998, 7:319-335.
9. Collobert R., Sinz F., Weston J., Bottou L. Large scale transductive SVMs. Journal of Machine Learning Research 2006, 7:1687-1712.
10. Collobert R., Sinz F., Weston J., Bottou L. Trading convexity for scalability. Proceedings of the 23rd international conference on machine learning 2006, 201-208. ACM, New York, NY, USA.
11. Dempster A.P., Laird N.M., Rubin D.B. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B, Statistical Methodology 1977, 39:1-38.
12. Eves H.W. Elementary matrix theory 1966, Allyn and Bacon, Inc., Boston.
13. Fan J., Li R. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 2001.
14. Frank, A., & Asuncion, A. (2010). UCI machine learning repository Irvine, CA: University of California, School of Information and Computer Science. http://archive.ics.uci.edu/ml.
15. Fu W.J. Penalized regression: the bridge versus the lasso. Journal of Computational and Graphical Statistics 1998, 7:397-416.
16. Golub T.R., Slonim D.K., Tamayo P., Huard C., Gaasenbeek M., Mesirov J.P., et al. Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 1999, 286:531-537.
17. Gordon G.J., Jensen R.V., Hsiao L.-L., Gullans S.R., Blumenstock J.E., Ramaswamy S., et al. Translation of microarray data into clinically relevant cancer diagnostic tests using gene expression ratios in lung cancer and mesothelioma. Cancer Research 2002, 62.
18. Hermes, L., & Buhmann, J.M. (2000). Feature selection for support vector machines. In Proceedings. 15th international conference on pattern recognition, Vol. 2 (pp. 712-715).
19. Krause N., Singer Y. Leveraging the margin more carefully. Proceedings of the twenty-first international conference on machine learning 2004, 63-70. ACM, New York, NY, USA.
20. Le Hoai M., Le Thi H.A., Pham Dinh T., Huynh V.N. Block clustering based on difference of convex functions (DC) programming and DC algorithms. Neural Computation 2013, 25:259-278.
21. Le Thi H.A., Belghiti T., Pham Dinh T. A new efficient algorithm based on DC programming and DCA for clustering. Journal of Global Optimization 2006, 37:593-608.
22. Le Thi H.A., Le Hoai M., Nguyen V.V., Pham Dinh T. A DC programming approach for feature selection in support vector machines learning. Advances in Data Analysis and Classification 2008, 2:259-278.
23. Le Thi H.A., Le Hoai M., Pham Dinh T. Optimization based DC programming and DCA for hierarchical clustering. European Journal of Operational Research 2007, 183:1067-1085.
24. Le Thi H.A., Le Hoai M., Pham Dinh T. New and efficient DCA based algorithms for minimum sum-of-squares clustering. Pattern Recognition 2014, 47:388-401.
25. Le Thi H.A., Pham Dinh T. Solving a class of linearly constrained indefinite quadratic problems by DC algorithms. Journal of Global Optimization 1997, 11:2776-2807.
26. Le Thi H.A., Pham Dinh T. Large scale global molecular optimization from exact distance matrices by a D.C. optimization approach. SIAM Journal on Optimization 2003, 14:77-114.
27. Le Thi H.A., Pham Dinh T. The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems. Annals of Operations Research 2005, 133:23-46.
28. Le Thi H.A., Vo X.T., Pham Dinh T. Robust feature selection for svms under uncertain data. Lecture Notes in Computer Science 2013, vol. 7987:151-165. Springer, Berlin, Heidelberg. P. Perner (Ed.).
29. Liu Y., Shen X. Multicategory ψ-learning. Journal of the American Statistical Association 2006, 101:500-509.
30. Liu Y., Shen X., Doss H. Multicategory ψ-learning and support vector machine: computational tools. Journal of Computational and Graphical Statistics 2005, 14:219-236.
31. Liu Y., Wu Y. Variable selection via a combination of the l0 and l1 penalties. Journal of Computational and Graphical Statistics 2007, 16:782-798.
32. Neumann J., Schnörr C., Steidl G. Combined SVM-based feature selection and classification. Machine Learning 2005, 61:129-150.
33. Ong C.S., Le Thi H.A. Learning sparse classifiers with difference of convex functions algorithms. Optimization Methods & Software 2013, 28:830-854.
34. Pant R., Trafalis T.B., Barker K. Support vector machine classification of uncertain and imbalanced data using robust optimization. Proceedings of the 15th WSEAS international conference on computers 2011, 369-374. World Scientific and Engineering Academy and Society (WSEAS), Stevens Point, Wisconsin, USA.
35. Peleg D., Meir R. A bilinear formulation for vector sparsity optimization. Signal Processing 2008, 8:375-389.
36. Pham Dinh T., Le Thi H.A. Convex analysis approach to DC programming: theory, algorithms and applications. Acta Mathematica Vietnamica 1997, 22:289-357.
37. Pham Dinh T., Le Thi H.A. A DC optimization algorithms for solving the trust-region subproblem. SIAM Journal on Optimization 1998, 8:476-505.
38. Pham Dinh T., Le Thi H.A. Recent advances in DC programming and DCA. Transactions on Computational Collective Intelligence 2014, 8342:1-37.
39. Rakotomamonjy A. Variable selection using SVM-based criteria. Journal of Machine Learning Research 2003, 3:1357-1370.
40. Rao B.D., Kreutz-Delgado K. An affine scaling methodology for best basis selection. IEEE Transactions on Signal Processing 1999, 47:187-200.
41. Rinaldi F. Mathematical programming methods for minimizing the zero norm over polyhedral sets 2009, (Ph.D. thesis), Operations Research at Sapienza University of Rome.
42. Shivaswamy P.K., Bhattacharyya C., Smola A.J. Second order cone programming approaches for handling missing and uncertain data. Journal of Machine Learning Research 2006, 7:1238-1314.
43. ℓ0-norm. Modelling, computation and optimization in information systems and management sciences, Vol. 14 2008, 358-367. Springer, Berlin, Heidelberg. H.A. Le Thi, P. Bouvry, T. Pham Dinh (Eds.).
44. Thiao, M., Pham Dinh, T., & Le Thi, H.A. (2010). A DC programming approach for sparse eigenvalue problem. In Proceeding of ICML 2010 (pp. 1063-1070).
45. Tibshirani R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B 1996, 58:267-288.
46. Wang H., Li G., Jiang G. Robust regression shrinkage and consistent variable selection via the lad-lasso. Journal of Business & Economics Statistics 2007, 25:347-355.
47. Weston J., Elisseeff A., Schölkopf B., Kaelbling P. Use of the zero-norm with linear models and kernel methods. Journal of Machine Learning Research 2003, 3:1439-1461.
48. Wu Y., Liu Y. Robust truncated-hinge-loss support vector machines. Journal of the American Statistical Association 2007, 102:974-983.
49. Yuille A.L., Rangarajan A. The concave-convex procedure (CCCP). Neural Computation 2003, 15:915-936.
50. Zhang C.H. Nearly unbiased variable selection under minimax concave penalty. Annals of Statistics 2010, 38:894-942.
51. Zou H. The adaptive lasso and its oracle properties. Journal of the American Statistical Association 2006, 101:1418-1429.