Sparse semi-supervised support vector machines by DC programming and DCA

Neurocomputing

Volumn 153, Issue , 2015, Pages 62-76

  Le, Hoai Minh ^a,b   Le Thi, Hoai An ^b   Nguyen, Manh Cuong ^b  

^a TON DUC THANG UNIVERSITY   (Viet Nam)

^b UNIVERSITÉ DE LORRAINE   (France)

Author keywords

DC approximation; DC programming and DCA; Exact penalty; Feature selection; Non convex optimization; Semi supervised SVM

Indexed keywords

CLASSIFICATION (OF INFORMATION); CONVEX OPTIMIZATION; FEATURE EXTRACTION; FUNCTIONS; OPTIMIZATION;

DC APPROXIMATION; DC PROGRAMMING AND DCA; EXACT PENALTY; NONCONVEX OPTIMIZATION; SEMI-SUPERVISED;

SUPPORT VECTOR MACHINES;

ALGORITHM; ARTICLE; CLASSIFICATION; DIFFERENCE OF CONVEX; DIFFERENCE OF CONVEX ALGORITHM; LINEAR SYSTEM; MATHEMATICAL ANALYSIS; PROBLEM SOLVING; PROCESS OPTIMIZATION; SEMI SUPERVISED SUPPORT VECTOR MACHINE; SUPPORT VECTOR MACHINE;

EID: 84961287929     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2014.11.051     Document Type: Article

References (63)

1. Amaldi E., Kann V. On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems. Theor. Comput. Sci. 1998, 209:237-260.
2. Kristin P. Bennett Ayhan Demiriz, Semi-supervised support vector machines, in: Proceedings of the Conference on Advances in Neural Information Processing Systems II, Cambridge, MA, USA, 1999, pp. 368-374.
3. P.S. Bradley, O.L. Mangasarian, Feature selection via concave minimization and support vector machines, in: Proceeding of ICML'98, SF, CA, USA, 1998, pp. 82-90.
4. T.D. Bie, N. Cristianini, Convex methods for transduction, in: Advances in Neural Information Proceeding Systems 16, MIT Press, 2004, p. 73.
5. O. Chapelle, A. Zien, Semi-supervised classification by low density separation, in: Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics, Barbados, 2005, pp. 57-64.
6. O. Chapelle, V. Sindhwani, S. Keerthi, Branch and bound for semi-supervised support vector machines, in: Advances in Neural Information Processing Systems, MIT Press, Cambridge vol. 17, 2006, pp. 217-224.
7. Chapelle O., Sindhwani V., Keerthi S.S. Optimization techniques for semi-supervised support vector machines. J. Mach. Learn. Res. 2008, 9:203-233.
8. Choi H., Kim J., Kim Y. A sparse large margin semi-supervised learning method. J. Korean Stat. Soc. 2010, 39(4):479.
9. Collobert R., Sinz F., Weston J., Bottou L. Large scale transductive SVMs. J. Mach. Learn. 2006, 7:1687-1712.
10. W. Emara, M.K.M. Karnstedt, K. Sattler, D. Habich, W. Lehner, An approach for incremental semi-supervised SVM, in: Seventh IEEE International Conference on Data Mining Workshops (ICDMW 2007), 2007, pp. 539-544.
11. Fan J., Li R. Variable selection via nonconcave penalized likelihood and its Oracle Properties. J. Am. Stat. Assoc. 2001, 96:1348-1360.
12. Fung G., Mangasarian O. Semi-supervised support vector machines for unlabeled data classification. Optim. Methods Softw. 2001, 15:29.
13. F. Gieseke, A. Airola, T. Pahikkala, O. Kramer, Sparse Quasi-Newton optimization for semi-supervised support vector machines, in: Proceedings of the 1st International Conference on Pattern Recognition Applications and Methods (ICPRAM), 2012, pp. 45-54.
14. Gribonval R., Nielsen M. Sparse representation in union of bases. IEEE Trans. Inf. Theory 2003, 49:3320-3325.
15. T. Joachims, Transductive inference for text classification using support vector machines, in: 16th International Conference on Machine Learning, SF, USA, 1999, pp. 200-209.
16. B. Heisele, P. Ho, T. Poggio, Face recognition with support vector machines: global versus Component-based approach, in: Proceedings of 8th International Conference on Computer Vision, 2001, pp. 688-694.
17. L. Hermes, J.M. Buhmann, Feature selection for support vector machines, in: Proceedings 15th International Conference on Pattern Recognition, vol. 2, 2000, pp. 712-715.
18. Hjelmas E. Face detection. a survey. Comput. Vis. Image Underst. 2001, 83:236-274.
19. Hui Z. The adaptive lasso and its oracle properties. J. Am. Stat. Assoc. 2006, 101(476):1418-1429.
20. N. Krause, Y. Singer, Leveraging the margin more carefully, in: Proceeding of ICML '04, NY, USA, 2004, pp. 63-71.
21. I. Kukenys, Human face detection with support vector machines (Thesis), University of Otago, 2010.
22. H.M. Le, H.A. Le Thi, M.C. Nguyen, DCA based algorithms for feature selection in semi-supervised support vector machines. Machine learning and data mining in pattern recognition, in: Petra Perner (Ed.), Lecture Notes in Computer Science, vol. 7988, 2013, pp. 528-542.
23. H.A. Le Thi, Analyse numérique des algorithmes de l'Optimisation d. c. Approches locales et globales, Codes et simulations numériques en grande dimension. Applications. Thèse de doctorat, Université de Rouen, 1994.
24. H.A. Le Thi, Contribution à l'optimisation non convexe et l'optimisation globale: Théorie, Algoritmes et Applications, Habilitation à Diriger des Recherches, Université de Rouen, 1997.
25. Le Thi H.A., Pham Dinh T. Solving a class of linearly constrained indefinite quadratic problems by DC algorithms. J. Glob. Optim. 1997, 11(3):253-285.
26. Le Thi H.A., Pham Dinh T. The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems. Ann. Oper. Res. 2005, 133:23-46.
27. Le Thi H.A., Belghiti T., Pham Dinh T. A new efficient algorithm based on DC programming and DCA for clustering. J. Glob. Optim. 2006, 37:593-608.
28. Le Thi H.A., Le Hoai M., Pham Dinh T. Optimization based DC programming and DCA for hierarchical clustering. Eur. J. Oper. Res. 2007, 183:1067-1085.
29. Le Thi H.A., Le Hoai M., Nguyen N.V., Pham Dinh T. A DC programming approach for feature selection in support vector machines learning. J. Adv. Data Anal. Classif. 2008, 2(3):259-278.
30. Le Thi H.A., Nguyen V.V., Ouchani S. Gene selection for cancer classification using DCA. J. Fonct. Comput. Sci. Technol. 2009, 3(6):15.
31. LeThi H.A., PhamDinh T., Huynh V.N. Exact penalty and error bounds in DC programming. Journal of Global Optimization 2012, 52(3):509-535.
32. H.A. Le Thi, A New Approximation for the ℓ0-norm, Research Report LITA EA 3097, University of Lorraine, France, 2012.
33. H.A. Le Thi , M. Le Hoai, T. Pham Dinh, Feature selection in machine learning: an exact penalty approach using a difference of convex function algorithm, Mach. Learn., published online 04 Jul, 2014,. doi:10.1007/s10994-014-5455-y.
34. Le Thi H.A., Vo X.T., Pham Dinh T. Feature selection for linear SVMs under uncertain data. robust optimization based on difference of convex functions algorithms. Neural Netw. 2014, 59:36-50.
35. Le Thi H.A., Nguyen M.C. Self-organizing maps by difference of convex functions optimization. Data Min. Knowl. Discov. 2014, 28(5-6):1336-1365.
36. H.A. Le Thi, DC Programming and DCA. http://lita.sciences.univ-metz.fr/~lethi.
37. Y.F. Li, J.T. Kwok, Z.H. Zhou, Semi-supervised learning using label Mean, in: Proceedings of the 26th International Conference on Machine Learning, Montreal, Canada, 2009, pp. 1-8.
38. Liu Y., Shen X., Doss H. Multicategory ψ-learning and support vector machine. computational tools. J. Comput. Graph. Stat. 2005, 14:219-236.
39. J. Ma, X. Zhang, A full smooth semi-support vector machine based on the cubic spline function, in: 6th International Conference on Biomedical Engineering and Informatics (BMEI), 2013, pp. 650-655.
40. Neumann J., Schnörr C., Steidl G. Combined SVM-based feature selection and classification. Mach. Learn. 2005, 61(1-3):129-150.
41. E. Osuna, R. Freund, F. Girosi, Training support vector machines: an application to face detection, in: Proceedings of Computer Vision and Pattern Recognition, 1997, pp. 130-136.
42. Pham Dinh T., Le Thi H.A. Convex analysis approach to d.c. programming. theory, algorithm and applications. Acta Math. Vietnam. 1997, 22:289-355.
43. T. Pham Dinh, H.A. Le Thi, Recent advances on DC programming and DCA. Transactions on computational intelligence XIII, in: Lecture Notes in Computer Science, vol. 8342, 2014, pp. 1-37.
44. Pham Dinh T., Nguyen Canh N., Le Thi H.A. An efficient combination of DCA and B&B using DC/SDP relaxation for globally solving binary quadratic programs. J. Glob. Optim. 2010, 48(4):595-632.
45. Peleg D., Meir R. A bilinear formulation for vector sparsity optimization. Signal Process. 2008, 8(2):375-389.
46. Ong C.S., Le Thi H.A. Learning sparse classifiers with difference of convex functions algorithms. Optim. Methods Softw. 2013, 28:4.
47. Qi Tian Y., Shi Y., Yu X. Cost-Sensitive support vector machine for semi-supervised learning. Proc. Comput. Sci. 2013, 18:1684-1689.
48. M. Thiao, T. Pham Dinh, H.A. Le Thi, DC programming approach for a class of nonconvex programs involving l0 norm, in: "Modelling, Computation and Optimization in Information Systems and Management Sciences", Communications in Computer and Information Science CCIS, vol. 14, Springer, 2008, pp. 358-367.
49. Rakotomamonjy A. Variable selection using SVM-based criteria. J. Mach. Learn. Res. 2003, 3:1357-1370.
50. C. Ronan, S. Fabian, W. Jason, B. Lé, Trading Convexity for Scalability, in: Proceedings of the 23rd International Conference on Machine Learning ICML 2006, Pittsburgh, Pennsylvania, 2006, pp. 201-208.
51. L. Yang, L.Wang, Simultaneous feature selection and classification via semi-supervised models, in: Proceeding ICNC '07, Third International Conference on Natural Computation-Cover, 2007, pp. 646-650.
52. M. Saquib Sarfraz, O. Hellwich, Z. Riaz, Feature extraction and representation for face recognition, Face Recognit. ISBN 978-953-307-060-5, 2010, 1-21.
53. V. Sindhwani, S. Keerthi, O. Chapelle, Deterministic annealing for semi-supervised kernel machines, in: Proceedings of ICML'06, NY, USA, 2006, pp. 841-848.
54. V. Sindhwani, S. Keerthi, Large scale semi-supervised linear SVMs, in: Proceeding SIGIR '06 Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2006, pp. 477-484.
55. Tibshirani R. Regression shrinkage and selection via the lasso. J. R. Stat. Soc. 1996, 46:431-439.
56. M. Thiao, T. Pham Dinh, H.A. Le Thi A DC programming approach for Sparse Eigenvalue Problem, in: Proceeding of ICML 2010, 2010, 1063-1070.
57. Vapnik V., Sterin A. On structural risk minimization or overall risk in a problem of pattern recognition. Autom. Remote Control 1977, 10(3):1495-1503.
58. C. Zhang, Z. Zhang, A Survey of Recent Advances in Face Detection, Technical Report, Number MSR-TR-2010-66, Microsoft Research, 2010.
59. Zhang R., Liu T.B., Zheng M.W. Semi-supervised learning for classification with uncertainty. Adv. Mater. Res. 2012, 433-440:3584-3590.
60. Zou H. The adaptive lasso and its oracle properties. J. Am. Stat. Ass. 2006, 101:1418-1429.
61. X. Zhu, A.B. Goldberg, Introduction to semi-supervised learning, Morgan and Claypool, ISBN: 1598295470 9781598295474, 2009.
62. Wang H., Li G., Jiang G. Robust regression shrinkage and consistent variable selection via the LAD-LASSO. J. Bus. Econ. Stat. 2007, 25(3):347-355.
63. Weston J., Elisseeff A., Scholkopf B., Tipping M. Use of the zero-norm with linear models and kernel methods. J. Mach. Learn. Res. 2003, 3:1439-1461.