A new boosting design of Support Vector Machine classifiers

Information Fusion

Volumn 25, Issue , 2015, Pages 63-71

  Mayhua López, Efraín ^a   Gómez Verdejo, Vanessa ^b   Figueiras Vidal, Aníbal R ^b  

^a UNIVERSIDAD CATÓLICA SAN PABLO   (Peru)

^b UNIVERSIDAD CARLOS III DE MADRID   (Spain)

Author keywords

Ensemble classifiers; Linear programming; Real AdaBoost; Subsampling; Support vector machines

Indexed keywords

ADAPTIVE BOOSTING; ALGORITHMS; DESIGN; ITERATIVE METHODS; LINEAR PROGRAMMING; STRUCTURE (COMPOSITION);

BOOSTING ALGORITHM; COMPUTATIONAL EFFORT; ENSEMBLE CLASSIFIERS; MONOLITHIC ARCHITECTURE; PERFORMANCE DESIGN; SUBSAMPLING; SUPPORT VECTOR MACHINE (SVMS); SUPPORT VECTOR MACHINE CLASSIFIERS;

SUPPORT VECTOR MACHINES;

EID: 84924807118     PISSN: 15662535     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.inffus.2014.10.005     Document Type: Article

References (63)

1. V. Vapnik Estimation of Dependences Based on Empirical Data 1982 Springer-Verlag Secaucus, NJ, USA
2. V. Vapnik The Nature of Statistical Learning Theory 1995 Springer New York, NY
3. V. Vapnik Statistical Learning Theory 1998 Wiley New York, NY
4. G. Kimeldorf, and G. Wahba Some results on Tchebycheffian spline functions J. Math. Anal. Appl. 33 1971 82 95
5. B. Schölkopf, and A. Smola Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond 2002 MIT Press Cambridge, MA
6. R. Herbrich Learning Kernel Classifiers: Theory and Algorithms 2001 MIT Press Cambridge, MA, USA
7. J. Shawe-Taylor, and N. Cristianini Kernel Methods for Pattern Analysis 2004 Cambridge University Press New York, NY
8. L. Wang Support Vector Machines: Theory and Applications 2005 Springer New York, NY
9. I. Steinwart, and A. Christmann Support Vector Machines 2008 Springer New York, NY
10. J. Shawe-Taylor, and S. Sun A review of optimization methodologies in support vector machines Neurocomputing 74 17 2011 3609 3618
11. J.L. Rojo-Álvarez, M. Martínez-Ramón, A.R. Figueiras-Vidal, A. García-Armada, and A. Artés-Rodríguez A robust support vector algorithm for nonparametric spectral analysis IEEE Signal Process. Lett. 10 2003 320 323
12. J.L. Rojo-Álvarez, M. Martínez-Ramón, M. de Prado-Cumplido, A. Artés-Rodríguez, and A.R. Figueiras-Vidal Support vector method for robust ARMA system identification IEEE Trans. Signal Proces. 52 2004 155 164
13. J.L. Rojo-Álvarez, G. Camps-Valls, M. Martínez-Ramón, E. Soria-Olivas, Á. Navia-Vázquez, and A.R. Figueiras-Vidal Support vector machines framework for linear signal processing Signal Proces. 85 12 2005 2316 2326
14. M. Martínez-Ramón, J.L. Rojo-Álvarez, G. Camps-Valls, J. Muñoz-Marí, Á. Navia-Vázquez, E. Soria-Olivas, and A.R. Figueiras-Vidal Support vector machines for nonlinear kernel ARMA system identification IEEE Trans. Neural Networks 17 2006 1617 1622
15. C.M. Bishop Pattern Recognition and Machine Learning 2006 Springer New York, NY pp. 332
16. J.A. Suykens, and J. Vandewalle Least squares support vector machine classifiers Neural Proces. Lett. 9 1999 293 300
17. P.S. Bradley, and O.L. Mangasarian Feature selection via concave minimization and support vector machines Proc. 15th Intl. Conf. on Machine Learning 1998 Morgan Kaufman San Francisco, CA 82 90
18. V. Kecman, I. Hadzic, Support vectors selection by linear programming, in: Proc. Intl. Joint Conf. on Neural Networks, vol. 5, Como, Italy, 2000, pp. 193-198.
19. W. Zhou, L. Zhang, and L. Jiao Linear programming support vector machines Pattern Recogn. 35 2002 2927 2936
20. J. Zhu, S. Rosset, T. Hastie, and R. Tibshirani 1-Norm support vector machines S. Thrun, L. Saul, B. Schölkopf, Advances in Neural Information Proc. Sys. vol. 16 2004 MIT Press Cambridge, MA 49 56
21. Rm on reduced SVMs Stat. Optim. Inform. Comput. 1 2013 82 106
22. S. El Jelali, A. Lyhyaoui, and A.R. Figueiras-Vidal Designing model based classifiers by emphasizing soft targets Fundam. Inf. 96 2009 419 433
23. Y. Liu, and X. Yao Ensemble learning via negative correlation Neural Networks 12 1999 1399 1404
24. Y. Liu, and X. Yao Simultaneous training of negatively correlated neural networks in an ensemble IEEE Trans. Syst. Man Cybern., Part B: Cybern. 29 1999 716 725
25. Y. Freund, and R.E. Schapire A decision-theoretic generalization of on-line learning and an application to boosting J. Comput. Syst. Sci. 55 1997 119 139
26. R.E. Schapire, and Y. Singer Improved boosting algorithms using confidence-rated predictions Mach. Learn. 37 1999 297 336
27. R.E. Schapire, and Y. Freund Boosting: Foundations and Algorithms 2012 MIT Press Cambridge, MA
28. H. Drucker, R.E. Schapire, and P. Simard Boosting performance in neural networks Intl. J. Pattern Recogn. Artif. Intell. 7 1993 705 719
29. Y. LeCun, L.D. Jackel, H.A. Eduard, N. Bottou, C. Cortes, J.S. Denker, H. Drucker, E. Sackinger, P. Simard, and V. Vapnik Learning algorithms for classification: a comparison on handwritten digit recognition J.H. Oh, C. Kwon, S. Cho, Neural Networks: The Statistical Mechanics Perspective 1995 World Scientific Singapore 261 276
30. H. Schwenk, and Y. Bengio Adaboosting neural networks W. Gerstner, A. Germond, M. Hasler, J.D. Nicoud, Proc. 7th Intl. Conf. on Artificial Neural Networks (LNCS 1327) 1997 Springer Berlin 967 972
31. E. Bauer, and R. Kohavi An empirical comparison of voting classification algorithms: bagging, boosting, and variants Mach. Learn. 36 1999 105 139
32. T.G. Dietterich An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization Mach. Learn. 40 2000 139 157
33. L. Breiman Prediction games and arcing algorithms Neural Comput. 11 1999 1493 1517
34. G. Rätsch, T. Onoda, and K.R. Müller Soft margins for AdaBoost Mach. Learn. 42 2001 287 320
35. G. Rätsch, and M.K. Warmuth Efficient margin maximizing with boosting J. Mach. Learn. Res. 6 2005 2131 2152
36. Y. Sun, S. Todorovic, and J. Li Reducing the overfitting of AdaBoost by controlling its data distribution skewness Intl. J. Pattern Recogn. Artif. Intell. 20 2006 1093 1116
37. V. Gómez-Verdejo, M. Ortega-Moral, J. Arenas-García, and A.R. Figueiras-Vidal Boosting by weighting critical and erroneous samples Neurocomputing 69 2006 679 685
38. V. Gómez-Verdejo, J. Arenas-García, and A.R. Figueiras-Vidal A dynamically adjusted mixed emphasis method for building boosting ensembles IEEE Trans. Neural Networks 19 2008 3 17
39. C.-X. Zhang, J.-S. Zhang, and G.-Y. Zhang An efficient modified boosting method for solving classification problems J. Comput. Appl. Math. 214 2008 381 392
40. C. Shen, and H. Li Boosting through optimization of margin distributions IEEE Trans. Neural Networks 21 4 2010 659 666
41. A. Aachad, A. Omari, and A.R. Figueiras-Vidal Neighborhood guided smoothed emphasis for real adaboost ensembles Neural Proc. Lett. 10 2014 1 11
42. J. Wickramaratna, S. Holden, and B. Buxton Performance degradation in boosting J. Kittler, F. Roli, Multiple Classifier Systems (LNCS 2096) 2001 Springer Berlin 11 21
43. M. Kawakita, and S. Eguchi Boosting method for local learning in statistical pattern recognition Neural Comput. 20 2008 2792 2838
44. E. Mayhua-López, V. Gómez-Verdejo, and A.R. Figueiras-Vidal Real AdaBoost with gate controlled fusion IEEE Trans. Neural Networks Learn. Syst. 23 12 2012 2003 2009
45. A. Omari, and A.R. Figueiras-Vidal Feature combiners with gate generated weights for classification IEEE Trans. Neural Networks Learn. Syst. 24 2013 158 163
46. W. Wu, Z. Yanan, W. Linlin, An AdaBoost algorithm with SVM based on nonlinear decision function, in: Proc. Intl. Conf. on Computational Intell. and Natural Computing, Wuhan, China, 2009, pp. 22-25.
47. N. Lima, A. Neto, J. de Melo, Creating an ensemble of diverse support vector machines using AdaBoost, in: Proc. Intl. Joint Conf. on Neural Networks, Atlanta, GA, USA, 2009, pp. 1802-1806.
48. T. Wei, Z. Qin, X. Cao, B. Leng, A boosting method based on SVM for relevance feedback in content-based 3D model retrieval, in: Proc. 2nd Intl. Conf. on Software Engineering and Data Mining, Chengdu, China, 2010, pp. 517-522.
49. X. Li, L. Wang, and E. Sung AdaBoost with SVM-based component classifiers Eng. Appl. Artif. Intell. 21 2008 785 795
50. H.-C. Kim, S. Pang, H.-M. Je, D. Kim, and S.Y. Bang Constructing support vector machine ensemble Pattern Recogn. 36 2003 2757 2767
51. P. Rangel, F. Lozano, E. García, Boosting of support vector machines with application to editing, in: Proc. 4th Intl. Conf. on Machine Learning and Applications, Los Angeles, CA, USA, 2005, pp. 6.
52. S. Muñoz Romero, J. Arenas-García, and V. Gómez-Verdejo Real AdaBoost ensembles with emphasized subsampling J. Cabestany, F. Sandoval, A. Prieto, J. Corchado, Bio-Inspired Systems: Computational and Ambient Intelligence (LNCS 5517) 2009 Springer Berlin 440 447
53. M. Villamizar, J. Andrade-Cetto, A. Sanfeliu, and F. Moreno-Noguer Bootstrapping boosted random ferns for discriminative and efficient object classification Pattern Recogn. 45 2012 3141 3153
54. Y. Lee, Y. Lin, and G. Wahba Multicategory support vector machines. Theory and application to the classification of microarray data and satellite radiance data J. Am. Stat. Assoc. 99 2004 67 81
55. Y. Ji, and S. Sun Multitask multiclass support vector machines: model and experiments Pattern Recogn. 46 3 2013 914 924
56. J. Zhu, H. Zou, S. Rosset, and T. Hastie Multi-class AdaBoost Stat. Interface 2 2009 349 360
57. L.I. Kuncheva Combining Pattern Classifiers: Methods and Algorithms 2004 Wiley Hoboken, NJ
58. L. Rokach Pattern Classification Using Ensemble Methods 2010 World Scientific Singapore
59. Z. Kalal, J. Matas, and K. Mikolajczyk Weighted sampling for large-scale boosting M. Everingham, C.J. Needham, R. Fraile, Proc. of the British Machine Vision Conference 2008 British Machine Vision Assoc. Leeds, UK 42.1 42.10
60. A. Frank, and A. Asuncion UCI Machine Learning Repository 2010 School of Information and Computer Sciences, University of California Irvine < http://archive.ics.uci.edu/ml >
61. J.Y. Kwok Moderating the outputs of support vector machine classifiers IEEE Trans. Neural Networks 10 1999 1018 1031
62. B.D. Ripley Neural networks and related methods for classification J. Roy. Stat. Soc. 56 1994 409 456
63. Mosek, The MOSEK Optimization Tools Manual Version 6.0 (Revision 66), MOSEK ApS, Copenhagen, Denmark, 2010.