An ensemble approach of dual base learners for multi-class classification problems

Information Fusion

Volumn 24, Issue , 2015, Pages 122-136

  Sesmero, M Paz ^a   Alonso Weber, Juan M ^a   Gutierrez, German ^a   Ledezma, Agapito ^a   Sanchis, Araceli ^a  

^a UNIVERSIDAD CARLOS III DE MADRID   (Spain)

Author keywords

Artificial Neural Networks; Diversity; Ensemble of classifiers; Feature Selection; Multi class classification

Indexed keywords

CLASSIFIERS; FEATURE EXTRACTION; LEARNING SYSTEMS; NEURAL NETWORKS;

BINARY DECOMPOSITIONS; CLASSIFICATION METHODS; DIVERSITY; ENSEMBLE OF CLASSIFIERS; EXPERIMENTAL EVALUATION; MULTI-CLASS CLASSIFICATION; MULTI-CLASS CLASSIFIER; MULTICLASS CLASSIFICATION PROBLEMS;

CLASSIFICATION (OF INFORMATION);

EID: 84922842596     PISSN: 15662535     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.inffus.2014.09.002     Document Type: Article

References (65)

1. R. Ranawana Multi-classifier systems: review and a roadmap for developers Int. J. Hybrid Intell. Syst. 3 2006 35 61
2. T.G. Dietterich, Ensemble Methods in Machine Learning, in: Mult. Classif. Syst. Lect. Notes Comput. Sci., Springer, Berlin Heidelberg, 2000, pp. 1-15.
3. G. Brown, J. Wyatt, R. Harris, and X. Yao Diversity creation methods: a survey and categorisation Inf. Fusion. 6 2005 5 20
4. D. Wolpert Stacked generalization Neural Networks 5 1992 241 259
5. D. Bahler, L. Navarro, Methods for Combining Heterogeneous Sets of Classifiers, in: Proc. the17th Natl. Conf. Artif. Intell. Work. New Res. Probl. Mach. Learn., 2000.
6. A. Ledezma, R. Aler, A. Sanchis, and D. Borrajo GA-stacking: evolutionary stacked generalization Intell. Data Anal. 14 2010 89 119
7. J.F. Kolen, and J.B. Pollack Backpropagation is sensitive to initial conditions Complex Syst. 4 1990 269 280
8. T.G. Dietterich An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization Mach. Learn. 40 2000 139 157
9. J. Quinlan, C4.5: Programs for Machine Learning, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1993.
10. T.G. Dietterich Machine-learning research: four current directions AI Mag. 18 1997 97 137
11. L. Breiman Bagging predictors Mach. Learn. 24 1996 123 140
12. R.E. Schapire The strength of weak learnability Mach. Learn. 5 1990 197 227
13. T.K. Ho The random subspace method for constructing decision forests IEEE Trans. Pattern Anal. Mach. Intell. 20 1998 832 844
14. D. Optiz, Feature Selection for Ensembles, in: Proc. 16th Int. Conf. Artif. Intell., 1999, pp. 379-384.
15. L.S. Oliveira, R. Sabourin, F. Bortolozzi, and C.Y. Suen A methodology for feature selection using multi-objective genetic algorithms for handwritten digit string recognition Int. J. Pattern Recognit. Artif. Intell. 17 2003 903 929
16. A. Tsymbal, M. Pechenizkiy, and P. Cunningham Diversity in search strategies for ensemble feature selection Inf. Fusion. 6 2005 83 98
17. R. Bryll, R. Gutierrez-Osuna, and F. Quek Attribute bagging: improving accuracy of classifier ensembles by using random feature subsets Pattern Recognit. 36 2003 1291 1302
18. A.J.C. Sharkey, and N.E. Sharkey Combining diverse neural nets Knowl. Eng. Rev. 12 1997 231 247
19. R. Anand, K.G. Mehrotra, C.K. Mohan, and S. Ranka An improved algorithm for neural network classification of imbalanced training sets IEEE Trans. Neural Networks 4 1993 962 969
20. T. Hastie, and R. Tibshirani Classification by Pairwise coupling Ann. Stat. 6 1998 451 471
21. G. Ou, and Y.L. Murphey Multi-class pattern classification using neural networks Pattern Recognit. 40 2007 4 18
22. T.G. Dietterich, and G. Bakiri Solving multiclass learning problems via error-correcting output codes J. Artif. Intell. Res. 2 1995 263 286
23. Y.L. Murphey, H. Wang, G. Ou, OAHO: An Effective Algorithm for Multi-Class Learning from Imbalanced Data, in: Proc. Int. Jt. Conf. Neural Networks, 2007, pp. 406-411.
24. R. Anand, K. Mehrotra, C.K. Mohan, and S. Ranka Efficient classification for multiclass problems using modular neural networks IEEE Trans. Neural Networks 6 1995 117 124
25. E.L. Allwein, R.E. Schapire, and Y. Singer Reducing multiclass to binary: a unifying approach for margin classifiers J. Mach. Learn. Res. 1 2000 113 141
26. E. Gelenbe, and K.F. Hussain Learning in the multiple class random neural network IEEE Trans. Neural Networks 13 2002 1257 1267
27. D.M.J. Tax, R.P.W. Duin, Using two-class classifiers for multiclass classification, in: Proc. the16th Int. Conf. Pattern Recognition., 2002, pp. 124-127.
28. P. Kraipeerapun, C. Fung, K. Wong, Multiclass Classification using Neural Networks and Interval Neutrosophic Sets, in: Proc. 5th WSEAS Int. Conf. Comput. Intell. Man-Machine Sist. Cybern., Venice, 2006, pp. 123-128.
29. O. Lézoray, and H. Cardot Comparing combination rules of Pairwise neural networks classifiers Neural Process. Lett. 27 2008 43 56
30. N. García-Pedrajas, and D. Ortiz-Boyer An empirical study of binary classifier fusion methods for multiclass classification Inf. Fusion. 12 2011 111 130
31. M. Galar, A. Fernández, E. Barrenechea, H. Bustince, and F. Herrera An overview of ensemble methods for binary classifiers in multi-class problems: experimental study on one-vs-one and one-vs-all schemes Pattern Recognit. 44 2011 1761 1776
32. T.H. Oong, and N.A. Mat Isa, one-against-all ensemble for multiclass pattern classification Appl. Soft Comput. 12 2012 1303 1308
33. M.P. Sesmero, J.M. Alonso-Weber, G. Gutiérrez, A. Ledezma, and A. Sanchis A new artificial neural network ensemble based on feature selection and class recoding Neural Comput. Appl. 21 2012 771 783
34. I. Iguyon, and A. Elisseeff An introduction to variable and feature selection J. Mach. Learn. Res. 3 2003 1157 1182
35. G. Chandrashekar, and F. Sahin A survey on feature selection methods Comput. Electr. Eng. 40 2014 16 28
36. H. Liu, and L. Yu Toward integrating feature selection algorithms for classification and clustering IEEE Trans. Knowl. Data Eng. 17 2005 491 502
37. B. Frénay, G. Doquire, and M. Verleysen Is mutual information adequate for feature selection in regression? Neural Networks 48 2013 1 7
38. J. Tang, H. Liu, Unsupervised Feature Selection for Linked Social Media Data, in: Proc. 18th ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., ACM, New York, NY, USA, 2012, pp. 904-912.
39. R. Battiti Using mutual information for selecting features in supervised neural net learning Neural Networks, IEEE Trans. 5 1994 537 550
40. G.H. John, R. Kohavi, K. Pfleger, Irrelevant Features and the Subset Selection Problem, in: Mach. Learn. Proc. Elev. Int. Conf., Morgan Kaufmann, 1994, pp. 121-129.
41. M.A. Hall, Correlation-based Feature Selection for Machine Learning, 1999.
42. E. Rich, and K. Knight Artificial Intelligence 1991 McGraw Hill
43. I.H. Witten, E. Frank, and M.A. Hall Data Mining: Practical Machine Learning Tools and Techniques, second ed. 2005 Morgan Kaufmann
44. M.P. Sesmero, J.M. Alonso-Weber, G. Gutierrez, A. Ledezma, A. Sanchis, Testing Feature Selection in Traffic Signs, in: Proc. 11th Int. Conf. Comput. Aided Syst. Theory - EUROCAST 2007, 2007, pp. 396-398.
45. L. Xu, P. Yan, T. Chang, Best First Strategy for Feature Selection, in: 9th Int. Conf. Pattern Recognit., 1988, pp. 706-708.
46. S. Russell, and P. Norvig Artificial Intelligence: A Modern Approach 2003 Prentice Hall
47. D.E. Rumelhart, G.E. Hinton, R.J. Williams, Learning Internal Representations by Error Propagation, in: D.E.R. James, L. McClelland (Eds.), Parallel Distrib. Process. Explor. Microstruct. Cogn., 1985, pp. 318-362.
48. N.C. Oza, and K. Tumer Classifier ensembles: select real-world applications Inf. Fusion. 9 2008 4 20
49. A. Frank, A. Asuncion, UCI Machine Learning Repository, Univ. California, Irvine, Sch. Inf. Comput. Sci., 2010.
50. J. Alcalá-Fdez, L. Sánchez, S. García, S. Ventura, J.M. Garrell, and J. Otero KEEL: a software tool to assess evolutionary algorithms for data mining problems Soft. Comput. 13 2009 307 318
51. M.P. Sesmero, Diseño. Análisis y Evaluación de Conjuntos de Clasificadores basados en Redes de Neuronas, Carlos III de Madrid, 2012.
52. Y. LeCun, THE MNIST DATABASE of handwritten digits, (n.d.).
53. G. Zhang Neural networks for classification: a survey, IEEE Trans Syst. Man, Cybern. Appl. Rev. 30 2000 451 462
54. L. Bruzzone, and S.B. Serpico Classification of imbalanced remote-sensing data by neural networks Pattern Recognit. Lett. 18 1997 1323 1328
55. J. Quinlan Induction of decision trees Mach. Learn. 1 1986 81 106
56. D. Optiz, and R. Maclin Popular ensemble methods: an empirical study J. Artif. Intell. Res. 11 1999 169 198
57. R.P.W. Duin, and D.M.J. Tax Experiments with classifier combining rules Mult. Classif. Syst. Lect. Notes Comput. Sci. 2000 1857 16 29
58. E. Alpaydin Combined 5×2 cv F Test for comparing supervised classification learning algorithms Neural Comput. 11 1999 1885 1892
59. T.G. Dietterich Approximate statistical tests for comparing supervised classification learning algorithms Neural Comput. 10 1998 1895 1923
60. D.J. Sheskin Handbook of Parametric and Nonparametric Statistical Procedures, second ed. 2000 Chapman & Hall/CRC
61. L.I. Kuncheva, C.J. Whitaker, Ten Measures of Diversity in Classifier Ensembles: Limits for Two Classifiers, in: Proc. IEEE Work. Intell. Sens., 2001, pp. 10/1-10/10.
62. L.I. Kuncheva, and C.J. Whitaker Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy Mach. Learn. 51 2003 181 207
63. E. Alpaydin Introduction to Machine Learning, second ed. 2010 MIT Press
64. C.A. Shipp, and L.I. Kuncheva Relationships between combination methods and measures of diversity in combining classifiers Inf. Fusion. 3 2002 135 148
65. J.J. Rodríguez, L.I. Kuncheva, and C.J. Alonso Rotation forest: a new classifier ensemble method IEEE Trans. Pattern Anal. Mach. Intell. 28 2006 1619 1630.