Ordering-based pruning for improving the performance of ensembles of classifiers in the framework of imbalanced datasets

Information Sciences

Volumn 354, Issue , 2016, Pages 178-196

  Galar, Mikel ^a   Fernández, Alberto ^b   Barrenechea, Edurne ^a   Bustince, Humberto ^a   Herrera, Francisco ^c,d  

^a PUBLIC UNIVERSITY OF NAVARRE   (Spain)

^b UNIVERSITY OF JAÉN   (Spain)

^c UNIVERSITY OF GRANADA   (Spain)

^d KING ABDULAZIZ UNIVERSITY   (Saudi Arabia)

Author keywords

Bagging; Boosting; Imbalanced datasets; Ordering based pruning; Tree based ensembles

Indexed keywords

ARTIFICIAL INTELLIGENCE; SOFTWARE ENGINEERING;

BAGGING; BOOSTING; IMBALANCED DATA-SETS; ORDERING-BASED PRUNING; TREE-BASED ENSEMBLES;

CLASSIFICATION (OF INFORMATION);

EID: 84962359556     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2016.02.056     Document Type: Article

References (83)

1. J. Alcala-Fdez, L. Sanchez, S. Garcia, M.J. del Jesus, S. Ventura, J.M. Garrell, J. Otero, C. Romero, J. Bacardit, V.M. Rivas, J.C. Fernandez, and F. Herrera KEEL: a software tool to assess evolutionary algorithms for data mining problems Soft Comput. 13 2009 307 318
2. R. Barandela, J.S. Sanchez, V. García, and E. Rangel Strategies for learning in class imbalance problems Pattern Recognit. 36 3 2003 849 851
3. R. Barandela, R. Valdovinos, and J. Sanchez New applications of ensembles of classifiers Pattern Anal. Appl. 6 3 2003 245 256
4. G.E.A.P.A. Batista, R.C. Prati, and M.C. Monard A study of the behaviour of several methods for balancing machine learning training data SIGKDD Explor. 6 1 2004 20 29
5. M. Bhardwaj, and V. Bhatnagar Towards an optimally pruned classifier ensemble Int. J. Mach. Learn. Cybern. 6 5 2015 699 718
6. J. Blaszczynski, and J. Stefanowski Neighbourhood sampling in bagging for imbalanced data Neurocomputing 150 2015 529 542
7. V. Bolon-Canedo, N.S.-M. no, A. Alonso-Betanzos, J.M. Benitez, and F. Herrera A review of microarray datasets and applied feature selection methods Inf. Sci. 282 2014 111 135
8. L. Breiman Bagging predictors Mach. Learn. 24 2 1996 123 140
9. A. Bria, N. Karssemeijer, and F. Tortorella Learning from unbalanced data: a cascade-based approach for detecting clustered microcalcifications Med. Image Anal. 18 2 2014 241 252
10. I. Brown, and C. Mues An experimental comparison of classification algorithms for imbalanced credit scoring data sets Expert Syst. Appl. 39 3 2012 3446 3453
11. F. Charte, A.J. Rivera, M.J. del Jesus, and F. Herrera Mlsmote: approaching imbalanced multilabel learning through synthetic instance generation Knowl. Based Syst. 89 2015 385 397
12. N.V. Chawla, K.W. Bowyer, L.O. Hall, and W.P. Kegelmeyer SMOTE: synthetic minority over-sampling technique J. Artif. Intell. Res. 16 2002 321 357
13. N.V. Chawla, N. Japkowicz, and A. Kotcz Editorial: special issue on learning from imbalanced data sets SIGKDD Explor. 6 1 2004 1 6
14. N.V. Chawla, A. Lazarevic, L.O. Hall, and K.W. Bowyer SMOTEBoost: improving prediction of the minority class in boosting Proceedings of 7th European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD'03) 2003 107 119
15. C.P. Chen, and C.-Y. Zhang Data-intensive applications, challenges, techniques and technologies: a survey on big data Inf. Sci. 275 2014 314 347
16. S. del Rio, V. Lopez, J.M. Benitez, and F. Herrera On the use of mapreduce for imbalanced big data using random forest Inf. Sci. 285 2014 112 137
17. J. Demšar Statistical comparisons of classifiers over multiple data sets J. Mach. Learn. Res. 7 2006 1 30
18. T.G. Dietterich An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization Mach. Learn. 40 2000 139 157
19. P. Domingos Metacost: A general method for making classifiers cost-sensitive Proceedings of the 5th International Conference on Knowledge Discovery and Data Mining (KDD'99) 1999 155 164
20. A. Fernandez, S. Rio, V. Lopez, A. Bawakid, M. del Jesus, J. Benitez, and F. Herrera Big data with cloud computing: an insight on the computing environment, mapreduce and programming framework WIREs Data Min. Knowl. Discov. 4 5 2014 380 409
21. Y. Freund, and R. Schapire A decision-theoretic generalization of on-line learning and an application to boosting J. Comput. Syst. Sci. 55 1 1997 119 139
22. M. Galar, A. Fernandez, E. Barrenechea, and F. Herrera Eusboost: enhancing ensembles for highly imbalanced data-sets by evolutionary undersampling Pattern Recognit. 46 12 2013 3460 3471
23. M. Galar, A. Fernandez, E. Barrenechea, H. Bustince, and F. Herrera A review on ensembles for class imbalance problem: bagging, boosting and hybrid based approaches IEEE Trans. Syst. Man Cybern. Part C: Appl. Rev. 42 4 2012 463 484
24. S. García, A. Fernandez, J. Luengo, and F. Herrera Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power Inf. Sci. 180 10 2010 2044 2064
25. S. Garcia, and F. Herrera An extension on "statistical comparisons of classifiers over multiple data sets" for all pairwise comparisons J. Mach. Learn. Res. 9 2008 2607 2624
26. S. Garcia, and F. Herrera Evolutionary under-sampling for classification with imbalanced data sets: proposals and taxonomy Evol. Comput. 17 3 2009 275 306
27. A. Ghazikhani, R. Monsefi, and H.S. Yazdi Online neural network model for non-stationary and imbalanced data stream classification Int. J. Mach. Learn. Cybern. 5 1 2014 51 62
28. L. Guo, and S. Boukir Margin-based ordered aggregation for ensemble pruning Pattern Recognit. Lett. 34 6 2013 603 609
29. M.M. Haque, M.K. Skinner, and L.B. Holder Imbalanced class learning in epigenetics J. Comput. Biol. 21 7 2014 492 507
30. H. He, and E.A. Garcia Learning from imbalanced data IEEE Trans. Knowl. Data Eng. 21 9 2009 1263 1284
31. D. Hernandez-Lobato, G.M.-M. noz, and A. Suarez Statistical instance-based pruning in ensembles of independent classifiers IEEE Trans. Pattern Anal. Mach. Intell. 31 2 2009 364 369
32. S. Hido, H. Kashima, and Y. Takahashi Roughly balanced bagging for imbalanced data Stat. Anal. Data Min. 2 2009 412 426
33. S. Holm A simple sequentially rejective multiple test procedure Scand. J. Stat. 6 1979 65 70
34. X. Hu Using rough sets theory and database operations to construct a good ensemble of classifiers for data mining applications N. Cercone, T.Y. Lin, X. Wu, IEEE International Conference in Data Mining (ICDM) 2001 IEEE Computer Society 233 240
35. J. Huang, and C.X. Ling Using AUC and accuracy in evaluating learning algorithms IEEE Trans. Knowl. Data Eng. 17 3 2005 299 310
36. S. Karakatic, and V. Podgorelec Improved classification with allocation method and multiple classifiers Inf. Fusion 31 2016 26 42
37. M.J. Kim, D.K. Kang, and H.B. Kim Geometric mean based boosting algorithm with over-sampling to resolve data imbalance problem for bankruptcy prediction Expert Syst. Appl. 42 3 2015 1074 1082
38. B. Krawczyk, and G. Schaefer A hybrid classifier committee for analysing asymmetry features in breast thermograms Appl. Soft Comput. J. 20 2014 112 118
39. B. Krawczyk, M. Wozniak, and G. Schaefer Cost-sensitive decision tree ensembles for effective imbalanced classification Appl. Soft Comput. 14 2014 554 562
40. L. Kuncheva, C. Whitaker, C. Shipp, and R. Duin Limits on the majority vote accuracy in classifier fusion Pattern Anal. Appl. 6 1 2003 22 31
41. L.I. Kuncheva Diversity in multiple classifier systems Inf. Fusion 6 1 2005 3 4
42. L.I. Kuncheva, and J.J. Rodriguez A weighted voting framework for classifiers ensembles Knowl. Inf. Syst. 38 2 2014 259 275
43. L.I. Kuncheva, and C.J. Whitaker Measures of diversity in classifier ensembles Mach. Learn. 51 2003 181 207
44. X.-Y. Liu, J. Wu, and Z.-H. Zhou Exploratory undersampling for class-imbalance learning IEEE 39 2 2009 539 550
45. V. Lopez, A. Fernandez, M.D. Jesus, and F. Herrera A hierarchical genetic fuzzy system based on genetic programming for addressing classification with highly imbalanced and borderline data-sets Knowl. Based Syst. 38 2013 85 104
46. V. Lopez, A. Fernandez, S. Garcia, V. Palade, and F. Herrera An insight into classification with imbalanced data: empirical results and current trends on using data intrinsic characteristics Inf. Sci. 250 20 2013 113 141
47. V. Lopez, A. Fernandez, and F. Herrera On the importance of the validation technique for classification with imbalanced datasets: addressing covariate shift when data is skewed Inf. Sci. 257 2014 1 13
48. V. Lopez, A. Fernandez, J.G. Moreno-Torres, and F. Herrera Analysis of preprocessing vs. cost-sensitive learning for imbalanced classification. Open problems on intrinsic data characteristics Expert Syst. Appl. 39 7 2012 6585 6608
49. Z. Lu, X. Wu, X. Zhu, and J. Bongard Ensemble pruning via individual contribution ordering B. Rao, B. Krishnapuram, A. Tomkins, Y. Qiang, KDD 2010 ACM 871 880
50. D.D. Margineantu, and T.G. Dietterich Pruning adaptive boosting D.H. Fisher, 14th International Conference on Machine Learning (ICML) 1997 Morgan Kaufmann 211 218
51. G.M.-M. noz, D. Hernandez-Lobato, and A. Suarez An analysis of ensemble pruning techniques based on ordered aggregation IEEE Trans. Pattern Anal. Mach. Intell. 31 2 2009 245 259
52. G.M.-M. noz, and A. Suarez Using boosting to prune bagging ensembles Pattern Recognit. Lett. 28 1 2007 156 165
53. L.L. Minku, A.P. White, and X. Yao The impact of diversity on online ensemble learning in the presence of concept drift IEEE Trans. Knowl. Data Eng. 22 5 2010 730 742
54. L.L. Minku, and X. Yao Ddd: a new ensemble approach for dealing with concept drift IEEE Trans. Knowl. Data Eng. 24 4 2012 619 633
55. J.G. Moreno-Torres, T. Raeder, R. Alaiz-Rodriguez, N.V. Chawla, and F. Herrera A unifying view on dataset shift in classification Pattern Recognit. 45 1 2012 521 530
56. J.G. Moreno-Torres, J.A. Saez, and F. Herrera Study on the impact of partition-induced dataset shift on k-fold cross-validation IEEE Trans. Neural Netw. Learn. Syst. 23 8 2012 1304 1313
57. G.M.-M. noz, and A. Suarez Aggregation ordering in bagging International Confernece on Artificial Intelligence and Applications (IASTED) 2004 258 263
58. Y. Park, and J. Ghosh Ensembles of α-trees for imbalanced classification problems IEEE Trans. Knowl. Data Eng. 26 1 2014 131 143
59. R. Polikar Ensemble based systems in decision making IEEE Circuits Syst. Mag. 6 3 2006 21 45
60. R.C. Prati, G.E.A.P.A. Batista, and D.F. Silva Class imbalance revisited: a new experimental setup to assess the performance of treatment methods Knowl. Inf. Syst. 45 1 2015 247 270
61. J. Quinlan C4.5: Programs for Machine Learning 1993 Morgan Kauffman
62. L. Rokach Ensemble-based classifiers Artif. Intell. Rev. 33 1 2010 1 39
63. C. Seiffert, T. Khoshgoftaar, J.V. Hulse, and A. Napolitano RUSBoost: a hybrid approach to alleviating class imbalance IEEE 40 1 2010 185 197
64. J. Stefanowski Dealing with data difficulty factors while learning from imbalanced data S. Matwin, J. Mielniczuk, Challenges in Computational Statistics and Data Mining Studies in Computational Intelligence vol. 605 2016 Springer 333 363
65. Y. Sun, M.S. Kamel, A.K.C. Wong, and Y. Wang Cost-sensitive boosting for classification of imbalanced data Pattern Recognit. 40 12 2007 3358 3378
66. Y. Sun, A.K.C. Wong, and M.S. Kamel Classification of imbalanced data: a review Int. J. Pattern Recognit. Artif. Intell. 23 4 2009 687 719
67. C. Tamon, and J. Xiang On the boosting pruning problem R.L. de Mantaras, E. Plaza, 11th European Conference on Machine Learning (ECML) Lecture Notes in Computer Science vol. 1810 2000 Springer 404 412
68. K. Tumer, and J. Ghosh Error correlation and error reduction in ensemble classifiers Connect. Sci. 8 3-4 1996 385 403
69. I. Visentini, L. Snidaro, and G.L. Foresti Diversity-aware classifier ensemble selection via f-score Inf. Fusion 28 2016 24 43
70. S. Vluymans, D.S. Tarrago, Y. Saeys, C. Cornelis, F. Herrera Fuzzy rough classifiers for class imbalanced multi-instance data Pattern Recognit. 53 2016 36 45
71. S. Wang, L.L. Minku, and X. Yao Resampling-based ensemble methods for online class imbalance learning IEEE Trans. Knowl. Data Eng. 27 5 2015 1356 1368
72. S. Wang, and X. Yao Diversity analysis on imbalanced data sets by using ensemble models Proceedings of the 2009 IEEE Symposium on Computational Intelligence and Data Mining (CIDM'09) 2009 324 331
73. S. Wang, and X. Yao Relationships between diversity of classification ensembles and single-class performance measures IEEE Trans. Knowl. Data Eng. 25 1 2013 206 219
74. F. Wilcoxon Individual comparisons by ranking methods Biom. Bull. 1 6 1945 80 83
75. M. Wozniak, M.G. na, and E. Corchado A survey of multiple classifier systems as hybrid systems Inf. Fusion 16 2014 3 17
76. X. Wu, V. Kumar, J.R. Quinlan, J. Ghosh, Q. Yang, H. Motoda, G.J. McLachlan, A. Ng, B. Liu, P.S. Yu, Z.-H. Zhou, M. Steinbach, D.J. Hand, and D. Steinberg Top 10 algorithms in data mining Knowl. Inf. Syst. 14 1 2007 1 37
77. G. Yule On the association of attributes in statistics Philos. Trans. A 194 1900 257 319
78. B. Zadrozny, and C. Elkan Learning and making decisions when costs and probabilities are both unknown Proceedings of the 7th International Conference on Knowledge Discovery and Data Mining (KDD'01) 2001 204 213
79. J.H. Zar Biostatistical Analysis 1999 Prentice Hall Upper Saddle River, New Jersey
80. J. Zhai, S. Zhang, and C. Wang The classification of imbalanced large data sets based on mapreduce and ensemble of elm classifiers International Journal of Machine Learning and Cybernetics 2016 1 9 10.1007/s13042-015-0478-7 in press.
81. H. Zhang, and M. Li Rwo-sampling: A random walk over-sampling approach to imbalanced data classification Inf. Fusion 20 2014 99 116
82. Y. Zhang, S. Burer, and W.N. Street Ensemble pruning via semi-definite programming J. Mach. Learn. Res. 7 2006 1315 1338
83. Z.H. Zhou, J. Wu, and W. Tang Ensembling neural networks: Many could be better than all Artif. Intell. 137 2002 239 263