Empowering one-vs-one decomposition with ensemble learning for multi-class imbalanced data

Knowledge-Based Systems

Volumn 106, Issue , 2016, Pages 251-263

  Zhang, Zhongliang ^a   Krawczyk, Bartosz ^b   Garcìa, Salvador ^c   Rosales Pérez, Alejandro ^d   Herrera, Francisco ^c,e  

^a NORTHEASTERN UNIVERSITY   (China)

^b WROCLAW UNIVERSITY OF TECHNOLOGY   (Poland)

^c UNIVERSITY OF GRANADA   (Spain)

^d INSTITUTO NACIONAL DE ASTROFÍSICA   (Mexico)

^e KING ABDULAZIZ UNIVERSITY   (Saudi Arabia)

Author keywords

Binary decomposition; Classifier combination; Ensemble learning; Imbalanced data; Multi class classification

Indexed keywords

LEARNING SYSTEMS; PROBLEM SOLVING;

BINARY DECOMPOSITIONS; CLASSIFIER COMBINATION; ENSEMBLE LEARNING; IMBALANCED DATA; MULTI-CLASS CLASSIFICATION;

CLASSIFICATION (OF INFORMATION);

EID: 84971659662     PISSN: 09507051     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.knosys.2016.05.048     Document Type: Article

References (63)

1. M.H.B. Abidine, and B. Fergani A new multi-class wsvm classification to imbalanced human activity dataset J.Comput. 9 7 2014 1560 1565
2. R. Baeza-Yates, and B. Ribeiro-Neto Modern Information Retrieval 1999 ACM press New York
3. R. Barandela, R.M. Valdovinos, and J.S. Sanchez New applications of ensembles of classifiers Pattern Anal. Appl. 6 3 2003 245 256
4. G.E.A.P.A. Batista, R.C. Prati, and M.C. Monard A study of the behavior of several methods for balancing machine learning training data ACM SIGKDD Explor. Newslett. 6 1 2004 20 29
5. J. Blaszczynski, and J. Stefanowski Neighbourhood sampling in bagging for imbalanced data Neurocomputing 150 2015 529 542
6. L. Breiman, J. Friedman, C.J. Stone, and R.A. Olshen Classification and Regression Trees 1984 CRC press
7. L. Cerf, D. Gay, N. Selmaoui-Folcher, B. Cremilleux, and J.F. Boulicaut Parameter-free classification in multi-class imbalanced data sets Data Knowl. Eng. 87 2013 109 129
8. N.V. Chawla, K.W. Bowyer, L.O. Hall, and W.P. Kegelmeyer Smote: synthetic minority over-sampling technique J. Artif. Intell. Res. 16 2002 321 357
9. N.V. Chawla, N. Japkowicz, and A. Kotcz Editorial: special issue on learning from imbalanced data sets ACM SIGKDD Explor. Newslett. 6 1 2004 1 6
10. N.V. Chawla, A. Lazarevic, L.O. Hall, and K.W. Bowyer Knowledge Discovery in Databases: PKDD 2003 Chapter SMOTEBoost: Improving prediction of the minority class in boosting Lecture Notes in Computer Science vol. 2838 2003 Springer-Verlag 107 119
11. D.A. Cieslak, T.R. Hoens, N.V. Chawla, and W.P. Kegelmeyer Hellinger distance decision trees are robust and skew-insensitive Data Min. Knowl. Discov. 24 1 2012 136 158
12. W.M. Czarnecki, and J. Tabor Two ellipsoid support vector machines Expert Syst. Appl. 41 18 2014 8211 8224
13. W.M. Czarnecki, and J. Tabor Multithreshold entropy linear classifier: theory and applications Expert Syst. Appl. 42 13 2015 5591 5606
14. S. Datta, and S. Das Near-bayesian support vector machines for imbalanced data classification with equal or unequal misclassification costs Neural Netw. 70 2015 39 52
15. J.-F. Díez-Pastor, J.-J. Rodríguez Diez, C. García-Osorio, and L.I. Kuncheva Random balance: ensembles of variable priors classifiers for imbalanced data Knowl.Based Syst. 85 2015 96 111
16. T. Fawcett An introduction to ROC analysis Pattern Recog. lett. 27 8 2006 861 874
17. R. Feraud, and F. Clerot A methodology to explain neural network classification Neural Netw. 15 2 2002 237 246
18. A. Fernandez, V. Lopez, M. Galar, M.J. del Jesus, and F. Herrera Analysing the classification of imbalanced data-sets with multiple classes: binarization techniques and ad-hoc approaches Knowl.Based Syst. 42 2013 97 110
19. F. Fernandez-Navarro, C. Hervas-Martȡnez, and P.A. Gutierrez A dynamic over-sampling procedure based on sensitivity for multi-class problems Pattern Recog. 44 8 2011 1821 1833
20. C. Ferri, J. Hernandez-Orallo, and R. Modroiu An experimental comparison of performance measures for classification Pattern Recog. Lett. 30 1 2009 27 38
21. J.H. Friedman Another Approach to Polychotomous Classification Technical report 1996
22. M. Galar, A. Fernandez, E. Barrenechea, H. Bustince, and F. Herrera A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches IEEE Trans. Syst. Man Cybernetics-Part C 42 4 2012 463 484
23. M. Galar, A. Fernandez, E. Barrenechea, and F. Herrera Empowering difficult classes with a similarity-based aggregation in multi-class classification problems Inform. Sci. 264 2014 135 157
24. M. Galar, A. FernÈíndez, E. Barrenechea, H. Bustince, and F. Herrera An overview of ensemble methods for binary classifiers in multi-class problems: Experimental study on one-vs-one and one-vs-all schemes Pattern Recog. 44 8 2011 1761 1776
25. M. Galar, A. FernÈíndez, E. Barrenechea, H. Bustince, and F. Herrera Dynamic classifier selection for one-vs-one strategy: avoiding non-competent classifiers Pattern Recog. 46 12 2013 3412 3424
26. M. Galar, A. FernÈíndez, E. Barrenechea, and F. Herrera Drcw-ovo: distance-based relative competence weighting combination for one-vs-one strategy in multi-class problems Pattern Recog. 48 1 2015 28 42
27. S. Garcia, J. Derrac, I. Triguero, C.J. Carmona, and F. Herrera Evolutionary-based selection of generalized instances for imbalanced classification Knowl. Based Syst. 25 1 2012 3 12
28. S. Garcia, A. Fernandez, J. Luengo, and F. Herrera Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power Inform. Sci. 180 10 2010 2044 2064
29. S. García, J. Luengo, and F. Herrera Tutorial on practical tips of the most influential data preprocessing algorithms in data mining Knowl. Based Syst. 98 2016 1 29
30. Guo H.; Li Y.; Li Y.; Liu X.; and Li J. Bpso-adaboost-knn ensemble learning algorithm for multi-class imbalanced data classification Eng. Appl. AI 49 2016 176 193
31. H. Haibo, and E.A. Garcia Learning from imbalanced data IEEE Trans. Knowl. Data Eng. 21 9 2009 1263 1284
32. D.J. Hand, and R.J. Till A simple generalisation of the area under the ROC curve for multiple class classification problems Mach. Learn. 45 2 2001 171 186
33. Kang S.; Cho S.; and Kang P. Constructing a multi-class classifier using one-against-one approach with different binary classifiers Neurocomputing 149 2015 677 682
34. B. Krawczyk Combining one-vs-one decomposition and ensemble learning for multi-class imbalanced data R. Burduk, K. Jackowski, M. Kurzyński, M. Woźniak, A. Żołnierek, Proceedings of the 9th International Conference on Computer Recognition Systems CORES 2015 2016 27 36
35. B. Krawczyk, M. Galar, Ł. Jeleń, and F. Herrera Evolutionary undersampling boosting for imbalanced classification of breast cancer malignancy Appl. Soft Comput. 2015 http://dx.doi.org/10.1016/j.asoc.2015.08.060
36. B. Krawczyk, M. Woźniak, and G. Schaefer Cost-sensitive decision tree ensembles for effective imbalanced classification Appl. Soft Comput. 14 2014 554 562
37. M. Kurzyński, and M. Woźniak Combining classifiers under probabilistic models: experimental comparative analysis of methods Expert Syst. 29 4 2012 374 393
38. Li Y.; Guo H.; Liu X.; Li Y.; and Li J. Adapted ensemble classification algorithm based on multiple classifier system and feature selection for classifying multi-class imbalanced data Knowl. Based Syst. 94 2016 88 104
39. Liu B.; Hao Z.F.; and Tsang E.C.C. Nesting one-against-one algorithm based on SVMS for pattern classification IEEE Trans. Neural Netw. 19 12 2008 2044 2052
40. Liu X.Y.; Wu J.X.; and Zhou Z.H. Exploratory undersampling for class-imbalance learning IEEE Trans. Syst. Man Cybernetics-Part B 39 2 2009 539 550
41. V. Lopez, A. Fernandez, S. Garcia, V. Palade, and F. Herrera An insight into classification with imbalanced data: empirical results and current trends on using data intrinsic characteristics Inform. Sci. 250 2013 113 141
42. V. Lopez, A. Fernandez, and F. Herrera On the importance of the validation technique for classification with imbalanced datasets: Addressing covariate shift when data is skewed Inform. Sci. 257 2014 1 13
43. J.G. Moreno-Torres, J.A. Saez, and F. Herrera Study on the impact of partition-induced dataset shift on k-fold cross-validation IEEE Trans. Neural Netw. Learn. Syst. 23 8 2012 1304 1312
44. W. Penar, and M. Woźniak Cost-sensitive methods of constructing hierarchical classifiers Expert Syst. 27 3 2010 146 155
45. M.D. Pérez-Godoy, A.J. Rivera, C.J. Carmona, and M.J. del Jesús Training algorithms for Radial Basis Function Networks to tackle learning processes with imbalanced data-sets Appl. Soft. Comput. 25 2014 26 39
46. W. Prachuabsupakij, and N. Soonthornphisaj Clustering and combined sampling approaches for multi-class imbalanced data classification Adv. Inform. Technol. Indus. Appl. 136 2012 717 724
47. P. Pramokchon, and P. Piamsanga Recent advances in information and communication technology Chapter Reducing Effects of Class Imbalance Distribution in Multi-class Text Categorization Advances in Intelligent Systems and Computing vol. 265 2014 Springer-Verlag 263 272
48. J.A. Sáez, B. Krawczyk, M. Woźniak, Analyzing the oversampling of different classes and types of examples in multi-class imbalanced datasets, Pattern Recog. 10.1016/j.patcog.2016.03.012.
49. C. Seiffert, T.M. Khoshgoftaar, J. Van Hulse, and A. Napolitano Rusboost: a hybrid approach to alleviating class imbalance IEEE Trans. Syst. Man Cybernetics-Part A 40 1 2010 185 197
50. Sun Y.; Wong A.K.C.; and M.S. Kamel Classification of imbalanced data: a review J. Pattern Recog. Artif. Intell. 23 4 2009 687 719
51. M.A. Tahir, J. Kittler, and F. Yan Inverse random under sampling for class imbalance problem and its application to multi-label classification Pattern Recog. 45 10 2012 3738 3750
52. Tang Y.C.; Zhang Y.Q.; N.V. Chawla, and S. Krasser Svms modeling for highly imbalanced classification IEEE Trans. Syst. Man Cybernetics-Part B 39 1 2009 281 288
53. P. Trajdos, and M. Kurzyński An extension of multi-label binary relevance models based on randomized reference classifier and local fuzzy confusion matrix Intelligent Data Engineering and Automated Learning - IDEAL 2015 - 16th International Conference Wroclaw, Poland, October 14-16, 2015, Proceedings 2015 69 76
54. V. Vapnik Statistical Learning Theory 1998 Wiley New York
55. P. Vorraboot, S. Rasmequan, K. Chinnasarn, and C. Lursinsap Improving classification rate constrained to imbalanced data between overlapped and non-overlapped regions by hybrid algorithms Neurocomputing 152 2015 429 443
56. Wang S.; and Yao X. Diversity analysis on imbalanced data sets by using ensemble models IEEE Symposium on Computational Intelligence and Data Mining, 2009 2009 324 331
57. Wang S.; and Yao X. Multiclass imbalance problems: analysis and potential solutions IEEE Trans. Syst. Man Cybernetics-Part B 42 4 2012 1119 1130
58. G. Weiss, and Y. Tian Maximizing classifier utility when there are data acquisition and modeling costs Data Min. Knowl. Discov. 17 2 2008 253 282
59. G.M. Weiss Mining with rarity: a unifying framework ACM SIGKDD Explor. Newslett. 6 1 2004 7 19
60. F. Wilcoxon Individual comparisons by ranking methods Biomet. Bull. 1 6 1945 80 83
61. T. Wilk, and M. Woźniak Soft computing methods applied to combination of one-class classifiers Neurocomputing 75 1 2012 185 193
62. M. Woźniak, M. Graña, and E. Corchado A survey of multiple classifier systems as hybrid systems Inform. Fusion 16 2014 3 17
63. Zhou Z.H.; and Liu X.Y. On multi-class cost-sensitive learning Comput. Intell. 26 3 2010 232 257