Cost-sensitive decision tree ensembles for effective imbalanced classification

Applied Soft Computing Journal

Volumn 14, Issue PART C, 2014, Pages 554-562

  Krawczyk, Bartosz ^a   Woźniak, Michał ^a   Schaefer, Gerald ^b  

^a WROCLAW UNIVERSITY OF TECHNOLOGY   (Poland)

^b LOUGHBOROUGH UNIVERSITY   (United Kingdom)

Author keywords

Classifier fusion; Classifier selection; Cost sensitive classification; Decision tree; Ensemble classifier; Evolutionary algorithms; Imbalanced classification; Machine learning; Multiple classifier system

Indexed keywords

CLASSIFIER FUSION; CLASSIFIER SELECTION; COST-SENSITIVE CLASSIFICATION; ENSEMBLE CLASSIFIERS; IMBALANCED CLASSIFICATION; MULTIPLE CLASSIFIER SYSTEMS;

COSTS; DATA MINING; DECISION TREES; EVOLUTIONARY ALGORITHMS; LEARNING SYSTEMS;

CLASSIFICATION (OF INFORMATION);

EID: 84889092504     PISSN: 15684946     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.asoc.2013.08.014     Document Type: Article

References (48)

1. J. Alcala-Fdez, A. Ferndedez, J. Luengo, J. Derrac, S. Garcia, L. Sanchez, and F. Herrera Keel data-mining software tool: data set repository integration of algorithms and experimental analysis framework Journal of Multiple-Valued Logic and Soft Computing 17 2-3 2011 255 287
2. E. Alpaydin Combined 5x2 cv f test for comparing supervised classification learning algorithms Neural Computation 11 8 1999 1885 1892
3. Ashlock Daniel Evolutionary Computation for Modeling and Optimization 1st ed. 2006 Springer New York, USA
4. B. Biggio, G. Fumera, and F. Roli Bayesian analysis of linear combiners Proceedings of the 7th international conference on Multiple classifier systems, MCS'07 2007 Springer-Verlag Berlin, Heidelberg 292 301 (Pubitemid 350270626)
5. J. Blaszczynski, M. Deckert, J. Stefanowski, and S. Wilk Integrating selective pre-processing of imbalanced data with ivotes ensemble, vol. 6086 LNAI of Lecture Notes in Computer Science 2010 148 157
6. L. Breiman, J.H. Friedman, R.A. Olshen, and C.J. Stone Classification and Regression Trees 1984 Chapman and Hall
7. N.V. Chawla, K.W. Bowyer, L.O. Hall, and W.P. Kegelmeyer Smote: synthetic minority over-sampling technique Journal of Artificial Intelligence Research 16 2002 321 357 (Pubitemid 43057176)
8. N.V. Chawla, A. Lazarevic, L.O. Hall, and K.W. Bowyer Smoteboost: Improving prediction of the minority class in boosting Lecture Notes in Artificial Intelligence 2838 2003 107 119 (Pubitemid 37231089)
9. S. Chen, H. He, and E.A. Garcia Ramoboost: Ranked minority oversampling in boosting IEEE Transactions on Neural Networks 21 10 2010 1624 1642
10. X. Chen, and M. Wasikowski Fast: a roc-based feature selection metric for small samples and imbalanced data classification problems Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 2008 124 132
11. R.O. Duda, P.E. Hart, and D.G. Stork Pattern Classification 2 ed. 2001 Wiley New York
12. R.P.W. Duin The combining classifier: to train or not to train? In Pattern Recognition, 2002 Proceedings 16th International Conference on 2 2002 765 770 (Pubitemid 44844850)
13. T. Fawcett An introduction to roc analysis Pattern Recognition Letters 27 8 2006 861 874
14. V. Garcia, R.A. Mollineda, and J.S. Sanchez On the k-nn performance in a challenging scenario of imbalance and overlapping Pattern Analysis and Applications 11 3-4 2008 269 280
15. H. He, Y. Bai, E.A. Garcia, and S. Li Adasyn: adaptive synthetic sampling approach for imbalanced learning Proceedings of the International Joint Conference on Neural Networks 2008 1322 1328
16. H. He, and E.A. Garcia Learning from imbalanced data IEEE Transactions on Knowledge and Data Engineering 21 9 2009 1263 1284
17. T.K. Ho The random subspace method for constructing decision forests IEEE Transactions on Pattern Analysis and Machine Intelligence 20 1998 August 832 844 (Pubitemid 128741345)
18. K. Jackowski, and M. Woźniak Algorithm of designing compound recognition system on the basis of combining classifiers with simultaneous splitting feature space into competence areas Pattern Analysis and Applications 12 4 2009 415 425
19. A.K. Jain, R.P.W. Duin, and Jianchang Mao Statistical pattern recognition: a review IEEE Transactions on Pattern Analysis and Machine Intelligence 22 1 2000 jan 4 37 (Pubitemid 32132953)
20. W. Khreich, E. Granger, A. Miri, and R. Sabourin Iterative boolean combination of classifiers in the roc space: an application to anomaly detection with hmms Pattern Recognition 43 8 2010 2732 2752
21. B. Krawczyk Diversity in ensembles for one-class classification M. Pechenizkiy, M. Wojciechowski, New Trends in Databases and Information Systems, volume 185 of Advances in Intelligent Systems and Computing 2012 Springer Berlin Heidelberg 119 129
22. B. Krawczyk Pattern recognition approach to classifying cyp 2c19 isoform Central European Journal of Medicine 7 1 2012 38 44
23. B. Krawczyk, L. Jelen, A. Krzyzak, and T. Fevens Oversampling methods for classification of imbalanced breast cancer malignancy data L. Bolc, R. Tadeusiewicz, L. Chmielewski, K. Wojciechowski, Computer Vision and Graphics, volume 7594 of Lecture Notes in Computer Science 2012 Springer Berlin/Heidelberg 483 490
24. B. Krawczyk, G. Schaefer, and M. Woźniak Breast thermogram analysis using a cost-sensitive multiple classifier system Proceedings of the IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI 2012) 2012 507 510
25. B. Krawczyk, and M. Woźniak Designing cost-sensitive ensemble - genetic approach R. Choras, Image Processing and Communications Challenges 3, Volume 102 of Advances in Intelligent and Soft Computing 2011 Springer Berlin/Heidelberg 227 234
26. B. Krawczyk, and M. Woźniak Combining diverse one-class classifiers E. Corchado, V. Snasel, Ajith Abraham, Michal Wozniak, Manuel Grana, and Sung-Bae Cho, editors, Hybrid Artificial Intelligent Systems, Volume 7209 of Lecture Notes in Computer Science 2012 Springer Berlin/Heidelberg 590 601
27. B. Krawczyk, M. Woźniak, and G. Schaefer Improving minority class prediction using cost-sensitive ensembles 16th WSC Conference on Soft Computing in Industrial Applications 2012
28. L. Kuncheva, J.C. Bezdek, and R.P.W. Duin Decision templates for multiple classifier fusion: an experimental comparison Pattern Recognition 34 2 2001 299 314 (Pubitemid 32871876)
29. L.I. Kuncheva Combining Pattern Classifiers: Methods and Algorithms 2004 Wiley-Interscience
30. C.X. Ling, Q. Yang, J. Wang, and S. Zhang Decision trees with minimal costs Proceedings, Twenty-First International Conference on Machine Learning ICML 2004 2004 544 551 (Pubitemid 40290851)
31. X. Liu, J. Wu, and Z. Zhou Exploratory undersampling for class-imbalance learning IEEE Transactions on Systems Man and Cybernetics Part B: Cybernetics 39 2 2009 539 550
32. Y. Liu, and Y. Chen Face recognition using total margin-based adaptive fuzzy support vector machines IEEE Transactions on Neural Networks 18 1 2007 178 192 (Pubitemid 46062926)
33. V. Lopez, A. Fernandez, J.G. Moreno-Torres, and F. Herrera Analysis of preprocessing vs. cost-sensitive learning for imbalanced classification. open problems on intrinsic data characteristics Expert Systems with Applications 39 7 2012 6585 6608
34. K. Napierala, and J. Stefanowski Identification of different types of minority class examples in imbalanced data Volume 7209 LNAI of Lecture Notes in Computer Science 2012 139 150
35. M. Nuenz The use of background knowledge in decision tree induction Machine Learning 6 1991 231 250
36. M. Srinivas, M. Lalit, and Patnaik Genetic algorithms: a survey Computer 27 June 1994 17 26
37. Y. Sun, M.S. Kamel, A.K.C. Wong, and Y. Wang Cost-sensitive boosting for classification of imbalanced data Pattern Recognition 40 12 2007 3358 3378 (Pubitemid 47223287)
38. Y. Sun, A.K.C. Wong, and M.S. Kamel Classification of imbalanced data: a review International Journal of Pattern Recognition and Artificial Intelligence 23 4 2009 687 719
39. M. van Erp, L. Vuurpijl, and L. Schomaker An overview and comparison of voting methods for pattern recognition Frontiers in Handwriting Recognition. Proceedings Eighth International Workshop on 2002 195 200
40. S. Wang, and X. Yao Diversity analysis on imbalanced data sets by using ensemble models 2009 IEEE Symposium on Computational Intelligence and Data Mining CIDM 2009 - Proceedings 2009 324 331
41. D.H. Wolpert The supervised learning no-free-lunch theorems Proc. 6th Online World Conference on Soft Computing in Industrial Applications 2001 25 42
42. M. Woźniak Experiments on linear combiners E. Pietka, J. Kawa, Information Technologies in Biomedicine, Volume 47 of Advances in Soft Computing 2008 Springer Berlin/Heidelberg 445 452
43. M. Woźniak, and K. Jackowski Some remarks on chosen methods of classifier fusion based on weighted voting E. Corchado, X. Wu, E. Oja, A. Herrero, B. Baruque, Hybrid Artificial Intelligence Systems, Volume 5572 of Lecture Notes in Computer Science 2009 Springer Berlin/Heidelberg 541 548
44. M. Woźniak, and M. Zmyslony Designing combining classifier with trained fuser - analytical and experimental evaluation Neural Network World 20 7 2010 925 934
45. M. Woźniak, and M. Zmyslony Designing Fusers on the Basis of Discriminants - Evolutionary and Neural Methods of Training, Volume 6076 LNAI of Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 2010
46. L. Xu, A. Krzyzak, and C.Y. Suen Methods of combining multiple classifiers and their applications to handwriting recognition IEEE Transactions on Systems, Man and Cybernetics 22 May/June(3) 1992 418 435
47. Z. Yang, W.H. Tang, A. Shintemirov, and Q.H. Wu Association rule mining-based dissolved gas analysis for fault diagnosis of power transformers IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews 39 6 2009 597 610
48. M. Zmyslony, B. Krawczyk, and M. Woźniak Combined classifiers with neural fuser for spam detection A. Herrero, V. Snasel, A. Abraham, I. Zelinka, B. Baruque, H. Quintin, J.L. Calvo, J. Sedano, E. Corchado, International Joint Conference CISIS12-ICEUTE12-SOCO12 Special Sessions, Volume 189 of Advances in Intelligent Systems and Computing 2012 Springer Berlin Heidelberg 245 252