An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics

Information Sciences

Volumn 250, Issue , 2013, Pages 113-141

  López, Victoria ^a   Fernández, Alberto ^b   García, Salvador ^b   Palade, Vasile ^c   Herrera, Francisco ^a  

^a UNIVERSITY OF GRANADA   (Spain)

^b UNIVERSITY OF JAÉN   (Spain)

^c UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

Cost sensitive learning; Dataset shift; Imbalanced dataset; Noisy data; Sampling; Small disjuncts

Indexed keywords

COST-SENSITIVE LEARNING; DATASET SHIFTS; IMBALANCED DATASET; NOISY DATA; SMALL DISJUNCTS;

LEARNING ALGORITHMS; SAMPLING;

STATISTICAL TESTS;

EID: 84883447718     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2013.07.007     Document Type: Article

References (152)

1. R. Alaiz-Rodríguez, A. Guerrero-Curieses, and J. Cid-Sueiro Improving classification under changes in class and within-class distributions Proceedings of the 10th International Work-Conference on Artificial Neural Networks (IWANN '09) 2009 Springer-Verlag Berlin, Heidelberg 122 130
2. R. Alaiz-Rodríguez, and N. Japkowicz Assessing the impact of changing environments on classifier performance Proceedings of the 21st Canadian Conference on Advances in Artificial Intelligence (CCAI'08) 2008 Springer-Verlag Berlin, Heidelberg 13 24
3. J. Alcalá-Fdez, A. Fernández, J. Luengo, J. Derrac, S. García, L. Sánchez, and F. Herrera KEEL data-mining software tool: data set repository, integration of algorithms and experimental analysis framework Journal of Multi-Valued Logic and Soft Computing 17 2-3 2011 255 287
4. J. Alcalá;-Fdez, L. Sánchez, S. García, M.J. del Jesus, S. Ventura, J.M. Garrell, J. Otero, C. Romero, J. Bacardit, V.M. Rivas, J.C. Fernández, and F. Herrera KEEL: a software tool to assess evolutionary algorithms for data mining problems Soft Computing 13 2009 307 318
5. A. Anand, G. Pugalenthi, G.B. Fogel, and P.N. Suganthan An approach for classification of highly imbalanced data using weighting and undersampling Amino Acids 39 5 2010 1385 1391
6. R. Baeza-Yates, and B. Ribeiro-Neto Modern Information Retrieval 1999 Addison Wesley
7. R. Barandela, J.S. Sánchez, V. García, and E. Rangel Strategies for learning in class imbalance problems Pattern Recognition 36 3 2003 849 851
8. R. Barandela, R.M. Valdovinos, and J.S. Sánchez New applications of ensembles of classifiers Pattern Analysis Applications 6 3 2003 245 256
9. G.E.A.P.A. Batista, R.C. Prati, and M.C. Monard A study of the behaviour of several methods for balancing machine learning training data SIGKDD Explorations 6 1 2004 20 29
10. R. Batuwita, V. Palade, AGm: a new performance measure for class imbalance learning. application to bioinformatics problems, in: Proceedings of the 8th International Conference on Machine Learning and Applications (ICMLA 2009), 2009, pp. 545-550.
11. R. Batuwita, and V. Palade microPred: effective classification of pre-miRNAs for human miRNA gene prediction Bioinformatics 25 8 2009 989 995
12. R. Batuwita, V. Palade, Efficient resampling methods for training support vector machines with imbalanced datasets, in: Proceedings of the 2010 International Joint Conference on Neural Networks (IJCNN), 2010.
13. R. Batuwita, and V. Palade FSVM-CIL: fuzzy support vector machines for class imbalance learning IEEE Transactions on Fuzzy Systems 18 3 2010 558 571
14. R. Batuwita, and V. Palade Adjusted geometric-mean: a novel performance measure for imbalanced bioinformatics datasets learning Journal of Bioinformatics and Computational Biology 10 4 2012
15. R. Batuwita, and V. Palade Class imbalance learning methods for support vector machines H. He, Y. Ma, Imbalanced Learning: Foundations, Algorithms, and Applications 2013 Wiley 83 96
16. S. Bickel, M. Brückner, and T. Scheffer Discriminative learning under covariate shift Journal of Machine Learning Research 10 2009 2137 2155
17. J. Błaszczyński, M. Deckert, J. Stefanowski, and S. Wilk Integrating selective pre-processing of imbalanced data with ivotes ensemble M. Szczuka, M. Kryszkiewicz, S. Ramanna, R. Jensen, Q. Hu, Rough Sets and Current Trends in Computing LNSC vol. 6086 2010 Springer Berlin/Heidelberg 148 157
18. J.P. Bradford, C. Kunz, R. Kohavi, C. Brunk, C.E. Brodley, Pruning decision trees with misclassification costs, in: Proceedings of the 10th European Conference on Machine Learning (ECML'98), 1998, pp. 131-136.
19. A.P. Bradley The use of the area under the roc curve in the evaluation of machine learning algorithms Pattern Recognition 30 7 1997 1145 1159
20. C.E. Brodley, and M.A. Friedl Identifying mislabeled training data Journal of Artificial Intelligence Research 11 1999 131 167
21. C. Bunkhumpornpat, K. Sinapiromsaran, C. Lursinsap, Safe-level-SMOTE: Safe-level-synthetic minority over-sampling TEchnique for handling the class imbalanced problem. In: Proceedings of the 13th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining PAKDD'09, 2009, pp. 475-482.
22. C. Bunkhumpornpat, K. Sinapiromsaran, and C. Lursinsap DBSMOTE: density-based synthetic minority over-sampling technique Applied Intelligence 36 3 2012 664 684
23. J.Q. Candela, M. Sugiyama, A. Schwaighofer, and N.D. Lawrence Dataset Shift in Machine Learning 2009 The MIT Press
24. D.R. Carvalho, and A.A. Freitas A hybrid decision tree/genetic algorithm method for data mining Information Sciences 163 1-3 2004 13 35
25. P.K. Chan, S.J. Stolfo, Toward scalable learning with non-uniform class and cost distributions: a case study in credit card fraud detection, in: Proceedings of the 4th International Conference on Knowledge Discovery and Data Mining (KDD'98). 1998. pp. 164-168.
26. E.Y. Chang, B. Li, G. Wu, K. Goh, Statistical learning for effective visual information retrieval, in: Proceedings of the 2003 International Conference on Image Processing (ICIP'03), vol. 3, 2003, pp. 609-612.
27. N.V. Chawla, K.W. Bowyer, L.O. Hall, and W.P. Kegelmeyer SMOTE: synthetic minority over-sampling technique Journal of Artificial Intelligent Research 16 2002 321 357
28. N.V. Chawla, D.A. Cieslak, L.O. Hall, and A. Joshi Automatically countering imbalance and its empirical relationship to cost Data Mining and Knowledge Discovery 17 2 2008 225 252
29. N.V. Chawla, N. Japkowicz, and A. Kotcz Editorial: special issue on learning from imbalanced data sets SIGKDD Explorations 6 1 2004 1 6
30. N.V. Chawla, A. Lazarevic, L.O. Hall, K.W. Bowyer, SMOTEBoost: Improving prediction of the minority class in boosting, in: Proceedings of 7th European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD'03), 2003, pp. 107-119.
31. X. Chen, T. Fang, H. Huo, and D. Li Graph-based feature selection for object-oriented classification in VHR airborne imagery IEEE Transactions on Geoscience and Remote Sensing 49 1 2011 353 365
32. D.A. Cieslak, N.V. Chawla, Analyzing pets on imbalanced datasets when training and testing class distributions differ, in: Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD08). Osaka, Japan, 2008, pp. 519-526.
33. D.A. Cieslak, and N.V. Chawla A framework for monitoring classifiers' performance: when and why failure occurs? Knowledge and Information Systems 18 1 2009 83 108
34. G. Cohen, M. Hilario, H. Sax, S. Hugonnet, and A. Geissbuhler Learning from imbalanced data in surveillance of nosocomial infection Artificial Intelligence in Medicine 37 2006 7 18
35. C. Cortes, and V. Vapnik Support vector networks Machine Learning 20 1995 273 297
36. J. Davis, and M. Goadrich The relationship between precisionrecall and ROC curves Proceedings of the 23th International Conference on Machine Learning (ICML'06) 2006 ACM 233 240
37. M. Denil, T. Trappenberg, Overlap versus imbalance, in: Proceedings of the 23rd Canadian Conference on Advances in Artificial Intelligence (CCAI'10), Lecture Notes on Artificial Intelligence, vol. 6085, 2010, pp. 220-231.
38. P. Domingos, Metacost: a general method for making classifiers cost-sensitive, in: Proceedings of the 5th International Conference on Knowledge Discovery and Data Mining (KDD'99), 1999, pp. 155-164.
39. D.J. Drown, T.M. Khoshgoftaar, and N. Seliya Evolutionary sampling and software quality modeling of high-assurance systems IEEE Transactions on Systems, Man, and Cybernetics, Part A 39 5 2009 1097 1107
40. C. Drummond, and R.C. Holte Cost curves: an improved method for visualizing classifier performance Machine Learning 65 1 2006 95 130
41. P. Ducange, B. Lazzerini, and F. Marcelloni Multi-objective genetic fuzzy classifiers for imbalanced and cost-sensitive datasets Soft Computing 14 7 2010 713 728
42. C. Elkan, The foundations of cost-sensitive learning, in: Proceedings of the 17th IEEE International Joint Conference on Artificial Intelligence (IJCAI'01), 2001, pp. 973-978.
43. A. Estabrooks, T. Jo, and N. Japkowicz A multiple resampling method for learning from imbalanced data sets Computational Intelligence 20 1 2004 18 36
44. W. Fan, S.J. Stolfo, J. Zhang, and P.K. Chan Adacost: misclassification cost-sensitive boosting Proceedings of the 16th International Conference on Machine Learning (ICML'96) 1999 Morgan Kaufmann Publishers Inc. San Francisco, CA, USA 97 105
45. A. Fernández, M.J. del Jesus, and F. Herrera On the 2-tuples based genetic tuning performance for fuzzy rule based classification systems in imbalanced data-sets Information Sciences 180 8 2010 1268 1291
46. A. Fernández, S. García, M.J. del Jesus, and F. Herrera A study of the behaviour of linguistic fuzzy rule based classification systems in the framework of imbalanced data-sets Fuzzy Sets and Systems 159 18 2008 2378 2398
47. A. Fernandez, S. García, J. Luengo, E. Bernadó-Mansilla, and F. Herrera Genetics-based machine learning for rule induction: state of the art, taxonomy and comparative study IEEE Transactions on Evolutionary Computation 14 6 2010 913 941
48. A. Fernández, V. López, M. Galar, M.J. del Jesus, and F. Herrera Analysing the classification of imbalanced data-sets with multiple classes: binarization techniques and ad-hoc approaches Knowledge-Based Systems 42 2013 97 110
49. J.H. Friedman, R. Kohavi, Y. Yun, Lazy decision trees, in: Proceedings of the AAAI/IAAI, vol. 1, 1996, pp. 717-724.
50. M. Galar, A. Fernández, E. Barrenechea, H. Bustince, and F. Herrera A review on ensembles for class imbalance problem: bagging, boosting and hybrid based approaches IEEE Transactions on Systems, Man, and Cybernetics - part C: Applications and Reviews 42 4 2012 463 484
51. S. García, J. Derrac, I. Triguero, C.J. Carmona, and F. Herrera Evolutionary-based selection of generalized instances for imbalanced classification Knowledge Based Systems 25 1 2012 3 12
52. S. García, A. Fernández, and F. Herrera Enhancing the effectiveness and interpretability of decision tree and rule induction classifiers with evolutionary training set selection over imbalanced problems Applied Soft Computing 9 2009 1304 1314
53. S. García, and F. Herrera Evolutionary under-sampling for classification with imbalanced data sets: proposals and taxonomy Evolutionary Computation 17 3 2009 275 306
54. V. García, R.A. Mollineda, J.S. Sánchez, A new performance evaluation method for two-class imbalanced problems, in: Proceedings of the Structural and Syntactic Pattern Recognition (SSPR'08) and Statistical Techniques in Pattern Recognition (SPR'08), Lecture Notes on Computer Science, vol. 5342, 2008, pp. 917-925.
55. V. García, R.A. Mollineda, and J.S. Sánchez On the k-NN performance in a challenging scenario of imbalance and overlapping Pattern Analysis Applications 11 3-4 2008 269 280
56. V. García, R.A. Mollineda, J.S. Sánchez, Theoretical analysis of a performance measure for imbalanced data, in: 20th International Conference on Pattern Recognition (ICPR'10), 2010, pp. 617-620.
57. V. García, R.A. Mollineda, and J.S. Sánchez Classifier performance assessment in two-class imbalanced problems Internal Communication. 2012
58. V. García, J.S. Sánchez, and R.A. Mollineda On the effectiveness of preprocessing methods when dealing with different levels of class imbalance Knowledge Based Systems 25 1 2012 13 21
59. N. García-Pedrajas, J. Pérez-Rodríguez, M. García-Pedrajas, D. Ortiz-Boyer, and C. Fyfe Class imbalance methods for translation initiation site recognition in DNA sequences Knowledge Based Systems 25 1 2012 22 34
60. A. Globerson, C.H. Teo, A. Smola, and S. Roweis An adversarial view of covariate shift and a minimax approach J. Quiñonero Candela, M. Sugiyama, A. Schwaighofer, N.D. Lawrence, Dataset Shift in Machine Learning 2009 The MIT Press 179 198
61. H. Guo, and H.L. Viktor Learning from imbalanced data sets with boosting and data generation: the DataBoost-IM approach SIGKDD Explorations Newsletter 6 2004 30 39
62. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I.H. Witten The WEKA data mining software: an update SIGKDD Explorations 11 1 2009 10 18
63. H. Han, W.Y. Wang, B.H. Mao, Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning, in: Proceedings of the 2005 International Conference on Intelligent Computing (ICIC'05), Lecture Notes in Computer Science, vol. 3644, 2005, pp. 878-887.
64. P.E. Hart The condensed nearest neighbor rule IEEE Transactions on Information Theory 14 1968 515 516
65. H. He, Y. Bai, E.A. Garcia, S. Li, ADASYN: adaptive synthetic sampling approach for imbalanced learning, in: Proceedings of the 2008 IEEE International Joint Conference on Neural Networks (IJCNN'08), 2008, pp. 1322-1328.
66. H. He, and E.A. Garcia Learning from imbalanced data IEEE Transactions on Knowledge and Data Engineering 21 9 2009 1263 1284
67. S. Hido, H. Kashima, and Y. Takahashi Roughly balanced bagging for imbalanced data Statistical Analysis and Data Mining 2 2009 412 426
68. R.C. Holte, L. Acker, B.W. Porter, Concept learning and the problem of small disjuncts, in: Proceedings of the International Joint Conferences on Artificial Intelligence, IJCAI'89, 1989, pp. 813-818.
69. S. Hu, Y. Liang, L. Ma, Y. He, MSMOTE: improving classification performance when training data is imbalanced, in: Proceedings of the 2nd International Workshop on Computer Science and Engineering (WCSE'09), vol. 2, 2009, pp. 13-17.
70. J. Huang, and C.X. Ling Using AUC and accuracy in evaluating learning algorithms IEEE Transactions on Knowledge and Data Engineering 17 3 2005 299 310
71. N. Japkowicz Concept-learning in the presence of between-class and within-class imbalances E. Stroulia, S. Matwin, Proceedings of the 14th Canadian Conference on Advances in Artificial Intelligence (CCAI'08) Lecture Notes in Computer Science vol. 2056 2001 Springer 67 77
72. N. Japkowicz, and S. Stephen The class imbalance problem: a systematic study Intelligent Data Analysis Journal 6 5 2002 429 450
73. T. Jo, and N. Japkowicz Class imbalances versus small disjuncts ACM SIGKDD Explorations Newsletter 6 1 2004 40 49
74. M.V. Joshi, V. Kumar, and R.C. Agarwal Evaluating boosting algorithms to classify rare classes: comparison and improvements Proceedings of the 2001 IEEE International Conference on Data Mining (ICDM'01) 2001 IEEE Computer Society Washington, DC, USA 257 264
75. T.M. Khoshgoftaar, J. Van Hulse, and A. Napolitano Comparing boosting and bagging techniques with noisy and imbalanced data IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans 41 3 2011 552 568
76. M. Kubat, S. Matwin, Addressing the curse of imbalanced training sets: one-sided selection, in: Proceedings of the 14th International Conference on Machine Learning (ICML'97), 1997, pp. 179-186.
77. L.I. Kuncheva, and J.J. Rodrguez A weighted voting framework for classifiers ensembles Knowledge and Information Systems 2013 10.1007/s10115-012-0586-6 in press
78. N. Kwak Feature extraction for classification problems and its application to face recognition Pattern Recognition 41 5 2008 1718 1734
79. J. Laurikkala, Improving identification of difficult small classes by balancing class distribution, in: Proceedings of the 8th Conference on AI in Medicine in Europe: Artificial Intelligence Medicine (AIME'01), 2001, pp. 63-66.
80. C. Li Classifying imbalanced data using a bagging ensemble variation (BEV) Proceedings of the 45th Annual Southeast Regional Conference vol. 45 2007 ACM-SE ACM New York, NY, USA 203 208
81. M. Lin, K. Tang, and X. Yao Dynamic sampling approach to training neural networks for multiclass imbalance classification IEEE Transactions on Neural Networks and Learning Systems 24 4 2013 647 660
82. W. Lin, and J.J. Chen Class-imbalanced classifiers for high-dimensional data Briefings in Bioinformatics 14 1 2013 13 26
83. C.X. Ling, C. Li, Data mining for direct marketing: Problems and solutions, in: Proceedings of the 4th International Conference on Knownledge Discovery and Data Mining (KDD'98), 1998, pp. 73-79.
84. C.X. Ling, Q. Yang, J. Wang, and S. Zhang Decision trees with minimal costs C.E. Brodley, Proceedings of the 21st International Conference on Machine Learning (ICML'04) ACM International Conference Proceeding Series vol. 69 2004 ACM 69 77
85. X.-Y. Liu, J. Wu, and Z.-H. Zhou Exploratory undersampling for class-imbalance learning IEEE Transactions on System, Man and Cybernetics B 39 2 2009 539 550
86. H.-Y. Lo, C.-M. Chang, T.-H. Chiang, C.-Y. Hsiao, A. Huang, T.-T. Kuo, W.-C. Lai, M.-H. Yang, J.-J. Yeh, C.-C. Yen, and S.-D. Lin Learning to improve area-under-FROC for imbalanced medical data classification using an ensemble method SIGKDD Explorations 10 2 2008 43 46
87. V. López, A. Fernández, M.J. del Jesus, and F. Herrera A hierarchical genetic fuzzy system based on genetic programming for addressing classification with highly imbalanced and borderline data-sets Knowledge-Based Systems 38 2013 85 104
88. V. López, A. Fernández, J.G. Moreno-Torres, and F. Herrera Analysis of preprocessing vs. cost-sensitive learning for imbalanced classification. open problems on intrinsic data characteristics Expert Systems with Applications 39 7 2012 6585 6608
89. J. Luengo, A. Fernández, S. García, and F. Herrera Addressing data complexity for imbalanced data sets: analysis of SMOTE-based oversampling and evolutionary undersampling Soft Computing 15 10 2011 1909 1936
90. R. Martín-Félez, R.A., Mollineda, On the suitability of combining feature selection and resampling to manage data complexity, in: Proceedings of the Conferencia de la Asociacin Espaola de Inteligencia Artificial (CAEPIA'09), Lecture Notes on Artificial Intelligence, vol. 5988, 2010, pp. 141-150.
91. M.A. Mazurowski, P.A. Habas, J.M. Zurada, J.Y. Lo, J.A. Baker, and G.D. Tourassi Training neural network classifiers for medical decision making: the effects of imbalanced datasets on classification performance Neural Networks 21 2-3 2008
92. G.J. McLachlan Discriminant Analysis and Statistical Pattern Recognition 2004 John Wiley and Sons
93. L. Mena, and J.A. González Symbolic one-class learning from imbalanced datasets: application in medical diagnosis International Journal on Artificial Intelligence Tools 18 2 2009 273 309
94. J.G. Moreno-Torres, F. Herrera, A preliminary study on overlapping and data fracture in imbalanced domains by means of genetic programming-based feature extraction, in: Proceedings of the 10th International Conference on Intelligent Systems Design and Applications (ISDA'10), 2010, pp. 501-506.
95. J.G. Moreno-Torres, X. Llorà, D.E. Goldberg, and R. Bhargava Repairing fractures between data using genetic programming-based feature extraction: a case study in cancer diagnosis Information Sciences 222 2013 805 823
96. J.G. Moreno-Torres, T. Raeder, R. Aláiz-Rodríguez, N.V. Chawla, and F. Herrera A unifying view on dataset shift in classification Pattern Recognition 45 1 2012 521 530
97. K. Napierala, J. Stefanowski, S. Wilk, Learning from imbalanced data in presence of noisy and borderline examples, in: Proceedings of the 7th International Conference on Rough Sets and Current Trends in Computing (RSCTC'10), Lecture Notes on Artificial Intelligence, vol. 6086, 2010, pp. 158-167.
98. A. Orriols-Puig, and E. Bernadó-Mansilla Evolutionary rule-based systems for imbalanced datasets Soft Computing 13 3 2009 213 225
99. A. Orriols-Puig, E. Bernadó-Mansilla, D.E. Goldberg, K. Sastry, and P.L. Lanzi Facetwise analysis of XCS for problems with class imbalances IEEE Transactions on Evolutionary Computation 13 2009 260 283
100. J. Platt Fast training of support vector machines using sequential minimal optimization Advances in Kernel Methods - Support Vector Learning 1998 MIT Press Cambridge, MA 42 65
101. R. Polikar Ensemble based systems in decision making IEEE Circuits and Systems Magazine 6 3 2006 21 45
102. R.C. Prati, G.E.A.P.A., Batista, Class imbalances versus class overlapping: an analysis of a learning system behavior, in: Proceedings of the 2004 Mexican International Conference on Artificial Intelligence (MICAI'04), 2004, pp. 312-321.
103. R.C. Prati, G.E.A.P.A. Batista, and M.C. Monard A survey on graphical methods for classification predictive performance evaluation IEEE Transactions on Knowledge and Data Engineering 23 11 2011 1601 1618
104. J.R. Quinlan C4.5: Programs for Machine Learning. 1993 Morgan Kauffman
105. T. Raeder, G. Forman, and N.V. Chawla Learning from imbalanced data: evaluation matters D.E. Holmes, L.C. Jain, Data Mining: Found. and Intell. Paradigms vol. ISRL 23 2012 Springer-Verlag 315 331
106. S.J. Raudys, and A.K. Jain Small sample size effects in statistical pattern recognition: recommendations for practitioners IEEE Transactions on Pattern Analysis and Machine Intelligence 13 3 1991 252 264
107. P. Riddle, R. Segal, and O. Etzioni Representation design and brute-force induction in a boeing manufacturing domain Applied Artificial Intelligence 8 1994 125 147
108. L. Rokach Ensemble-based classifiers Artificial Intelligence Review 33 1 2010 1 39
109. J.A. Sáez, J. Luengo, and F. Herrera A first study on the noise impact in classes for fuzzy rule based classification systems Proceedings of the 2010 IEEE International Conference on Intelligent Systems and Knowledge Engineering (ISKE'10) 2010 IEEE Press 153 158
110. R.E. Schapire, A brief introduction to boosting, in: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI'99), 1999, pp. 1401-1406.
111. C. Seiffert, T.M. Khoshgoftaar, J. Van Hulse, and A. Folleco An empirical study of the classification performance of learners on imbalanced and noisy software quality data Information Sciences 2013 10.1016/j.ins.2010.12.016 in press
112. C. Seiffert, T.M. Khoshgoftaar, J. Van Hulse, and A. Napolitano RUSBoost: a hybrid approach to alleviating class imbalance IEEE Transactions on System, Man and Cybernetics A 40 1 2010 185 197
113. J.P. Shaffer Modified sequentially rejective multiple test procedures Journal of the American Statistical Association 81 395 1986 826 831
114. H. Shimodaira Improving predictive inference under covariate shift by weighting the log-likelihood function Journal of Statistical Planning and Inference 90 2 2000 227 244
115. J. Stefanowski, S. Wilk, Improving rule based classifiers induced by MODLEM by selective pre-processing of imbalanced data, in: Proceedings of the RSKD Workshop at ECML/PKDD'07, 2007, pp. 54-65.
116. J. Stefanowski, S. Wilk, Selective pre-processing of imbalanced data for improving classification performance, in: Proceedings of the 10th International Conference on Data Warehousing and Knowledge, Discovery (DaWaK08), 2008, pp. 283-292.
117. Y. Sun, M.S. Kamel, A.K.C. Wong, and Y. Wang Cost-sensitive boosting for classification of imbalanced data Pattern Recognition 40 12 2007 3358 3378
118. Y. Sun, A.K.C. Wong, and M.S. Kamel Classification of imbalanced data: a review International Journal of Pattern Recognition and Artificial Intelligence 23 4 2009 687 719
119. Y. Tang, Y.-Q. Zhang, N.V. Chawla, and S. Kresser SVMs modeling for highly imbalanced classification IEEE Transactions on Systems, Man and Cybernetics, Part B 9 1 2009 281 288
120. D. Tao, X. Tang, X. Li, and X. Wu Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval IEEE Transactions on Pattern Analysis and Machine Intelligence 28 7 2006 1088 1099
121. K.M. Ting, The problem of small disjuncts: its remedy in decision trees, in: Proceedings of the 10th Canadian Conference on Artificial Intelligence (CCAI'94), 1994, pp. 91-97.
122. K.M. Ting, A comparative study of cost-sensitive boosting algorithms, in: Proceedings of the 17th International Conference on Machine Learning (ICML'00), Stanford, CA, USA, 2000, pp. 983-990.
123. K.M. Ting An instance-weighting method to induce cost-sensitive trees IEEE Transactions on Knowledge and Data Engineering 14 3 2002 659 665
124. I. Tomek Two modifications of CNN IEEE Transactions on Systems Man and Communications 6 1976 769 772
125. C.-H. Tsai, L.-C. Chang, and H.-C. Chiang Forecasting of ozone episode days by cost-sensitive neural network methods Science of the Total Environment 407 6 2009 2124 2135
126. P.D. Turney Cost-sensitive classification: empirical evaluation of a hybrid genetic decision tree induction algorithm Journal of Artificial Intelligence Research 2 1995 369 409
127. J. Van Hulse, T.M. Khoshgoftaar, A. Napolitano, An empirical comparison of repetitive undersampling techniques, in: Proceedings of the 2009 IEEE International Conference on Information Reuse, Integration (IRI'09), 2009, pp. 29-34.
128. B.X. Wang, N. Japkowicz, Imbalanced data set learning with synthetic samples, in: Proceedings of the IRIS Machine Learning Workshop, 2004.
129. J. Wang, J. You, Q. Li, and Y. Xu Extract minimum positive and maximum negative features for imbalanced binary classification Pattern Recognition 45 3 2012 1136 1145
130. S. Wang, X. Yao, Diversity analysis on imbalanced data sets by using ensemble models, in: Proceedings of the 2009 IEEE Symposium on Computational Intelligence and Data Mining (CIDM'09), 2009, pp. 324-331.
131. S. Wang, and X. Yao Relationships between diversity of classification ensembles and single-class performance measures IEEE Transactions on Knowledge and Data Engineering 25 1 2013 206 219
132. Z. Wang, and V. Palade Building interpretable fuzzy models for high dimensional data analysis in cancer diagnosis BMC Genomics 12 (S2):S5 2011
133. M. Wasikowski, and X.-W. Chen Combating the small sample class imbalance problem using feature selection IEEE Transactions on Knowledge and Data Engineering 22 10 2010 1388 1400
134. G.M. Weiss Timeweaver: a genetic algorithm for identifying pre-dictive patterns in sequences of events W. Banzhaf, J. Daida, A.E. Eiben, M.H. Garzon, V. Honavar, M. Jakiela, R.E. Smith, Proceedings of the Genetic and Evolutionary Computation Conference GECCO'99 vol. 1 1999 Morgan Kaufmann Orlando, Florida, USA 718 725
135. G.M. Weiss Mining with rarity: a unifying framework SIGKDD Explorations 6 1 2004 7 19
136. G.M. Weiss Mining with rare cases O. Maimon, L. Rokach, The Data Mining and Knowledge Discovery Handbook 2005 Springer 765 776
137. G.M. Weiss The impact of small disjuncts on classifier learning R. Stahlbock, S.F. Crone, S. Lessmann, Data Mining: Annals of Information Systems vol. 8 2010 Springer 193 226
138. G.M. Weiss, and F.J. Provost Learning when training data are costly: the effect of class distribution on tree induction Journal of Artificial Intelligence Research 19 2003 315 354
139. G.M. Weiss, and Y. Tian Maximizing classifier utility when there are data acquisition and modeling costs Data Mining and Knowledge Discovery 17 2 2008 253 282
140. D.L. Wilson Asymptotic properties of nearest neighbor rules using edited data IEEE Transactions on Systems, Man and Cybernetics 2 3 1972 408 421
141. R. Yan, Y. Liu, R. Jin, A. Hauptmann, On predicting rare classes with SVM ensembles in scene classification, in: Proceedings of the 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP'03), vol. 3, 2003, pp. 21-24.
142. P. Yang, L. Xu, B.B. Zhou, Z. Zhang, and A.Y. Zomaya A particle swarm based hybrid system for imbalanced medical data sampling BMC Genomics 10 Suppl. 3 2009 art. no. S34.
143. Q. Yang, and X. Wu 10 challenging problems in data mining research International Journal of Information Technology and Decision Making 5 4 2006 597 604
144. Y. Yang, X. Wu, and X. Zhu Conceptual equivalence for contrast mining in classification learning Data & Knowledge Engineering 67 3 2008 413 429
145. S. Yen, Y. Lee, Under-sampling approaches for improving prediction of the minority class in an imbalanced dataset, in: Proceedings of the 2006 International Conference on Intelligent, Computing (ICIC06), 2006, pp. 731-740.
146. K. Yoon, S. Kwek, An unsupervised learning approach to resolving the data imbalanced issue in supervised learning problems in functional genomics, in: Proceedings of the 5th International Conference on Hybrid Intelligent Systems (HIS'05), 2005, pp. 303-308.
147. B. Zadrozny, C. Elkan, Learning and making decisions when costs and probabilities are both unknown, in: Proceedings of the 7th International Conference on Knowledge Discovery and Data Mining (KDD'01), 2001, pp. 204-213.
148. B. Zadrozny, J. Langford, N. Abe, Cost-sensitive learning by cost-proportionate example weighting, in: Proceedings of the 3rd IEEE International Conference on Data Mining (ICDM'03), 2003, pp. 435-442.
149. J. Zhang, I. Mani, KNN approach to unbalanced data distributions: a case study involving information extraction, in: Proceedings of the 20th International Conference on Machine Learning (ICML'03), Workshop Learning from Imbalanced Data Sets, 2003.
150. Z.-H. Zhou, and X.-Y. Liu Training cost-sensitive neural networks with methods addressing the class imbalance problem IEEE Transactions on Knowledge and Data Engineering 18 1 2006 63 77
151. X. Zhu, and X. Wu Class noise vs. attribute noise: a quantitative study Artificial Intelligence Review 22 3 2004 177 210
152. W. Zong, G.-B. Huang, and Y. Chen Weighted extreme learning machine for imbalance learning Neurocomputing 101 2013 229 242