Improving imbalanced learning through a heuristic oversampling method based on k-means and SMOTE

Information Sciences

Volumn 465, Issue , 2018, Pages 1-20

  Douzas, Georgios ^a   Bacao, Fernando ^a   Last, Felix ^a  

^a NOVA INFORMATION MANAGEMENT SCHOOL NOVA IMS   (Portugal)

Author keywords

Class imbalanced learning; Classification; Clustering; Oversampling; Supervised learning; Within class imbalance

Indexed keywords

CLASSIFICATION (OF INFORMATION); PROBLEM ORIENTED LANGUAGES; SUPERVISED LEARNING;

CLASSIFICATION ALGORITHM; CLASSIFICATION RESULTS; CLUSTERING; IMBALANCED LEARNING; OVER SAMPLING; PYTHON PROGRAMMING LANGUAGE; SYNTHETIC MINORITY OVER-SAMPLING TECHNIQUES; WITHIN CLASS;

HEURISTIC METHODS;

EID: 85049450664     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2018.06.056     Document Type: Article

References (49)

1. Alcalá-Fdez, J., Fernández, A., Luengo, J., Derrac, J., García, S., Sánchez, L., Herrera, F., Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J. Mult. Valued Logic Soft Comput., 17, 2011.
2. Batista, G.E.A.P.A., Prati, R.C., Monard, M.C., A study of the behavior of several methods for balancing machine learning training data. ACM SIGKDD Explor. Newslett. 6:1 (2004), 20–29, 10.1145/1007730.1007735.
3. Bunkhumpornpat, C., Sinapiromsaran, K., Lursinsap, C., Safe-level-smote: Safe-level-synthetic minority over-sampling technique for handling the class imbalanced problem. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5476 LNAI, 2009, 475–482, 10.1007/978-3-642-01307-2_43.
4. Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P., Smote: synthetic minority over-sampling technique. J. Arti. Intell. Res. 16 (2002), 321–357, 10.1613/jair.953.
5. Chawla, N.V., Cieslak, D.A., Hall, L.O., Joshi, A., Automatically countering imbalance and its empirical relationship to cost. Data Min. Knowl. Discov. 17:2 (2008), 225–252, 10.1007/s10618-008-0087-0.
6. Chawla, N.V., Japkowicz, N., Drive, P., Editorial: special issue on learning from imbalanced data sets. ACM SIGKDD Explor. Newslett. 6:1 (2004), 1–6, 10.1145/1007730.1007733.
7. Cieslak, D.A., Chawla, N.V., Striegel, A., Combating imbalance in network intrusion datasets. Proceedings of the IEEE International Conference on Granular Computing, 2006, IEEE, 732–737, 10.1109/GRC.2006.1635905.
8. Cieslak, D.A., Hoens, T.R., Chawla, N.V., Kegelmeyer, W.P., Hellinger distance decision trees are robust and skew-insensitive. Data Min. Knowl. Discov. 24:1 (2012), 136–158, 10.1007/s10618-011-0222-1.
9. Demšar, J., Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7 (2006), 1–30.
10. Domingos, P., Metacost: a general method for making classifiers cost-sensitive. Proceedings of the Fifth International Conference on Knowledge Discovery and Data Mining, 1999, 155–164, 10.1145/312129.312220.
11. Douzas, G., Bacao, F., Self-organizing map oversampling (SOMO) for imbalanced data set learning. Expert Syst. Appl. 82 (2017), 40–52, 10.1016/j.eswa.2017.03.073.
12. Díez-Pastor, J.F., Rodrí-guez, J.J., García-Osorio, C.I., Kuncheva, L.I., Diversity techniques improve the performance of the best imbalance learning ensembles. Inf. Sci. (Ny) 325 (2015), 98–117, 10.1016/j.ins.2015.07.025.
13. Estabrooks, A., Jo, T., Japkowicz, N., A multiple resampling method for learning from imbalanced data sets. Comput. Intell. 20:1 (2004), 18–36, 10.1111/j.0824-7935.2004.t01-1-00228.x.
14. Fernández, A., López, V., Galar, M., Del Jesus, M.J., Herrera, F., Analysing the classification of imbalanced data-sets with multiple classes: binarization techniques and ad-hoc approaches. Knowl. Based Syst. 42 (2013), 97–110, 10.1016/j.knosys.2013.01.018.
15. E. Fix, J.L. Hodges Jr., Discriminatory analysis - nonparametric discrimination: Consistency properties. California Univ Berkeley, 1951.
16. Friedman, J.H., Greedy function approximation: a gradient boosting machine. Ann. Stat. 29:5 (2001), 1189–1232, 10.1214/aos/1013203451.
17. Friedman, M., The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J. Am. Stat. Assoc., 32(200), 1937, 675, 10.2307/2279372.
18. M. Galar, A. Fernandez, E. Barrenechea, H. Bustince, F. Herrera, A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches, 2012. doi: 10.1109/TSMCC.2011.2161285.
19. Galar, M., Fernandez, A., Barrenechea, E., Bustince, H., Herrera, F., Ordering-based pruning for improving the performance of ensembles of classifiers in the framework of imbalanced datasets. Inf. Sci. (Ny) 354 (2016), 178–196, 10.1016/j.ins.2016.02.056.
20. I. Guyon, Design of experiments of the NIPS 2003 variable selection benchmark, 2003.
21. Han, H., Wang, W.-Y., Mao, B.-H., Borderline-smote: a new over-sampling method in imbalanced data sets learning. Adv. Intell. Comput. 17:12 (2005), 878–887, 10.1007/11538059_91.
22. Hawkins, D.M., The problem of overfitting. J. Chem. Inf. Comput. Sci. 44:1 (2004), 1–12, 10.1002/chin.200419274.
23. He, H., Garcia, E.A., Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 21:9 (2009), 1263–1284, 10.1109/TKDE.2008.239.
24. Holm, S., A simple sequentially rejective multiple test procedure. Scand. J. Stat. 6:2 (1979), 65–70.
25. Holte, R.C., Acker, L., Porter, B.W., et al. Concept learning and the problem of small disjuncts. Proceedings of the IJCAI, vol. 89, 1989, 813–818.
26. Japkowicz, N., Assessment metrics for imbalanced learning. He, H., Ma, Y., (eds.) Imbalanced Learning, 2013, John Wiley & Sons, 187–206, 10.1002/9781118646106.ch8.
27. Jo, T., Japkowicz, N., Class imbalances versus small disjuncts. ACM SIGKDD Explor. Newslett. 6:1 (2004), 40–49.
28. Kotsiantis, S., Kanellopoulos, D., Pintelas, P., Handling imbalanced datasets: a review. Science 30:1 (2006), 25–36, 10.1007/978-0-387-09823-4_45.
29. S. Kotsiantis, P. Pintelas, D. Anyfantis, M. Karagiannopoulos, Robustness of learning techniques in handling class noise in imbalanced datasets, 2007, doi: 10.1007/978-0-387-74161-1_3.
30. Lemaître, G., Nogueira, F., Aridas, C.K., Imbalanced-learn: a python toolbox to tackle the curse of imbalanced datasets in machine learning. J. Mach. Learn. Res. 18:17 (2017), 1–5.
31. M. Lichman, Uci machine learning repository, 2013.
32. Lin, W.-C., Tsai, C.-F., Hu, Y.-H., Jhang, J.-S., Clustering-based undersampling in class-imbalanced data. Inf. Sci. (Ny) 409–410 (2017), 17–26, 10.1016/j.ins.2017.05.008.
33. Ma, L., Fan, S., Cure-smote algorithm and hybrid algorithm for feature selection and parameter optimization based on random forests. BMC Bioinf., 18(1), 2017, 169, 10.1186/s12859-017-1578-z.
34. MacQueen, J., Some methods for classification and analysis of multivariate observations. Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, 1967, 281–297.
35. McCullagh, P., Generalized linear models. Eur. J. Oper. Res. 16:3 (1984), 285–292, 10.1016/0377-2217(84)90282-0.
36. Nekooeimehr, I., Lai-Yuen, S.K., Adaptive semi-unsupervised weighted oversampling (A-SUWO) for imbalanced datasets. Expert Syst. Appl. 46 (2016), 405–416, 10.1016/j.eswa.2015.10.031.
37. Nickerson, A., Japkowicz, N., Milios, E.E., Using unsupervised learning to guide resampling in imbalanced data sets. Proceedings of the AISTATS, 2001, 261–265.
38. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E., Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12 (2011), 2825–2830.
39. Prati, R.C., Batista, G., Monard, M.C., Learning with class skews and small disjuncts. Proceedings of the SBIA, 2004, 296–306, 10.1007/978-3-540-28645-5_30.
40. Provost, F., Machine Learning from Imbalanced Data Sets 101. Proceedings of the AAAI ’2000 Workshop on Imbalanced Data Sets, vol. 68, 2000, AAAI Press, 1–3.
41. Rivera, W.A., Noise reduction a priori synthetic over-sampling for class imbalanced data sets. Inf. Sci. (Ny) 408 (2017), 146–161, 10.1016/j.ins.2017.04.046.
42. Santos, M.S., Abreu, P.H., García-Laencina, P.J., Simão, A., Carvalho, A., A new cluster-based oversampling method for improving survival prediction of hepatocellular carcinoma patients. J. Biomed. Inf. 58 (2015), 49–59, 10.1016/j.jbi.2015.09.012.
43. Sculley, D., Web-scale k-means clustering. Proceedings of the Nineteenth International Conference on World Wide Web, 2010, ACM, 1772690, 1177–1178, 10.1145/1772690.1772862.
44. Song, J., Huang, X., Qin, S., Song, Q., A bi-directional sampling based on k-means method for imbalance text classification. Proceedings of the International Conference on Computer and Information Science (ICIS), 2016, 1–5, 10.1109/icis.2016.7550920.
45. Sáez, J.A., Luengo, J., Stefanowski, J., Herrera, F., SMOTE-IPF: addressing the noisy and borderline examples problem in imbalanced classification by a re-sampling method with filtering. Inf. Sci. (Ny) 291 (2015), 184–203, 10.1016/j.ins.2014.08.051.
46. Vanhoeyveld, J., Martens, D., Imbalanced classification in sparse and large behaviour datasets. Data Min. Knowl. Discov., 2017, 1–58, 10.1007/s10618-017-0517-y.
47. Weiss, G.M., McCarthy, K., Zabar, B., Cost-sensitive learning vs. sampling: which is best for handling unbalanced classes with unequal error costs?. DMIN 7 (2007), 35–41.
48. Wu, J., Xiong, H., Chen, J., COG: local decomposition for rare class analysis. Data Min. Knowl. Discov. 20:2 (2010), 191–220, 10.1007/s10618-009-0146-1.
49. Zhu, B., Baesens, B., vanden Broucke, S.K.L.M., An empirical comparison of techniques for the class imbalance problem in churn prediction. Inf. Sci. (Ny) 408 (2017), 84–99, 10.1016/j.ins.2017.04.015.