Near-Bayesian Support Vector Machines for imbalanced data classification with equal or unequal misclassification costs

Neural Networks

Volumn 70, Issue , 2015, Pages 39-52

  Datta, Shounak ^a   Das, Swagatam ^a  

^a INDIAN STATISTICAL INSTITUTE   (India)

Author keywords

Bayes error; Decision boundary shift; Imbalanced data; Multi class classification; Support vector machines; Unequal costs

Indexed keywords

CLASSIFICATION (OF INFORMATION); COSTS; OPTIMIZATION; PHILOSOPHICAL ASPECTS; VECTORS;

BAYES ERROR; COMPUTATIONALLY EFFICIENT; DECISION BOUNDARY; IMBALANCED DATA; MULTI-CLASS CLASSIFICATION; SEQUENTIAL MINIMAL OPTIMIZATION ALGORITHMS; SUPPORT VECTOR MACHINE (SVMS); TWO-CLASS CLASSIFICATION PROBLEMS;

SUPPORT VECTOR MACHINES;

ALGORITHM; ARTICLE; BAYES THEOREM; INFORMATION PROCESSING; MATHEMATICAL COMPUTING; MATHEMATICAL VARIABLE; MODIFIED SEQUENTIAL MINIMAL OPTIMIZATION ALGORITHM; NEAR BAYESIAN SUPPORT VECTOR MACHINE; PRIORITY JOURNAL; STATISTICAL PARAMETERS; SUPPORT VECTOR MACHINE; ARTIFACT; STATISTICAL ANALYSIS;

ALGORITHMS; ARTIFACTS; BAYES THEOREM; DATA INTERPRETATION, STATISTICAL; DATASETS AS TOPIC; SUPPORT VECTOR MACHINE;

EID: 84937927040     PISSN: 08936080     EISSN: 18792782     Source Type: Journal    
DOI: 10.1016/j.neunet.2015.06.005     Document Type: Article

References (52)

1. Abe S. Analysis of support vector machines. Proceedings of the 2002 12th IEEE workshop on neural networks for signal processing, 2002 2002, 89-98. IEEE. 10.1109/NNSP.2002.1030020.
2. Akbani R., Kwek S., Japkowicz N. Applying support vector machines to imbalanced datasets. Lecture notes in computer science 2004, Vol. 3201:39-50. Springer. 10.1007/978-3-540-30115-8_7.
3. Amari S.-i., Wu S. Improving support vector machine classifiers by modifying kernel functions. Neural Networks 1999, 12:783-789. 10.1016/S0893-6080(99)00032-5.
4. Bao-Liang L., Xiao-Lin W., Yang Y., Hai Z. Learning from imbalanced datasets with a min-max modular support vector machine. Frontiers of Electrical and Electronic Engineering in China 2011, 6:56-71. 10.1007/s11460-011-0127-1.
5. Batuwita R., Palade V. Efficient resampling methods for training support vector machines with imbalanced datasets. The 2010 international joint conference on neural networks (IJCNN) 2010, 1-8. IEEE. 10.1109/IJCNN.2010.5596787.
6. Batuwita R., Palade V. Class imbalance learning methods for support vector machines. Imbalanced learning: foundations, algorithms, and applications 2013, 83-99. John Wiley & Sons, Inc. 10.1002/9781118646106.ch5.
7. Brazdil, P., & Gama, J. (1991). Statlog repository. URL: [2007-10-22]. http://www.liacc%2Cup.pt/ML/statlog/datasets.html.
8. Burges C.J. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 1998, 2:121-167.
9. Cao P., Zhao D., Zaiane O. An optimized cost-sensitive SVM for imbalanced data learning. Lecture notes in computer science 2013, Vol. 7819:280-292. Springer. 10.1007/978-3-642-37456-2_24.
10. Cervantes J., Li X., Yu W. Imbalanced data classification via support vector machines and genetic algorithms. Connection Science 2014, 26:335-348. 10.1080/09540091.2014.924902.
11. Chang C.-C., Lin C.-J. LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST) 2011, 2:27:1-27:27.
12. Chawla N.V., Bowyer K.W., Hall L.O., Kegelmeyer W.P. SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research 2002, 16:321-357. 10.1613/jair.953.
13. Choi, J.M. (2010). A selective sampling method for imbalanced data learning on support vector machines (Ph.D. thesis). Ames, IA, USA.
14. Cortes C., Vapnik V. Support-vector networks. Machine Learning 1995, 20:273-297. 10.1023/A:1022627411411.
15. Duan W., Jing L., Lu X.Y. Imbalanced data classification using cost-sensitive support vector machine based on information entropy. Advanced Materials Research 2014, 989:1756-1761. 10.4028/www.scientific.net/AMR.989-994.1756.
16. Ertekin S. Learning in extreme conditions: Online and active learning with massive, imbalanced and noisy data 2009, (Ph.D. thesis), University Park, PA, USA.
17. Fernández A., García S., Herrera F. Addressing the classification with imbalanced data: open problems and new challenges on class distribution. Lecture notes in computer science 2011, Vol. 6678:1-10. Springer. 10.1007/978-3-642-21219-2_1.
18. Galar M., Fernandez A., Barrenechea E., Bustince H., Herrera F. A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Transactions on Systems, Man, and Cybernetics Part C: Applications and Reviews 2012, 42:463-484. 10.1109/TSMCC.2011.2161285.
19. García S., Fernández A., Luengo J., Herrera F. Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power. Information Sciences 2010, 180:2044-2064. Special Issue on Intelligent Distributed Information Systems. 10.1016/j.ins.2009.12.010.
20. Gonen M., Tanugur A.G., Alpaydin E. Multiclass posterior probability support vector machines. IEEE Transactions on Neural Networks 2008, 19:130-139. 10.1109/TNN.2007.903157.
21. He H., Garcia E.A. Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering 2009, 21:1263-1284. 10.1109/TKDE.2008.239.
22. Imam T., Ting K.M., Kamruzzaman J. z-SVM: An SVM for improved classification of imbalanced data. Lecture notes in computer science 2006, Vol.~4304:264-273. Springer. 10.1007/11941439_30.
23. Keerthi S.S., Shevade S.K., Bhattacharyya C., Murthy K.R.K. Improvements to Platt's SMO algorithm for SVM classifier design. Neural Computation 2001, 13:637-649. 10.1162/089976601300014493.
24. Koknar-Tezel S., Latecki L.J. Improving SVM classification on imbalanced data sets in distance spaces. Ninth IEEE international conference on data mining, 2009. ICDM'09 2009, 259-267. IEEE. 10.1109/ICDM.2009.59.
25. Kubat M., Matwin S., et al. Addressing the curse of imbalanced training sets: one-sided selection. Proceedings of the fourteenth international conference on machine learning 1997, 179-186. Morgan Kaufmann, Nashville, USA.
26. Lee J., Wu Y., Kim H. Unbalanced data classification using support vector machines with active learning on scleroderma lung disease patterns. Journal of Applied Statistics 2014, 42:676-689. 10.1080/02664763.2014.978270.
27. Lichman, M. (2013). UCI machine learning repository. URL: . http://archive.ics.uci.edu/ml.
28. Lin Y., Lee Y., Wahba G. Support vector machines for classification in nonstandard situations. Machine Learning 2002, 46:191-202. 10.1023/A:1012406528296.
29. Maratea A., Petrosino A., Manzo M. Adjusted f-measure and kernel scaling for imbalanced data learning. Information Sciences 2014, 257:331-341. 10.1016/j.ins.2013.04.016.
30. Masnadi-Shirazi H., Vasconcelos N. Risk minimization, probability elicitation, and cost-sensitive SVMs. Proceedings of the 27th international conference on machine learning 2010, 759-766. Omnipress.
31. Peng L., Ting-ting B., Yang L. SVM classification for high-dimensional imbalanced data based on SNR and under-sampling. International Journal of Multimedia and Ubiquitous Engineering 2015, 10:105-112. 10.14257/ijmue.2015.10.4.11.
32. Peng L., Xiao-yang Y., Ting-ting B., Jiu-ling H. Imbalanced data SVM classification method based on cluster boundary sampling and DT-KNN pruning. International Journal of Signal Processing, Image Processing and Pattern Recognition 2014, 7:61-68.
33. Phoungphol P. A classification framework for imbalanced data 2013, (Ph.D. thesis).
34. Platt J. Fast training of support vector machines using sequential minimal optimization. Advances in kernel methods Support Vector Learning. Vol. 3 1999.
35. Raskutti B., Kowalczyk A. Extreme re-balancing for SVMs: A case study. SIGKDD Explorations Newsletter 2004, 6:60-69. 10.1145/1007730.1007739.
36. Rätsch, G. (2001). IDA benchmark repository. URL: . http://ida.first.fhg.de/projects/bench/benchmarks.htm.
37. Stecking R., Schebesch K.B. Classification of large imbalanced credit client data with cluster based SVM. Studies in classification, data analysis, and knowledge organization 2012, 443-451. Springer. 10.1007/978-3-642-24466-7_45.
38. Tang Y., Zhang Y.-Q., Chawla N.V., Krasser S. SVMs modeling for highly imbalanced classification. IEEE Transactions on Systems, Man and Cybernetics, Part B: Cybernetics 2009, 39:281-288. 10.1109/TSMCB.2008.2002909.
39. Tao Q., Wu G.-W., Wang F.-Y., Wang J. Posterior probability support vector machines for unbalanced data. IEEE Transactions on Neural Networks 2005, 16:1561-1573. 10.1109/TNN.2005.857955.
40. Veropoulos, K., Campbell, C., & Cristianini, N. et al. (1999). Controlling the sensitivity of support vector machines. In Proceedings of the international joint conference on AI, IJCAI (pp. 55-60).
41. Wang Q. A hybrid sampling SVM approach to imbalanced data classification. Abstract and applied analysis 2014, Hindawi Publishing Corporation. 10.1155/2014/972786.
42. Wang B.X., Japkowicz N. Boosting support vector machines for imbalanced datasets. Knowledge and Information Systems 2010, 25:1-20. 10.1007/s10115-009-0198-y.
43. Wang S., Li Z., Chao W., Cao Q. Applying adaptive over-sampling technique based on data density and cost-sensitive SVM to imbalanced learning. The 2012 international joint conference on neural networks (IJCNN) 2012, 1-8. IEEE. 10.1109/IJCNN.2012.6252696.
44. Wilcoxon F. Individual comparisons by ranking methods. Springer series in statistics 1992, 196-202. Springer. 10.1007/978-1-4612-4380-9_16.
45. Wu, G., & Chang, E.Y. (2003). Adaptive feature-space conformal transformation for imbalanced-data learning. In Proceedings of the twentieth international conference on machine learning (pp. 816-823).
46. Wu G., Chang E.Y. KBA: Kernel boundary alignment considering imbalanced data distribution. IEEE Transactions on Knowledge and Data Engineering 2005, 17:786-795. 10.1109/TKDE.2005.95.
47. Wu S.-H., Lin K.-P., Chien H.-H., Chen C.-M., Chen M.-S. On generalizable low false-positive learning using asymmetric support vector machines. IEEE Transactions on Knowledge and Data Engineering 2013, 25:1083-1096. 10.1109/TKDE.2012.46.
48. Yang C.-Y., Yang J.-S., Wang J.-J. Margin calibration in SVM class-imbalanced learning. Neurocomputing 2009, 73:397-411. 10.1016/j.neucom.2009.08.006.
49. Yu J., Cheng F., Xiong H., Qu W., Chen X.-w. A Bayesian approach to support vector machines for the binary classification. Neurocomputing 2008, 72:177-185. 10.1016/j.neucom.2008.06.010.
50. Zhang Y., Fu P., Liu W., Chen G. Imbalanced data classification based on scaling kernel-based support vector machine. Neural Computing and Applications 2014, 25:927-935. 10.1007/s00521-014-1584-2.
51. Zhao Z., Zhong P., Zhao Y. Learning SVM with weighted maximum margin criterion for classification of imbalanced data. Mathematical and Computer Modelling 2011, 54:1093-1099. 10.1016/j.mcm.2010.11.040.
52. Zughrat, A., Mahfouf, M., Yang, Y., & Thornton, S. (2014). Support vector machines for class imbalance rail data classification with bootstrapping-based over-sampling and under-sampling. In 19th world congress of the international federation of automatic control (pp. 8756-8761). Cape Town, South Africa, . http://dx.doi.org/10.3182/20140824-6-ZA-1003.00794.