CURE-SMOTE algorithm and hybrid algorithm for feature selection and parameter optimization based on random forests

BMC Bioinformatics

Volumn 18, Issue 1, 2017, Pages

  Ma, Li ^a   Fan, Suohai ^a  

^a JINAN UNIVERSITY   (China)

Author keywords

Feature selection; Imbalance data; Intelligence algorithm; Parameter optimization; Random forests

Indexed keywords

CLASSIFICATION (OF INFORMATION); CURING; DATA HANDLING; DECISION TREES; FEATURE EXTRACTION; OPTIMIZATION; PARAMETER ESTIMATION;

CLASSIFICATION RESULTS; CLUSTERING USING REPRESENTATIVES; GENERALIZATION ABILITY; IMBALANCE DATUM; INTELLIGENCE ALGORITHMS; PARAMETER OPTIMIZATION; RANDOM FORESTS; SYNTHETIC MINORITY OVER-SAMPLING TECHNIQUES;

CLUSTERING ALGORITHMS;

CLASSIFICATION; EXPERIMENTAL MODEL; INFORMATION PROCESSING; INTELLIGENCE; NOISE; RANDOM FOREST; SIMULATION; ALGORITHM; AREA UNDER THE CURVE; CLUSTER ANALYSIS; RECEIVER OPERATING CHARACTERISTIC;

ALGORITHMS; AREA UNDER CURVE; CLUSTER ANALYSIS; ROC CURVE;

EID: 85015659687     PISSN: None     EISSN: 14712105     Source Type: Journal    
DOI: 10.1186/s12859-017-1578-z     Document Type: Article

References (79)

1. Ho TK. Random decision forests [C]//Document Analysis and Recognition. Proceedings of the Third International Conference on IEEE. 1995;1:278-82.
2. Ho TK. The random subspace method for constructing decision forests. IEEE Trans Pattern Anal Mach Intell. 1998;20(8):832-44.
3. Breiman L. Random forests. Mach Learn. 2001;45(1):5-32.
4. Hassan H, Badr A, Abdelhalim MB. Prediction of O-glycosylation sites using random forest and GA-tuned PSO technique. Bioinform Biol Insights. 2015;9:103.
5. Cerrada M, Zurita G, Cabrera D, et al. Fault diagnosis in spur gears based on genetic algorithm and random forest. Mech Syst Signal Process. 2016;70:87-103.
6. Malik AJ, Shahzad W, Khan FA. Network intrusion detection using hybrid binary PSO and random forests algorithm. Security and Communication Networks. 2015;8(16):2646-60.
7. López V, Fernández A, García S, et al. An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics. Inform Sci. 2013;250:113-41.
8. Sun Y, Wong AKC, Kamel MS. Classification of imbalanced data: A review. Int J Pattern Recognit Artif Intell. 2009;23(04):687-719.
9. Khoshgoftaar TM, Golawala M, Hulse JV. An empirical study of learning from imbalanced data using random forest [C]//19th IEEE International Conference on. IEEE Tools with Artificial Intelligence. 2007;2:310-7.
10. Batista GE, Prati RC, Monard MC. A study of the behavior of several methods for balancing machine learning training data. ACM Sigkdd Explorations Newsletter. 2004;6(1):20-9.
11. Chen JJ, Tsai CA, Young JF, et al. Classification ensembles for imbalanced class sizes in predictive toxicology. SAR QSAR Environ Res. 2005;16(6):517-29.
12. Pan X, Zhu L, Fan YX, et al. Predicting protein-RNA interaction amino acids using random forest based on submodularity subset selection. Comput Biol Chem. 2014;53:324-30.
13. Wu Q, Ye Y, Zhang H, et al. ForesTexter: an efficient random forest algorithm for imbalanced text categorization. Knowl-Based Syst. 2014;67:105-16.
14. Han M, Zhu XR. Hybrid algorithm for classification of unbalanced datasets. Control Theory & Applications. 2011;28(10):1485-9.
15. Tahir M, Khan A, Majid A, et al. Subcellular localization using fluorescence imagery: Utilizing ensemble classification with diverse feature extraction strategies and data balancing. Appl Soft Comput. 2013;13(11):4231-43.
16. Ali S, Majid A, Javed SG, et al. Can-CSC-GBE: Developing Cost-sensitive Classifier with Gentleboost Ensemble for breast cancer classification using protein amino acids and imbalanced data. Comput Biol Med. 2016;73:38-46.
17. Majid A, Ali S, Iqbal M, et al. Prediction of human breast and colon cancers from imbalanced data using nearest neighbor and support vector machines. Comput Methods Programs Biomed. 2014;113(3):792-808.
18. Robnik-Sikonja M. Improving random forests [M]//Machine Learning: ECML 2004. Springer Berlin Heidelberg, 2004: 359-370.
19. Li H B, Wang W, Ding H W, et al. Trees Weighting Random Forests Method for Classifying High-Dimensional Noisy Data [C]//2010 IEEE 7th International Conference on IEEE e-Business Engineering (ICEBE), 2010:160-163.
20. Jian-geng L, Gao Z-k. Setting of class weights in random forest for small-sample data. Comput Eng Appl. 2009;45(26):131-4.
21. Ma J-y, Wu X-z, Xie B-c. Quasi-adaptive random forest for classification. Application of Statistics and Management. 2010;29(5):805-11.
22. Strobl C, Boulesteix AL, Kneib T, Augustin T, Zeileis A. Conditional variable importance for random forests. BMC bioinformatics. 2008;9(1):1.
23. Li S, James Harner E, Adjeroh DA. Random KNN feature selection-a fast and stable alternative to Random Forests. BMC bioinformatics. 2011;12(1):1.
24. Yang F, Lu W, Luo L, et al. Margin optimization based pruning for random forest. Neuro computing. 2012;94:54-63.
25. Efron B, Tibshirani R. An introduction to the boostrap [M]. NewYork: Chapman & Hall; 1993.
26. Breiman L. Bagging predictors. Mach Learn. 1996;24(2):123-40.
27. Quinaln J R. C4.5: programs for machine learning [M]. Morgan kuafmann, 1993.
28. Breiman L, Friedman J, Olshen R, and Stone C. Classification and Regression Trees. Boca Raton, FL: CRC Press; 1984.
29. He H, Garcia EA. Learning from imbalanced data. IEEE Trans Knowl Data Eng. 2009;21(9):1263-84.
30. Lusa L. Class prediction for high-dimensional class-imbalanced data. BMC bioinformatics. 2010;11(1):523.
31. Yan H, Zha W-x. Comparison on classification performance between random forests and support vector machine. Software. 2012;33(6):107-10.
32. Chawla NV, Bowyer KW, Hall LO, et al. SMOTE: Synthetic minority over-sampling technique. J Artif Intell Res. 2002;16:321-57.
33. Chawla NV, Lazarevic A, Hall LO, et al. SMOTE Boost. Improving prediction of the minority class in Boosting. In: Proceedings of the 7th European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD 2003), Lecture Notes in Computer Science, vol 2838. Springer-Verlag: Berlin; 2003. p. 107-19.
34. Blagus R, Lusa L. SMOTE for high-dimensional class-imbalanced data. BMC Bioinformatics. 2013;14:106.
35. Han H, Wan W Y, Mao B H. Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning [C]//LNCS 3644: ICIC 2005, Part I, 2005: 878-887.
36. Bunkhumpornpat C, Sinapiromsaran K, Lursinsap C. Safe-level-SMOTE: Safe-level-synthetic minority over-sampling technique for handling the class imbalanced problem. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining, Lecture Notes on Computer Science, vol 5476. Springer-Verlag: Berlin; 2009. p. 475-82.
37. Cieslak D A, Chawla N V, Striegel A. Combating imbalance in network intrusion datasets [C]//GrC. 2006: 732-737.
38. García V, Sánchez J S, Mollineda R A. On the use of surrounding neighbors for synthetic over-sampling of the minority class [C]//Proceedings of the 8th conference on Simulation, modeling and optimization. World Scientific and Engineering Academy and Society (WSEAS), 2008: 389-394.
39. Peng L, Wang X-l, Yuan-chao L. A classification method for imbalance data Set based on hybrid strategy. Acta Electron Sin. 2007;35(11):2161-5.
40. Zheng-feng C. Study on optimization of random forests algorithm [D]. Beijing: Capital University of Economics and Business; 2014.
41. Zhao W, Xu M, Jia X, et al. A Classification Method for Imbalanced Data Based on SMOTE and Fuzzy Rough Nearest Neighbor Algorithm. In: Yao Y, et al (eds) Rough Sets, Fuzzy Sets, Data Mining, and Granular Computing. Lecture Notes in Computer Science, vol 9437. Springer-Verlag: Berlin; 2015. p. 340-51.
42. Nekooeimehr I, Lai-Yuen SK. Adaptive semi-unsupervised weighted oversampling (A-SUWO) for imbalanced datasets. Expert Systems with Applications. 2016;46:405-16.
43. Sáez JA, Luengo J, Stefanowski J, et al. SMOTE-IPF: Addressing the noisy and borderline examples problem in imbalanced classification by a re-sampling method with filtering. Inform Sci. 2015;291:184-203.
44. Abdi L, Hashemi S. To combat multi-class imbalanced problems by means of over-sampling techniques. IEEE Trans Knowl Data Eng. 2016;28(1):238-51.
45. Guha S, Rastogi R, Shim K. CURE: an efficient clustering algorithm for large databases [C]//ACM SIGMOD Record. ACM. 1998;27(2):73-84.
46. Ya-jian Z, Xu C, Ji-guo L. Unsupervised anomaly detection method based on improved CURE clustering algorithm. J Communications. 2010;31(7):18-23.
47. Pavlidis P, Weston J, Cai J, et al. Gene functional classification from heterogeneous data. In: Proceedings of the fifth Annual International Conference on Computational Molecular Biology. 2001;249-55.
48. Sharma A, Imoto S, Miyano S, et al. Null space based feature selection method for gene expression data. Int J Mach Learn Cybern. 2012;3(4):269-76.
49. Ghalwash MF, Cao XH, Stojkovic I, et al. Structured feature selection using coordinate descent optimization. BMC bioinformatics. 2016;17(1):1.
50. Saeys Y, Inza I, Larrañaga P. A review of feature selection techniques in bioinformatics. Bioinformatics. 2007;23(19):2507-17.
51. Guo S, Guo D, Chen L, et al. A centroid-based gene selection method for microarray data classification. J Theor Biol. 2016;400:32-41.
52. Sharbaf FV, Mosafer S, Moattar MH. A hybrid gene selection approach for microarray data classification using cellular learning automata and ant colony optimization. Genomics. 2016;107(6):231-8.
53. Golub TR, Slonim DK, Tamayo P, et al. Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science. 1999;286(5439):531-7.
54. Furey TS, Cristianini N, Duffy N, et al. Support vector machine classification and validation of cancer tissue samples using microarray expression data. Bioinformatics. 2000;16(10):906-14.
55. Sharma A, Imoto S, Miyano S. A top-r feature selection algorithm for microarray gene expression data. IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB). 2012;9(3):754-64.
56. Chinnaswamy A, Srinivasan R. Hybrid Feature Selection Using Correlation Coefficient and Particle Swarm Optimization on Microarray Gene Expression Data. In: Snášel V, et al (eds) Innovations in Bio-Inspired Computing and Applications. Advances in Intelligent Systems and Computing, vol 424. Springer International Publishing Switzerland; 2016. p. 229-39.
57. Destrero A, Mosci S, De Mol C, et al. Feature selection for high-dimensional data. Comput Manag Sci. 2009;6(1):25-40.
58. Zhu S, Wang D, Yu K, et al. Feature selection for gene expression using model-based entropy. IEEE/ACM Trans Comput Biol Bioinform. 2010;7(1):25-36.
59. Kausar N, Majid A. Random forest-based scheme using feature and decision levels information for multi-focus image fusion. Pattern Anal Applic. 2016;19(1):221-36.
60. Menze BH, et al. A comparison of random forest and its Gini importance with standard chemometric methods for the feature selection and classification of spectral data. BMC bioinformatics. 2009;10(1):213.
61. Strobl C, et al. Bias in random forest variable importance measures: Illustrations, sources and a solution. BMC bioinformatics. 2007;8(1):1.
62. Zhou Q, Zhou H, Li T. Cost-sensitive feature selection using random forest: Selecting low-cost subsets of informative features. Knowl-Based Syst. 2016;95:1-11.
63. Díaz-Uriarte R, De Andres SA. Gene selection and classification of microarray data using random forest. BMC bioinformatics. 2006;7(1):1.
64. Lariviere B, Van den Poel D. Predicting customer retention and profitability by using random forests and regression forests techniques. Expert Systems with Applications. 2005;29:472-84.
65. Rodriguez-Galiano VF, Ghimire B, Rogan J, Chica-Olmo M, Rigol-Sanchez JP. An assessment of the effectiveness of a random forest classifier for landcover classification. ISPRS J Photogramm Remote Sens. 2012;67:93-104.
66. Bernard S, Heutte L, Adam S. Influence of Hyper parameters on Random Forest Accuracy [C]//Proceedings of the 8th International workshop on multiple classifier systems. Berlin, Heidelberg: Springer; 2009. p. 171-80.
67. Yu L, Chun-xia Z. Estimation of the hyper-parameter in random forest based on out-of-bag sample. J Syst Eng. 2011;26(4):566-72.
68. Martinez-Munoz G, Suarez A. Out-of-bag estimation of the optimal sample size in bagging. Pattern Recogn. 2010;43(1):143-52.
69. Ming-yuan Z, Yong T, Chong F, Ming-tian Z. Feature selection and parameter optimization for SVM based on genetic algorithm with feature chromosomes. Control and Decision. 2010;25(8):1133-8.
70. Lei L, Gao L, Shijie Z. Question of SVM kernel parameter optimization with particle swarm algorithm based on neural network. Comput Eng Appl. 2015;51(4):162-4.
71. Leifu GAO, Shijie ZHAO, Jing GAO. Application of artificial fish-swarm algorithm in SVM parameter optimization selection. Comput Eng Appl. 2013;49(23):86-90.
72. Xin-guang SHAO, Hui-zhong YANG, Gang CHEN. Parameters selection and application of support vector machines based on particle swarm optimization algorithm. Control Theory & Applications. 2006;23(5):740-4.
73. Kennedy J,Eberhart R. Particle Swarm Optimization [C]//IEEE International Conference on Neural Networks,1995 Proceedings,1995:1942-1948.
74. Xiao-lei L, Zhi-jiang S, Ji-xin Q. An optimizing method based on autonomous animals: Fish-swarm Algorithm. Systems Engineering-Theory & Practice. 2002;22(11):31-8.
75. Chen J, Tang YY, Fang B, et al. In silico prediction of toxic action mechanisms of phenols for imbalanced data with Random Forest learner. J Mol Graph Model. 2012;35:21-7.
76. Espíndola R P, Ebecken N F F. On extending f-measure and g-mean metrics to multi-class problems [C]//Sixth international conference on data mining, text mining and their business applications, Wessex Institute of Technology, UK. 2005, 35: 25-34.
77. Janitza S, Strobl C, Boulesteix AL. An AUC-based permutation variable importance measure for random forests. BMC bioinformatics. 2013;14(1):119.
78. Breiman L. Out-of-bag Estimation [R]. Berkeley: Statistics Department, University of California; 1996.
79. Mellor A, Boukir S, Haywood A, et al. Exploring issues of training data imbalance and mislabeling on random forest performance for large area land cover classification using the ensemble margin. ISPRS J Photogramm Remote Sens. 2015;105:155-68.