New margin-based subsampling iterative technique in modified random forests for classification

Knowledge-Based Systems

Volumn 182, Issue , 2019, Pages

  Feng, Wei ^a,b   Dauphin, Gabriel ^c   Huang, Wenjiang ^b   Quan, Yinghui ^a   Liao, Wenzhi ^d  

^a XIDIAN UNIVERSITY   (China)

^b INSTITUTE OF GEOLOGY AND GEOPHYSICS   (China)

^c UNIVERSITÉ PARIS 13   (France)

^d GHENT UNIVERSITY   (Belgium)

Author keywords

Classification; Diversity; Ensemble margin; Random forests; Sub sampling

Indexed keywords

ADAPTIVE BOOSTING; DECISION TREES; ITERATIVE METHODS; SUPPORT VECTOR MACHINES;

CLASSIFICATION MODELS; CLASSIFICATION RESULTS; DIVERSITY; ENSEMBLE CLASSIFIERS; ENSEMBLE MARGIN; IMBALANCED DATA-SETS; RANDOM FORESTS; SUB-SAMPLING;

CLASSIFICATION (OF INFORMATION);

EID: 85071325715     PISSN: 09507051     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.knosys.2019.07.016     Document Type: Article

References (56)

1. L. Jin, L. Feng, G. Liu, C. Wang, Personal web revisitation by context and content keywords with relevance feedback, IEEE Transactions on Knowledge and Data Engineering 29 (2017) 1508–1521
2. T. Tran, D. Phung, S. Venkatesh, Collaborative filtering via sparse markov random fields, Information Sciences 369 (2016) 221–237
3. B. Bhasuran, G. Murugesan, S. Abdulkadhar, J. Natarajan, Stacked ensemble combined with fuzzy matching for biomedical named entity recognition of diseases, Journal of Biomedical Informatics 64 (2016) 1–9
4. A. E. Ferreira, D. Alarcao, Real-time blind source separation system with applications to distant speech recognition, Applied Acoustics 113 (2016) 170–184
5. W. Feng, W. Bao, Weight-based rotation forest for hyperspectral image classification, IEEE Geoscience and Remote Sensing Letters 14 (2017) 2167–2171
6. J. Lu, V. Behbood, P. Hao, H. Zuo, S. Xue, G. Zhang, Transfer learning using computational intelligence: A survey, Knowledge-Based Systems 80 (2015) 14–23
7. J. Lu, X. Yang, G. Zhang, Support vector machine-based multi-source multi-attribute information integration for situation assessment, Expert Systems with Applications 34 (2008) 1333–1340
8. J. Abelln, C. J. Mantas, J. G. Castellano, S. Moral-Garca, Increasing diversity in random forest learning algorithm via imprecise probabilities, Expert Systems with Applications 97 (2018) 228–243
9. L. Breiman, Bagging predictors, Machine Learning 24 (1996) 123–140
10. B. Krawczyk, L. L. Minku, J. Gama, J. Stefanowski, M. Wozniak, Ensemble learning for data stream analysis: A survey, Information Fusion 37 (2017) 132–156
11. Z. H. Zhou, Ensemble Methods: Foundations and Algorithms, Chapman and Hall/CRC, 2012
12. L. Breiman, J. Friedman, R. Olshen, C. Stone, Classification and Regression Trees, Publisher: Wadsworth, 1984
13. J. Xia, S. Zhang, G. Cai, L. Li, Q. Pan, J. Yan, G. Ning, Adjusted weight voting algorithm for random forests in handling missing values, Pattern Recognition 69 (2017) 52–60
14. R. Houborg, M. F. McCabe, A hybrid training approach for leaf area index estimation via cubist and random forests machine-learning, ISPRS Journal of Photogrammetry and Remote Sensing 135 (2018) 173–188
15. D. Turner, A. Lucieer, Z. Malenovsk, D. King, S. A. Robinson, Assessment of antarctic moss health from multi-sensor uas imagery with random forest modelling, International Journal of Applied Earth Observation and Geoinformation 68 (2018) 168–179
16. L. Breiman, Random forests, Machine Learning 45 (2001) 5–32
17. Q. Dai, R. Ye, Z. Liu, Considering diversity and accuracy simultaneously for ensemble pruning, Applied Soft Computing 58 (2017) 75–91
18. H. Guo, H. Liu, R. Li, C. Wu, Y. Guo, M. Xu, Margin & diversity based ordering ensemble pruning, Neurocomputing 275 (2018) 237–246
19. W. Feng, W. Huang, J. Ren, Class imbalance ensemble learning based on the margin theory, Applied Sciences 8 (2018)
20. L. Kuncheva, C. Whitaker, Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy, Machine Learning 51 (2003) 181–207
21. J. I. Orlando, E. P., M. Fresno, M. B. Blaschko, An ensemble deep learning based approach for red lesion detection in fundus images, Computer Methods and Programs in Biomedicine 153 (2018) 115–127
22. W. Feng, S. Boukir, Class noise removal and correction for image classification using ensemble margin, in: 2015 IEEE International Conference on Image Processing (ICIP), pp. 4698–4702
23. H. Guo, Y. Li, S. Jennifer, M. Gu, Y. Huang, B. Gong, Learning from class-imbalanced data: Review of methods and applications, Expert Systems with Applications 73 (2017) 220–239
24. C. Chen, A. Liaw, L. Breiman, Using Random Forest to Learn Imbalanced Data, Technical Report 666, Department of Statistics, University of California, Berkeley., 2004.
25. S. Bernard, S. Adam, L. Heutte, Dynamic random forests, Pattern Recognition Letters 33 (2012) 1580–1586
26. C. Zhang, X. Bian, P. Liu, X. Tan, Q. Fan, W. Liu, L. Lin, Subagging for the improvement of predictive stability of extreme learning machine for spectral quantitative analysis of complex samples, Chemometrics and Intelligent Laboratory Systems 161 (2017) 43–48
27. R. K. H. Galvao, M. C. U. Araujo, M. N. Martins, G. E. Jose, M. J. C. Pontes, E. C. Silva, T. C. B. Saldanha, An application of subagging for the improvement of prediction accuracy of multivariate calibration models, Chemometrics and Intelligent Laboratory Systems 81 (2006) 60–67
28. G. Paleologo, A. Elisseeff, G. Antonini, Subagging for credit scoring models, European Journal of Operational Research 201 (2010) 490–499
29. R. Genuer, J. Poggi, C. Tuleau-Malot, N. Villa-Vialaneix, Random forests for big data, Big Data Research 9 (2017) 28–46
30. A. Buja, W. Stuetzle, Observations on bagging., Preprint. (2002).
31. P. Buhlmann, Bagging, subagging and bragging for improving some prediction algorithms, in: M. G. Akritas, D. N. Politis (Eds.), Recent Advances and Trends in Nonparametric Statistics, JAI, Amsterdam, 2003, pp. 19–34
32. L. Breiman, Pasting small votes for classification in large databases and on-line, Machine Learning 36 (1999) 85–103
33. G. Martinez-Munoz, A. Suarez, Out-of-bag estimation of the optimal sample size in bagging, Pattern Recognition 43 (2010) 143–152
34. L. Breiman, Out-Of-Bag Estimation, Technical Report, University of California, 1996.
35. C. Cortes, V. Vapnik, Support-vector networks, Machine Learning 20 (1995) 273–297
36. L. Li, A. Pratap, H. T. Lin, Y. S. Abu-Mostafa, Improving generalization by data categorization, in: Knowledge Discovery in Databases: PKDD 2005, volume 3721, Springer Berlin Heidelberg, 2005, pp. 157–168
37. E. Marchiori, Class conditional nearest neighbor for large margin instance selection, Pattern Analysis and Machine Intelligence, IEEE Transactions on 32 (2010) 364–370
38. M. Alshawabkeh, Hypothesis margin based weighting for feature selection using boosting: theory, algorithms and applications, Ph.D. thesis, Northeastern University, 2013
39. L. J. Li, B. Zou, Q. H. Hu, X. Q. Wu, D. R. Yu, Dynamic classifier ensemble using classification confidence, Neurocomputing 99 (2013) 581–591
40. C. H. Shen, H. X. Li, Boosting through optimization of margin distributions, Trans. Neur. Netw. 21 (2010) 659–666
41. Z. X. Xie, Y. Xu, Q. H. Hu, P. F. Zhu, Margin distribution based bagging pruning, Neurocomputing 85 (2012) 11–19
42. N. V. Chawla, K. W. Bowyer, L. O. Hall, W. P. Kegelmeyer, Smote: Synthetic minority over-sampling technique, J. Artif. Int. Res. 16 (2002) 321–357
43. M. S. Santos, J. P. Soares, P. H. Abreu, H. Araujo, J. Santos, Cross-validation for imbalanced datasets: Avoiding overoptimistic and overfitting approaches [research frontier], IEEE Computational Intelligence Magazine 13 (2018) 59–76
44. V. Vapnik, The Nature of Statistical Learning Theory, Springer-Verlag New York, Inc., 1995
45. Y. Freund, R. Schapire, A decision-theoretic generalization of on-line learning and an application to boosting, Journal of Computer and System Sciences 55 (1997) 119–139
46. K. Crammer, R. Gilad-bachrach, A. Navot, N. Tishby, Margin analysis of the lvq algorithm, in: Advances in Neural Information Processing Systems 2002, MIT press, 2002, pp. 462–469
47. R. Schapire, Y. Freund, P. Bartlett, W. Lee, Boosting the margin: a new explanation for the effectiveness of voting methods, The Annals of Statistics 26 (1998) 1651–1686
48. P. Probst, M. Wright, A. Boulesteix, Hyperparameters and tuning strategies for random forest, arXiv preprint arXiv:1804.03515 (2018).
49. R. Kohavi, D. Wolpert, Bias plus variance decomposition for zero-one loss functions, in: 13th International Conference of Machine Learning, ICML’96, pp. 275–283
50. J. A. Saeź, B. Krawczyk, M. Wozniaḱ, Analyzing the oversampling of different classes and types of examples in multi-class imbalanced datasets, Pattern Recognition 57 (2016) 164–178
51. M. Kapp, R. Sabourin, P. Maupin, An empirical study on diversity measures and margin theory for ensembles of classifiers, in: 10th International Conference on Information Fusion, pp. 1–8
52. A. Asuncion, D. Newman, UCI machine learning repository, 2007.
53. J. Demsar, Statistical comparisons of classifiers over multiple data sets., Journal of Machine Learning Research 7 (2006) 1–30
54. L. Wei, J. J. Chen, Class-imbalanced classifiers for high-dimensional data., Briefings in Bioinformatics 14 (2013) 13–26
55. C. Leistner, A. Saffari, P. M. Roth, H. Bischof, On robustness of on-line boosting - a competitive study, in: 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops, pp. 1362–1369
56. R. E. Schapire, The strength of weak learnability, Mach. Learn. 5 (1990) 197–227