Increasing diversity in random forest learning algorithm via imprecise probabilities

Expert Systems with Applications

Volumn 97, Issue , 2018, Pages 228-243

  Abellán, Joaquín ^a   Mantas, Carlos J ^a   Castellano, Javier G ^a   Moral García, Serafín ^a  

^a UNIVERSITY OF GRANADA   (Spain)

Author keywords

Classification; Ensemble schemes; Imprecise probabilities; Random forest; Uncertainty measures

Indexed keywords

CLASSIFICATION (OF INFORMATION); DECISION TREES; FORESTRY; PROBABILITY; UNCERTAINTY ANALYSIS;

CLASS NOISE; CLASSIFICATION ALGORITHM; ENSEMBLE SCHEMES; IMPRECISE PROBABILITIES; OVERFITTING; RANDOM FORESTS; UNCERTAINTY MEASURES;

LEARNING ALGORITHMS;

EID: 85042299569     PISSN: 09574174     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.eswa.2017.12.029     Document Type: Article

References (52)

1. Abellán, J., Uncertainty measures on probability intervals from the imprecise dirichlet model. International Journal of General Systems 35:5 (2006), 509–528, 10.1080/03081070600687643.
2. Abellán, J., Ensembles of decision trees based on imprecise probabilities and uncertainty measures. Information Fusion 14:4 (2013), 423–430.
3. Abellán, J., Klir, G., Moral, S., Disaggregated total uncertainty measure for credal sets. International Journal of General Systems 35:1 (2006), 29–44, 10.1080/03081070500473490.
4. Abellán, J., Mantas, C.J., Improving experimental studies about ensembles of classifiers for bankruptcy prediction and credit scoring. Expert Systems with Applications 41:8 (2014), 3825–3830, 10.1016/j.eswa.2013.12.003.
5. Abellán, J., Mantas, C.J., Castellano, J.G., Adaptativecc4.5: Credal c4.5 with a rough class noise estimator. Expert Systems with Applications 92:Supplement C (2018), 363–379, 10.1016/j.eswa.2017.09.057.
6. Abellán, J., Masegosa, A., A filter-wrapper method to select variables for the naive bayes classifier based on credal decision trees. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 17:06 (2009), 833–854, 10.1142/S0218488509006297.
7. Abellán, J., Masegosa, A.R., An ensemble method using credal decision trees. European journal of operational research 205:1 (2010), 218–226.
8. Abellán, J., Masegosa, A.R., Bagging schemes on the presence of class noise in classification. Expert Systems with Applications 39:8 (2012), 6827–6837, 10.1016/j.eswa.2012.01.013.
9. Abellán, J., Moral, S., Building classification trees using the total uncertainty criterion. International Journal of Intelligent Systems 18:12 (2003), 1215–1225, 10.1002/int.10143.
10. Abellán, J., Moral, S., Upper entropy of credal sets. applications to credal classification. International Journal of Approximate Reasoning 39:2–3 (2005), 235–255 Imprecise Probabilities and Their Applications. doi: 10.1016/j.ijar.2004.10.001.
11. Alcalá-Fdez, J., Sánchez, L., García, S., del Jesus, M., Ventura, S., Garrell, J., et al. Keel: A software tool to assess evolutionary algorithms for data mining problems. Soft Computing 13:3 (2009), 307–318, 10.1007/s00500-008-0323-y.
12. Azqhandi, M.A., Ghaedi, M., Yousefi, F., Jamshidi, M., Application of random forest, radial basis function neural networks and central composite design for modeling and/or optimization of the ultrasonic assisted adsorption of brilliant green on zns-np-ac. Journal of Colloid and Interface Science 505 (2017), 278–292, 10.1016/j.jcis.2017.05.098.
13. Bai, S., Growing random forest on deep convolutional neural networks for scene categorization. Expert Systems with Applications 71 (2017), 279–287.
14. Breiman, L., Bagging predictors. Machine Learning 24:2 (1996), 123–140, 10.1023/A:1018054314350.
15. Breiman, L., Random forests. Machine Learning 45:1 (2001), 5–32, 10.1023/A:1010933404324.
16. Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J., Classification and regression trees. Statistics/probability series, 1984, Wadsworth Publishing Company, Belmont, California, U.S.A.
17. Brown, G., Wyatt, J., Harris, R., Yao, X., Diversity creation methods: A survey and categorisation. Journal of Information Fusion 6 (2005), 5–20.
18. Demšar, J., Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7 (2006), 1–30.
19. Dietterich, T.G., Ensemble methods in machine learning. Proceedings of the first international workshop on multiple classifier systems MCS ’00, 2000, Springer-Verlag, London, UK, UK, 1–15.
20. Dietterich, T.G., An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning 40:2 (2000), 139–157, 10.1023/A:1007607513941.
21. Frenay, B., Verleysen, M., Classification in the presence of label noise: a survey. IEEE Transactions on Neural Networks and Learning Systems 25:5 (2014), 845–869, 10.1109/TNNLS.2013.2292894.
22. Freund, Y., Schapire, R.E., Experiments with a new boosting algorithm. Saitta, L., (eds.) Proceedings of the thirteenth international conference on machine learning (ICML 1996), 1996, Morgan Kaufmann, 148–156.
23. Friedman, M., The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association 32 (1937), 675–701.
24. Friedman, M., A comparison of alternative tests of significance for the problem of m rankings. The Annals of Mathematical Statistics 11:1 (1940), 86–92, 10.1214/aoms/1177731944.
25. Gauba, H., Kumar, P., Roy, P.P., Singh, P., Dogra, D.P., Raman, B., Prediction of advertisement preference by fusing {EEG} response and sentiment analysis. Neural Networks 92 (2017), 77–88 Advances in Cognitive Engineering Using Neural Networks. doi: 10.1016/j.neunet.2017.01.013.
26. Hand, D., Discrimination and classification. 1981, John Wiley.
27. Hand, D.J., Construction and assessment of classification rules. 1997, John Wiley and Sons, New York.
28. Jiang, X., Abdel-Aty, M., Hu, J., Lee, J., Investigating macro-level hotzone identification and variable importance using big data: A random forest models approach. Neurocomputing 181 (2016), 53–63 Big Data Driven Intelligent Transportation Systems. doi: 10.1016/j.neucom.2015.08.097.
29. Klir, G.J., Uncertainty and information: foundations of generalized information theory. 2005, John Wiley And Sons, Inc., 10.1002/0471755575.
30. Krauss, C., Do, X.A., Huck, N., Deep neural networks, gradient-boosted trees, random forests: Statistical arbitrage on the s&p 500. European Journal of Operational Research 259:2 (2017), 689–702, 10.1016/j.ejor.2016.10.031.
31. Kulkarni, V.Y., Petare, M., Sinha, P.K., Analyzing random forest classifier with different split measures. Proceedings of the second international conference on soft computing for problem solving (SOCPROS 2012), 2012, Springer India, New Delhi, 691–699, 10.1007/978-81-322-1602-5_74.
32. Li, T., Ni, B., Wu, X., Gao, Q., Li, Q., Sun, D., On random hyper-class random forest for visual classification. Neurocomputing 172 (2016), 281–289, 10.1016/j.neucom.2014.10.101.
33. Lichman, M. (2013). UCI machine learning repository. http://archive.ics.uci.edu/ml.
34. Mantas, C.J., Abellán, J., Analysis and extension of decision trees based on imprecise probabilities: Application on noisy data. Expert Systems with Applications 41:5 (2014), 2514–2525, 10.1016/j.eswa.2013.09.050.
35. Mantas, C.J., Abellán, J., Credal-c4.5: Decision tree based on imprecise probabilities to classify noisy data. Expert Systems with Applications 41:10 (2014), 4625–4637, 10.1016/j.eswa.2014.01.017.
36. Mantas, C.J., Abellán, J., Castellano, J.G., Analysis of credal-c4.5 for classification in noisy domains. Expert Systems with Applications 61 (2016), 314–326, 10.1016/j.eswa.2016.05.035.
37. Menze, B.H., Kelm, B.M., Splitthoff, D.N., Koethe, U., Hamprecht, F.A., On oblique random forests. Proceedings of the 2011 european conference on machine learning and knowledge discovery in databases - Volume Part ii ECML PKDD'11, 2011, Springer-Verlag, Berlin, Heidelberg, 453–469.
38. Nemenyi, P., Distribution-free multiple comparisons, 1963, Princeton University, New Jersey, USA Doctoral dissertation.
39. Pearl, J., Probabilistic reasoning in intelligent systems. 1988, Morgan Kaufmann, San Mateo, CA.
40. Quinlan, J.R., Induction of decision trees. Machine Learning 1:1 (1986), 81–106, 10.1023/A:1022643204877.
41. Quinlan, J.R., C4.5: Programs for machine learning. 1993, Morgan Kaufmann Publishers Inc, San Francisco, CA, USA.
42. Raileanu, L.E., Stoffel, K., Theoretical comparison between the gini index and information gain criteria. Annals of Mathematics and Artificial Intelligence 41:1 (2004), 77–93, 10.1023/B:AMAI.0000018580.96245.c6.
43. Ren, Y., Zhang, L., Suganthan, P.N., Ensemble classification and regression-recent developments, applications and future directions [review article]. IEEE Computational Intelligence Magazine 11:1 (2016), 41–53, 10.1109/MCI.2015.2471235.
44. Sáez, J.A., Luengo, J., Herrera, F., Evaluating the classifier behavior with noisy data considering performance and robustness: The equalized loss of accuracy measure. Neurocomputing 176 (2014), 26–35 Recent Advancements in Hybrid Artificial Intelligence Systems and its Application to Real-World Problems, selected papers from the {HAIS} 2013 conference. doi: 10.1016/j.neucom.2014.11.086.
45. Shannon, C.E., A mathematical theory of communication. Bell System Technical Journal 27:3 (1948), 379–423, 10.1002/j.1538-7305.1948.tb01338.x.
46. Tsymbal, A., Pechenizkiy, M., Cunningham, P., Diversity in search strategies for ensemble feature selection. Information Fusion 6:1 (2005), 83–98, 10.1016/j.inffus.2004.04.003.
47. Walley, P., Inferences from multinomial data; learning about a bag of marbles (with discussion). Journal of the Royal Statistical Society. Series B (Methodological) 58:1 (1996), 3–57, 10.2307/2346164.
48. Wang, S., Yin, Y., Cao, G., Wei, B., Zheng, Y., Yang, G., Hierarchical retinal blood vessel segmentation based on feature and ensemble learning. Neurocomputing 149 (2015), 708–717, 10.1016/j.neucom.2014.07.059.
49. Witten, I.H., Frank, E., Data mining: Practical machine learning tools and techniques. Morgan Kaufmann Series in Data Management Systems, 2nd, 2005, Morgan Kaufmann Publishers Inc, San Francisco, CA, USA.
50. Zhang, L., Suganthan, P.N., Random forests with ensemble of feature spaces. Pattern Recognition 47:10 (2014), 3429–3437, 10.1016/j.patcog.2014.04.001.
51. Zhang, L., Suganthan, P.N., Oblique decision tree ensemble via multisurface proximal support vector machine. IEEE Trans. Cybernetics 45:10 (2015), 2165–2176, 10.1109/TCYB.2014.2366468.
52. Zhang, L., Suganthan, P.N., Benchmarking ensemble classifiers with novel co-trained kernal ridge regression and random vector functional link ensembles [research frontier]. IEEE Computer International Magazine 12:4 (2017), 61–72, 10.1109/MCI.2017.2742867.