A Random Forest approach using imprecise probabilities

Knowledge-Based Systems

Volumn 134, Issue , 2017, Pages 72-84

  Abellán, Joaquín ^a   Mantas, Carlos J ^a   Castellano, Javier G ^a  

^a UNIVERSITY OF GRANADA   (Spain)

Author keywords

Class noise; Classification; Imprecise probabilities; Random Forest; Uncertainty measures

Indexed keywords

DATA MINING; DECISION TREES; PROBABILITY; RANDOM PROCESSES; UNCERTAINTY ANALYSIS;

BASE CLASSIFIERS; CLASS NOISE; IMPRECISE PROBABILITIES; RANDOM FOREST CLASSIFIER; RANDOM FORESTS; RANDOMIZATION PROCESS; SINGLE DECISION; UNCERTAINTY MEASURES;

CLASSIFICATION (OF INFORMATION);

EID: 85028006288     PISSN: 09507051     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.knosys.2017.07.019     Document Type: Article

References (43)

1. Hand, D.J., Construction and Assessment of Classification Rules. 1997, John Wiley and Sons, New York.
2. Hand, D., Discrimination and Classification. 1981, John Wiley.
3. Quinlan, J.R., C4.5: Programs for Machine Learning. 1993, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
4. Pearl, J., Probabilistic Reasoning in Intelligent Systems. 1988, Morgan Kaufmann, San Mateo, CA.
5. Quinlan, J.R., Induction of decision trees. Mach. Learn. 1:1 (1986), 81–106, 10.1023/A:1022643204877.
6. Tsymbal, A., Pechenizkiy, M., Cunningham, P., Diversity in search strategies for ensemble feature selection. Inf. Fusion 6:1 (2005), 83–98, 10.1016/j.inffus.2004.04.003.
7. Breiman, L., Bagging predictors. Mach. Learn. 24:2 (1996), 123–140, 10.1023/A:1018054314350.
8. Freund, Y., Schapire, R.E., Experiments with a new boosting algorithm. Saitta, L., (eds.) Proceedings of the Thirteenth International Conference on Machine Learning (ICML 1996), 1996, Morgan Kaufmann, 148–156.
9. Breiman, L., Random forests. Mach. Learn. 45:1 (2001), 5–32, 10.1023/A:1010933404324.
10. D.H. Wolpert, The Supervised Learning No-Free-Lunch Theorems, Springer London, London, pp. 25–42. doi: 10.1007/978-1-4471-0123-9-3.
11. Fernández-Delgado, M., Cernadas, E., Barro, S., Amorim, D., Do we need hundreds of classifiers to solve real world classification problems?. J. Mach. Learn. Res. 15:1 (2014), 3133–3181.
12. Abellán, J., Ensembles of decision trees based on imprecise probabilities and uncertainty measures. Inf. Fusion 14:4 (2013), 423–430.
13. Abellán, J., An application of non-parametric predictive inference on multi-class classification high-level-noise problems. Expert Syst. Appl. 40:11 (2013), 4585–4592, 10.1016/j.eswa.2013.01.066.
14. Frenay, B., Verleysen, M., Classification in the presence of label noise: a survey. Neural Netw. Learn. Syst. IEEE Trans. 25:5 (2014), 845–869, 10.1109/TNNLS.2013.2292894.
15. Klir, G.J., Uncertainty and Information: Foundations of Generalized Information Theory. 2005, John Wiley And Sons, Inc., 10.1002/0471755575.
16. Walley, P., Inferences from multinomial data; learning about a bag of marbles (with discussion). J. R. Stat. Soc. Ser. B (Methodological) 58:1 (1996), 3–57, 10.2307/2346164.
17. Abellán, J., Moral, S., Building classification trees using the total uncertainty criterion. Int. J. Intell. Syst. 18:12 (2003), 1215–1225, 10.1002/int.10143.
18. Abellán, J., Klir, G., Moral, S., Disaggregated total uncertainty measure for credal sets. Int. J. Gen. Syst. 35:1 (2006), 29–44, 10.1080/03081070500473490.
19. Abellán, J., Moral, S., Upper entropy of credal sets. applications to credal classification imprecise Probabilities and Their Application. Int. J. Approximate Reasoning 39:2–3 (2005), 235–255, 10.1016/j.ijar.2004.10.001 Imprecise Probabilities and Their Applications.
20. Abellán, J., Masegosa, A., A filter-wrapper method to select variables for the naive bayes classifier based on credal decision trees. Int. J. Uncertainty Fuzziness Knowl. Based Syst. 17:06 (2009), 833–854, 10.1142/S0218488509006297.
21. Abellán, J., Masegosa, A.R., Bagging schemes on the presence of class noise in classification. Expert Syst. Appl. 39:8 (2012), 6827–6837, 10.1016/j.eswa.2012.01.013.
22. Abellán, J., Masegosa, A., An experimental study about simple decision trees for bagging ensemble on datasets with classification noise. Sossai, C., Chemello, G., (eds.) Symbolic and Quantitative Approaches to Reasoning with Uncertainty Lecture Notes in Computer Science, 5590, 2009, Springer Berlin Heidelberg, 446–456, 10.1007/978-3-642-02906-6_39.
23. Jaynes, E., On the rationale of maximum-entropy methods. Proc. IEEE 70:9 (1982), 939–952, 10.1109/PROC.1982.12425.
24. Shannon, C.E., A mathematical theory of communication. Bell Syst. Tech. J. 27:3 (1948), 379–423, 10.1002/j.1538-7305.1948.tb01338.x.
25. Abellán, J., Uncertainty measures on probability intervals from the imprecise dirichlet model. Int. J. Gen. Syst. 35:5 (2006), 509–528, 10.1080/03081070600687643.
26. Shahpari, A., Seyedin, S.A., Using mutual aggregate uncertainty measures in a threat assessment problem constructed by dempster–shafer network. IEEE Trans. Syst. Man Cybern. 45:6 (2015), 877–886, 10.1109/TSMC.2014.2378213.
27. Deng, Y., Deng entropy. Chaos Solitons Fractals 91 (2016), 549–553 doi: 110.1016/j.chaos.2016.07.014.
28. Yang, Y., Han, D., A new distance-based total uncertainty measure in the theory of belief functions. J. Knowl. Based Syst. 94:C (2016), 114–123, 10.1016/j.knosys.2015.11.014.
29. Abellán, J., Bossé, É., Drawbacks of uncertainty measures based on the pignistic transformation. IEEE Trans. Syst. Man Cybern.(99), 2016, 1–7, 10.1109/TSMC.2016.2597267.
30. Abellán, J., Analyzing properties of deng entropy in the theory of evidence. Chaos Solitons Fractals 95 (2017), 195–199, 10.1016/j.chaos.2016.12.024.
31. Buntine, W., Niblett, T., A further comparison of splitting rules for decision-tree induction. Mach. Learn. 8:1 (1992), 75–85, 10.1023/A:1022686419106.
32. Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J., Classification and Regression Trees. Statistics/Probability Series, 1984, Wadsworth Publishing Company, Belmont, California, U.S.A.
33. Witten, I.H., Frank, E., Data Mining: Practical Machine Learning Tools and Techniques. 2nd Morgan Kaufmann Series in Data Management Systems, 2005, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
34. Raileanu, L.E., Stoffel, K., Theoretical comparison between the gini index and information gain criteria. Ann. Math. Artif. Intell. 41:1 (2004), 77–93, 10.1023/B:AMAI.0000018580.96245.c6.
35. Kulkarni, V.Y., Petare, M., Sinha, P.K., Analyzing random forest classifier with different split measures. Proceedings of the Second International Conference on Soft Computing for Problem Solving (SocProS 2012), 2012, Springer India, New Delhi, 691–699, 10.1007/978-81-322-1602-5_74.
36. M. Lichman, UCI machine learning repository, 2013.
37. Dietterich, T.G., An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization. Mach. Learn. 40:2 (2000), 139–157, 10.1023/A:1007607513941.
38. Demšar, J., Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7 (2006), 1–30.
39. Alcalá-Fdez, J., Sánchez, L., García, S., del Jesus, M., Ventura, S., Garrell, J., Otero, J., Romero, C., Bacardit, J., Rivas, V., Fernández, J., Herrera, F., Keel: a software tool to assess evolutionary algorithms for data mining problems. Soft Comput. 13:3 (2009), 307–318, 10.1007/s00500-008-0323-y.
40. Friedman, M., The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J. Am. Stat. Assoc. 32 (1937), 675–701.
41. Friedman, M., A comparison of alternative tests of significance for the problem of m rankings. Ann. Math. Stat. 11:1 (1940), 86–92, 10.1214/aoms/1177731944.
42. Nemenyi, P., Distribution-free multiple comparisons, 1963, Princeton University Doctoral dissertation New Jersey, USA.
43. Sáez, J.A., Luengo, J., Herrera, F., Evaluating the classifier behavior with noisy data considering performance and robustness: the equalized loss of accuracy measure. Neurocomputing 176 (2014), 26–35 Recent Advancements in Hybrid Artificial Intelligence Systems and its Application to Real-World Problems, selected papers from the {HAIS} 2013 conference doi: 10.1016/j.neucom.2014.11.086.