A comparison of different off-centered entropies to deal with class imbalance for decision trees

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 5012 LNAI, Issue , 2008, Pages 634-643

  Lenca, Philippe ^a   Lallich, Stéphane ^b   Do, Thanh Nghi ^c   Pham, Nguyen Khang ^d  

^a ECOLE DES MINES DE NANTES   (France)

^b UNIVERSITÉ DE LYON   (France)

^c INRIA   (France)

^d CAMPUS DE BEAULIEU   (France)

Author keywords

Decision trees; Imbalance class; Off centered entropies; Shannon entropy

Indexed keywords

DECISION TREES; FORESTRY; TREES (MATHEMATICS);

C4.5 DECISION TREES; CLASS IMBALANCE PROBLEMS; CLASS PROBABILITIES; IMBALANCE CLASS; IMBALANCED DATA-SETS; PERFORMANCE OF CLASSIFIER; PRIORI DISTRIBUTION; SHANNON ENTROPY;

DATA MINING;

EID: 44649183291     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-540-68125-0_59     Document Type: Conference Paper

References (37)

1. Japkowicz, N. (ed.): Learning from Imbalanced Data, Sets/AAAI (2000)
2. Chawla, N., Japkowicz, N., Kolcz, A. (eds.): Learning from Imbalanced Data Sets/ICML (2003)
3. Chawla, N., Japkowicz, N., Kolcz, A. (eds.): Special Issue on Class Imbalances. SIGKDD Explorations, vol. 6 (2004)
4. Yang, Q., Wu, X.: 10 challenging problems in data mining research. International Journal of Information Technology & Decision Making 5(4), 597-604 (2006)
5. Japkowicz, N.: The class imbalance problem: Significance and strategies. In: IC-AI, pp. 111-117 (2000)
6. Japkowicz, N., Stephen, S.: The class imbalance problem: A systematic study. Intelligent Data Analysis 6(5), 429-450 (2002)
7. Visa, S., Ralescu, A.: Issues in mining imbalanced data sets - A review paper. In: Midwest AICS Conf., pp. 67-73 (2005)
8. Weiss, G.M., Provost, F.: The effect of class distribution on classifier learning. TR ML-TR 43, Department of Computer Science, Rutgers University (2001)
9. Weiss, G.M., Provost, F.: Learning when training data are costly: The effect of class distribution on tree induction. J. of Art. Int. Research 19, 315-354 (2003)
10. Liu, A., Ghosh, J., Martin, C.: Generative oversampling for mining imbalanced datasets. In: DMIN, pp. 66-72 (2007)
11. Kubat, M., Matwin, S.: Addressing the curse of imbalanced data sets: One-sided sampling. In: ICML, pp. 179-186 (1997)
12. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)
13. Drummond, C., Holte, R.: C4.5, class imbalance, and cost sensitivity: Why under-sampling beats over-sampling. In: Learning from Imbalanced Data Sets/ICML (2003)
14. Domingos, P.: Metacost: A general method for making classifiers cost sensitive. In: KDD, pp. 155-164 (1999)
15. Zhou, Z.H., Liu, X.Y.: On multi-class cost-sensitive learning. In: AAAI, pp. 567-572 (2006)
16. Weiss, G.M., McCarthy, K., Zabar, B.: Cost-sensitive learning vs. sampling: Which is best for handling unbalanced classes with unequal error costs? In: DMIN, pp. 35-41 (2007)
17. Ling, C.X., Yang, Q., Wang, J., Zhang, S.: Decision trees with minimal costs. In: ICML (2004)
18. Du, J., Cai, Z., Ling, C.X.: Cost-sensitive decision trees with pre-pruning. In: Kobti, Z., Wu, D. (eds.) Canadian AI 2007. LNCS (LNAI), vol. 4509, pp. 171-179. Springer, Heidelberg (2007)
19. Chawla, N.: C4.5 and imbalanced datasets: Investigating the effect of sampling method, probalistic estimate, and decision tree structure. In: Learning from Imbalanced Data Sets/ICML (2003)
20. Shannon, C.E.: A mathematical theory of communication. Bell System Technological Journal (27), 379-423, 623-656 (1948)
21. Wehenkel, L.: On uncertainty measures used for decision tree induction. In: IPMU, pp. 413-418 (1996)
22. Loh, W.Y., Shih, Y.S.: Split selection methods for classification trees. Statistica Sinica 7, 815-840 (1997)
23. Quinlan, J.R.: Induction of decision trees. Machine Learning 1(1), 81-106 (1986)
24. Theil, H.: On the estimation of relationships involving qualitative variables. American Journal of Sociology (76), 103-154 (1970)
25. Kvalseth, T.O.: Entropy and correlation: some comments. IEEE Trans. on Systems, Man and Cybernetics 17(3), 517-519 (1987)
26. Lallich, S., Vaillant, B., Lenca, P.: Parametrised measures for the evaluation of association rule interestingness. In: ASMDA, pp. 220-229 (2005)
27. Lallich, S., Vaillant, B., Lenca, P.: A probabilistic framework towards the parameterization of association rule interestingness measures. Methodology and Computing in Applied Probability 9, 447-463 (2007)
28. Zighed, D.A., Rakotomalala, R.: Graphes d'Induction - Apprentissage et Data Mining. Hermes (2000)
29. Lallich, S., Vaillant, B., Lenca, P.: Construction d'une entropie décentrée pour l'apprentissage supervisé. In: QDC/EGC 2007, pp. 45-54 (2007)
30. Lallich, S., Lenca, P., Vaillant, B.: Construction of an off-centered entropy for supervised learning. In: ASMDA, p. 8 (2007)
31. Goodman, L.A., Kruskal, W.H.: Measures of association for cross classifications, i. JASA 1(49), 732-764 (1954)
32. Lallich, S.: Mesure et validation en extraction des connaissances à partir des données. In: Habilitation à Diriger des Recherches, Université Lyon 2, France (2002) 33. Zighed, D.A., Marcellin, S., Ritschard, G.: Mesure d'entropie asymétrique et consistante. In: EGC, pp. 81-86 (2007)
33. Marcellin, S., Zighed, D.A., Ritschard, G.: An asymmetric entropy measure for decision trees. In: IPMU, pp. 1292-1299 (2006)
34. Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)
35. Michie, D., Spiegelhalter, D.J., Taylor, C.C. (eds.): Machine Learning, Neural and Statistical Classification. Ellis Horwood (1994)
36. Jinyan, L., Huiqing, L.: Kent ridge bio-medical data set repository. Technical report (2002)
37. Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Wadsworth International (1984)