Hellinger distance decision trees are robust and skew-insensitive

Data Mining and Knowledge Discovery

Volumn 24, Issue 1, 2012, Pages 136-158

  Cieslak, David A ^a   Hoens, T Ryan ^a   Chawla, Nitesh V ^a   Kegelmeyer, W Philip ^a  

^a UNIVERSITY OF NOTRE DAME   (United States)

Author keywords

Decision tree; Hellinger distance; Imbalanced data

Indexed keywords

COMMON PROBLEMS; DATA SETS; DECISION TREE TECHNIQUES; EMPIRICAL EVALUATIONS; ENSEMBLE METHODS; GAIN RATIO; HELLINGER DISTANCE; IMBALANCED DATA; PARAMETER SELECTION; ROBUST TESTS; SAMPLING METHOD; SAMPLING TECHNIQUE; SPLITTING CRITERION; STATISTICAL SIGNIFICANCE;

TREES (MATHEMATICS);

DECISION TREES;

EID: 84856621489     PISSN: 13845810     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10618-011-0222-1     Document Type: Article

References (41)

1. E Alpaydin 1999 Combined 5 × 2cv F test for comparing supervised classification learning algorithms Neural Comput 11 8 1885 1892 10.1162/089976699300016007
2. Asuncion A, Newman D (2007) UCI machine learning repository. http://www.ics.uci.edu/~mlearn/MLRepository.html
3. R Banfield LO Hall KW Bowyer WP Kegelmeyer 2007 A comparison of decision tree ensemble creation techniques IEEE Trans Pattern Anal Mach Intell 29 1 832 844 10.1109/TPAMI.2007.250609
4. G Batista R Prati M Monard 2004 A study of the behavior of several methods for balancing machine learning training data SIGKDD Explor 6 1 20 29 10.1145/1007730.1007735
5. L Breiman 1996 Bagging predictors Mach Learn 24 2 123 140 0858.68080 1425957
6. L Breiman 1998 Rejoinder to the paper 'Arcing Classifiers' by Leo Breiman Ann Stat 26 2 841 849 1635406
7. L Breiman 2001 Random forests Mach Learn 45 1 5 32 10.1023/A: 1010933404324 1007.68152
8. Breiman L, Friedman J, Stone CJ, Olshen R (1984) Classification and regression trees. Chapman and Hall, Boca Raton
9. Chang C, Lin C (2001) LIBSVM: a library for support vector machines. software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm. Accessed June 2011
10. Chawla NV (2003) C4.5 and imbalanced data sets: investigating the effect of sampling method, probabilistic estimate, and decision tree structure. In: ICML workshop on learning from imbalanced data sets II. Washington, DC, USA, pp 1-8
11. NV Chawla KW Bowyer LO Hall WP Kegelmeyer 2002 SMOTE: synthetic minority over-sampling technique J Artif Intell Res 16 321 357 0994.68128
12. NV Chawla N Japkowicz A Kolcz 2004 Editorial: learning from imbalanced datasets SIGKDD Explor 6 1 16 10.1145/1007730.1007733
13. NV Chawla DA Cieslak LO Hall A Joshi 2008 Automatically countering imbalance and its empirical relationship to cost Data Min Knowl Discov 17 2 225 252 10.1007/s10618-008-0087-0 2434765
14. Cieslak DA, Chawla NV (2008a) Learning decision trees for unbalanced data. In: European conference on machine learning (ECML). Antwerp, Belgium, pp 241-256
15. Cieslak DA, Chawla NV (2008b) Analyzing classifier performance on imbalanced datasets when training and testing distributions differ. In: Pacific-Asia conference on knowledge discovery and data mining (PAKDD). Osaka, Japan, pp 519-526
16. J Demšar 2006 Statistical comparisons of classifiers over multiple data sets J Mach Learn Res 7 1 30 1222.68184 2274360
17. TG Dietterich 1998 Approximate statistical tests for comparing supervised classiffication learning algorithms Neural Comput 10 7 1895 1923 10.1162/089976698300017197
18. T Dietterich 2000 An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization Mach Learn 40 2 139 157 10.1023/A:1007607513941
19. Dietterich T, Kearns M, Mansour Y (1996) Applying the weak learning framework to understand and improve C4.5. In: Proceeings of the 13th international conference on machine learning. Morgan Kaufmann, Bari, Italy, pp 96-104
20. Drummond C, Holte R (2000) Exploiting the cost (in)sensitivity of decision tree splitting criteria. In: International conference on machine learning (ICML). Stanford University, California, USA, pp 239-246
21. Drummond C, Holte R (2003) C4.5, Class imbalance, and cost sensitivity: why under-sampling beats over-sampling. In: ICML workshop on learning from imbalanced datasets II. Washington, DC, USA, pp 1-8
22. Flach PA (2003) The geometry of ROC space: understanding machine learning metrics through ROC isometrics. In: International conference on machine learning (ICML). Washington, DC, USA, pp 194-201
23. Freund Y, Schapire R (1996) Experiments with a new boosting algorithm. In: Proceedings of the 13th national conference on machine learning. Bari, Italy, pp 148-156
24. M Friedman 1940 A comparison of alternative tests of significance for the problem of m rankings Ann Math Stat 11 1 86 92 10.1214/aoms/1177731944 0063.01455
25. Halmos P (1950) Measure theory. Van Nostrand and Co., Princeton
26. D Hand R Till 2001 A simple generalisation of the area under the ROC curve for multiple class classification problems Mach Learn 45 171 186 10.1023/A:1010920819831 1007.68180
27. Hido S, Kashima H (2008) Roughly balanced bagging for imbalanced data. In: SIAM international conference on data mining (SDM). Atlanta, Georgia, USA, pp 143-152
28. T Ho 1998 The random subspace method for constructing decision forests IEEE Trans Pattern Anal Mach Intell 20 8 832 844 10.1109/34.709601
29. S Holm 1979 A simple sequentially rejective multiple test procedure Scand J Stat 6 2 65 70 0402.62058 538597
30. Japkowicz N (2000) Class imbalance problem: significance & strategies. In: International conference on artificial intelligence (ICAI). Las Vegas, Nevada, USA, pp 111-117
31. T Kailath 1967 The divergence and Bhattacharyya distance measures in signal selection IEEE Trans Commun 15 1 52 60 10.1109/TCOM.1967.1089532
32. Kubat M, Matwin S (1997) Addressing the curse of imbalanced training sets: one-sided selection. In: International conference on machine learning (ICML). Nashville, Tennessee, USA, pp 179-186
33. Nguyen X, Wainwright MJ, Jordan MI (2007) Estimating divergence functionals and the likelihood ratio by penalized convex risk minimization. In: Advances in neural information processing systems (NIPS). Vancouver, BC, Canada, pp 1-8
34. F Provost P Domingos 2003 Tree induction for probability-based ranking Mach Learn 52 3 199 215 10.1023/A:1024099825458 1039.68105
35. R Quinlan 1986 Induction of decision trees Mach Learn 1 81 106
36. C Rao 1995 A review of canonical coordinates and an alternative to correspondence analysis using Hellinger distance Questiio (Quaderns d'Estadistica i Investig Oper) 19 23 63 1167.62421
37. R Schapire Y Singer 1999 Improved boosting algorithms using confidence-rated predictions Mach Learn 37 297 336 10.1023/A:1007614523901 0945.68194
38. Van Hulse J, Khoshgoftaar T, Napolitano A (2007) Experimental perspectives on learning from imbalanced data. In: International conference on machine learning (ICML). Corvalis, Oregon, USA, pp 935-942
39. Vilalta R, Oblinger D (2000) A quantification of distance-bias between evaluation metrics in classification. In: International conference on machine learning (ICML). Stanford University, California, USA, pp 1087-1094
40. J Wu H Xiong J Chen 2010 Cog: local decomposition for rare class analysis Data Min Knowl Discov 20 191 220 10.1007/s10618-009-0146-1 2596456
41. Zadrozny B, Elkan C (2001) Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers. In: Proceedings of the 18th international conference on machine learning. Morgan Kaufmann, San Francisco, CA, USA, 2001, pp. 609-616