Evaluation of classifiers for an uneven class distribution problem

Applied Artificial Intelligence

Volumn 20, Issue 5, 2006, Pages 381-417

  Daskalaki, Sophia ^a   Kopanas, Ioannis ^a   Avouris, Nikolaos ^a  

^a UNIVERSITY OF PATRAS   (Greece)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; CLASSIFIERS; CUSTOMER SATISFACTION; DATA MINING; DATA REDUCTION; EVALUATION; PROBLEM SOLVING; SET THEORY;

ALTERNATIVE METHODOLOGIES; CLASS DISTRIBUTION PROBLEMS; CUSTOMER INSOLVENCY PROBLEMS; DECISION CRITERIA;

CLASSIFICATION (OF INFORMATION);

EID: 33646421421     PISSN: 08839514     EISSN: 10876545     Source Type: Journal    
DOI: 10.1080/08839510500313653     Document Type: Review

References (43)

1. Abbott, D. W. 1999. Combining models to improve classifier accuracy and robustness. In: Proceedings of the International Conference an Information Fusion (Fussion 99), Sunnyvale, California, USA, July 1999, Volume 1, pages 289-295.
2. Akbani, R., S. Kwek, and N. Japkowicz. 2004. Applying support vector machines to imbalanced datasets. In: ECML 2004, eds. J.-F. Boulicaut et al, LNAI 3201, pages 39-50. Springer-Verlag, Berlin Heidleberg.
3. Bauer, E. and R. Kohavi. 1999. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning 36:105-139.
4. Breiman, L. 1996. Bagging predictors. Machine Learning 24:123-140.
5. le Cessie, S. and J. C. van Houwelingen. 1992. Ridge estimators in logistic regression. Applied Statistics 41(1):191-201.
6. Chan, P. K. and S. J. Stolfo. 1997. On the accuracy of meta-learning for scalable data mining. J. of Intelligent Information Systems 8:5-28.
7. Chan, P. K. and S. J. Stolfo. 1998a. Toward scalable learning with non-uniform class and cost distributions: A case study in credit card fraud detection. In: Proceedings of the. Fourth Intl. Conf. On Knowledge Discovery and Data, Mining, September 1998, pages 164-168. AAAI Press.
8. Chan, P. K. and S. J. Stolfo. 1998b. Learning with non-uniform class and cost distributions: Effects and a multi-classifier approach. In: Work Notes KDD-98 Workshop on Distributed Data Mining, August 1998, pages 1-9.
9. Chan, P. K., F. Wei, A. Prodromidis, and S.J. Stolfo. 1999. Distributed data mining in credit card fraud detection. IEEE Intelligent Systems 14(6):67-74.
10. Chawla, N. V. 2003. C4.5 and imbalanced data sets: Investigating the effect of sampling method, probabilistic estimate and decision tree structure. Workshop on Learning from Imbalanced Datasets II, International Conference on Machine Learning, Washington, D.C.
11. Chawla, N. V., K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer. 2002. SMOTE: Synthetic minority over-sampling technique, Journal of Artificial Intelligence Research 16:321-357.
12. Daskalaki, S., I. Kopanas, M. Goudara, and N. Avouris. 2003. Data mining for decision support on customer insolvency in telecommunications business. European Journal of Operational Research 145(2):239-255.
13. Dietterich, T. G. 2000. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning 40:139-157.
14. Domingos, P. 1999. Metacost: A general method for making classifiers cost-sensitive. In: Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Diego, CA, USA, pages 155-164.
15. Drummond, C. and R. C. Holte. 2000a. Exploiting the cost (in)sensitivity of decision tree splitting criteria. In: Proceedings of the Seventeenth International Conference on Machine Learning, pages 239-246. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
16. Drummond, C. and R. C. Holte. 2000b. Explicitly representing expected cost: An alternative to ROC representation. In: Proceedings of the Sixth ACM SIGKDD International Conference an Knowledge Discovery and Data Mining, Boston, MA, USA, pages 198-207.
17. Drummond, C. and R. C. Holte. 2003. C4.5, class imbalance, and cost sensitivity: Why under-sampling beats over-sampling. Workshop on Learning from Imbalanced Datasets II, International Conference on Machine Learning, Washington, D.C., USA.
18. Drummond, C. and R. C. Holte. 2004. What ROC, curves can't do (and cost curves can). In: Proceedings of the ROC Analysis in Artificial Intelligence, First International Workshop, pages 19-26, Valencia, Spain, August 2004.
19. Elkan, C. 2001. The foundations of cost-sensitive learning. In: Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, Seattle, WA, USA, August 2001, pages 973-978. Morgan Kaufmann Publishers Inc.
20. Estabrooks, A., T. Jo, and N. Japkowicz. 2004. A multiple resampling method for learning from imbalances data sets. Computational Intelligence 20 (1): 18-36.
21. Ezawa, K.J. and S. W. Norton. 1996. Constructing Bayesian networks to predict uncollectible telecommunications accounts. IEEE Expert/Intelligent Systems & Their Applications 11(5):45-51.
22. Ezawa, K. J., M. Singh, and S. W. Norton. 1996. Learning goal oriented bayesian networks for telecommunications management. In: Proceedings of the Thirteenth International Conference on Machine Learning, Bari, Italy, July 1996, pages 139-147. Morgan Kaufmann Publishers Inc.
23. Fawcett, T. and F. Provost. 1997. Adaptive fraud detection. Data Mining and Knowledge Discovery 1:291-316.
24. Gur Ali, F. O. and W. A. Wallace. 1997. Bridging the gap between business objectives and parameters of data mining algorithms. Decision Support Systems 21:3-15.
25. Japkowicz, N. and S. Stephen. 2002. The class imbalance problem: A systematic study. Intelligent Data Analysis Journal 6(5): 429-449.
26. Kohavi, R. 1995. A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proceedings of the 14th International Joint Conference on Artificial Intelligence, Montreal, Quebec, Canada, August 95, pages 1137-1145. Morgan Kaufmann Publishers Inc.
27. Kopanas, I., N. M. Avouris, and S. Daskalaki. 2002. The role of knowledge modeling in a large scale data mining project. In: Methods and Applications of Artificial Intelligence LNAI 2308, eds. I. P. Vlahavas and C. D. Spyropoulos, 288-299. Berlin: Springer-Verlag.
28. Kubat, M., R. Holte, and S. Matwin. 1998. Machine learning for the detection of oil spills in satellite radar images. Machine Learning 30:195-215.
29. Kubat, M. and S. Matwin. 1997. Addressing the curse of imbalanced training sets: One-sided selection. In: Proceedings of the 14th International Conference on Machine Learning, Nashville, TN, USA, pages 179-186. Morgan Kaufmann.
30. Laurikkala, J. 2001. Improving identification of difficult small classes by balancing class distribution. In: Proceedings of the 8th Conference on AI in Medicine in Europe: Artificial Intelligence in Medicine, eds. S. Quaglini, P. Barahona, and S. Andreassen, LNAI 2101, 63-66. Springer-Verlag, London, UK.
31. Lewis, D. D. and W. Gale. 1994. A sequential algorithm for training text classifiers. In: Proceedings of the Seventh Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval, pages 3-12. Springer-Verlag.
32. Platt, J. 1999. Fast training of support vector machines using sequential minimal optimization. In: Advances in Kernel Methods - Support Verlor Learning, eds. B. Scholkopf, C. Burges, and A. Smola, pages 185-208, The MIT Press, Cambridge, MA, USA.
33. Provost, F. and T. Fawcett. 1997. Analysis and visualization of classifier performance: Comparison under imprecise class and cost distributions. In: Proceedings of the Third International Conference on Knowledge Discovery and Data Mining, pages 43-48. Menlo Park, CA: AAAI Press.
34. Provost, F. and T. Fawcett. 2001. Robust classification for imprecise environments. Machine Learning 42:203-231.
35. Provost, F., T. Fawcett, and R. Kohavi. 1998. The case against accuracy estimation for comparing induction algorithms. In: Proceedings of the Fifteenth International Conference on Machine Learning (LMLC-98), pages 43-48. Morgan Kaufmann.
36. Quinlan, J. 1992. C4.5: Programs for Machine Learning. San Francisco, CA: Morgan Kaufmann.
37. Riddle, P., R. Segal, and O. Etzioni. 1994. Representation design and brute-force induction in a Boeing manufacturing domain. Applied Artificial Intelligence 8:125-147.
38. Seewald, A. K. 2003. Towards a theoretical framework for ensemble classification. In: Proceedings of the 18th Int. Joint Conference on Artificial Intelligence (IJCAI-03), Acapulco, Mexico, August 2003, pages 1443-1444. Morgan Kaufmann.
39. Weiss, C. and F. Provost. 2001. The Effect, of Class Distribution on Classifer Learning. Technical Report ML-TR-43, Department of Computer Science, Rutgers University.
40. Weiss, G. and F. Provost. 2003. Learning when training data are costly: The effect of class distribution on tree induction. Journal of Artifcial Intelligence Research 19:315-354.
41. Wolpert, D. 1992. Stacked generalization. Neural Networks 5:241-259.
42. Woods, K., W. P. Kegelmeyer, Jr., and K. Bowyer. 1997. Combination of multiple classifiers using local accuracy estimates. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(4):405-410.
43. Zadrozny, B. and C. Elkan. 2001. Learning and making decisions when costs and probabilities are both unknown. In: Proceedings of the. Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, pages 204-213, ACM Press