Handling over-fitting in test cost-sensitive decision tree learning by feature selection, smoothing and pruning

Journal of Systems and Software

Volumn 83, Issue 7, 2010, Pages 1137-1147

  Wang, Tao ^a   Qin, Zhenxing ^a   Jin, Zhi ^b   Zhang, Shichao ^c  

^a UNIVERSITY OF TECHNOLOGY SYDNEY   (Australia)

^b PEKING UNIVERSITY   (China)

^c ZHEJIANG NORMAL UNIVERSITY   (China)

Author keywords

Classification; Cost sensitive learning; Over fitting

Indexed keywords

CLASS PROBABILITIES; COMPETING ALGORITHMS; COST RATIO; COST-SENSITIVE LEARNING; DATA SETS; DECISION TREE LEARNING; EFFICIENT STRATEGY; FEATURE SELECTION; MISCLASSIFICATION COSTS; OPTIMAL MODEL; OVERFITTING; REAL WORLD DATA; REAL-WORLD APPLICATION; TEST COST; TOTAL COSTS; TRAINING DATA;

COSTS; DECISION TREES; FEATURE EXTRACTION; OPTIMIZATION;

LEARNING ALGORITHMS;

EID: 77953324515     PISSN: 01641212     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jss.2010.01.002     Document Type: Article

References (29)

1. Bradley, A.P., Lovell, B.C., 1995. Cost-sensitive decision tree pruning: use of the ROC curve. In: Eighth Australian Joint Conference on Artificial Intelligence, vol. 1, Canbarra, Australia, p. 1.
2. Bradford, J.P., Kunz, C., Kohavi, R., Brunk, C., Brodley, C.E., 1998. Pruning decision trees with misclassification costs. In: Proceedings of 10th European Conference on Machine Learning, pp. 131-136.
3. Breiman L., Friedman J.H., Olshen R.A., and Stone C.J. Classification and Regression Trees vol. 1 (1984), Wadsworth, Belmont, CA
4. Domingos, P., 1999. MetaCost: a general method for making classifiers cost-sensitive. In: Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 155-164.
5. Du J., Cai Z., and Ling C.X. Cost-sensitive decision trees with pre-pruning. Lecture Notes in Computer Science. 4509 (2007) 171
6. Elkan, C., 2001. The foundations of cost-sensitive learning. In: Nebel, Bernhard (Ed.), Proceeding of the Seventeenth International Joint Conference of Artificial Intelligence, Morgan Kaufmann, 4-10 August 2001, Seattle, pp. 973-978.
7. Esposito F., Malerba D., and Semeraro G. A comparative analysis of methods for pruning decision trees. IEEE Transactions on Pattern Analysis and Machine Intelligence (1997)
8. Greiner R., Grove A.J., and Roth D. Learning cost-sensitive active classifiers. Artificial Intelligence 139 2 (2002) 137-174
9. Kohavi R., and John G.H. Wrappers for feature subset selection. Artificial intelligence 97 1-2 (1997) 273-324
10. Li, J., Li, X., Yao, X., 2005. Cost-sensitive classification with genetic programming. In: Evolutionary Computation, 2005. The 2005 IEEE Congress on, vol. 3.
11. Ling, C.X., Yang, Q., Wang, J., Zhang, S., 2004. Decision trees with minimal costs. In: Brodley, Carla E. (Ed.), Proceeding of the Twenty-first International Conference on Machine Learning, vol. 69, ACM Press, 4-8 July 2004, Banff, Alberta, pp. 69-76.
12. Margineantu D.D. Methods for cost-sensitive learning (2001), Oregon State University
13. Mingers J. Expert systems-rule induction with statistical data. Journal of the Operational Research Society (1987) 39-47
14. Niblett, T., Bratko, I., 1986. Learning decision rules in noisy domains. In: Developments in Expert Systems. Cambridge University Press.
15. Niculescu-Mizil, A., Caruana, R., 2005. Predicting good probabilities with supervised learning. Association for Computing Machinery Inc., One Astor Plaza, 1515 Broadway, New York, NY 10036-5701, USA.
16. Platt J.C. Probabilities for SV machines. Advances in Neural Information Processing Systems (1999) 61-74
17. Provost F., and Domingos P. Tree induction for probability-based ranking. Machine Learning 52 (2003) 199-215
18. Quinlan J.R. C4.5: Programs for Machine Learning (1993), Morgan Kaufmann, San Mateo
19. Quinlan J.R. Induction of decision trees. Machine Learning 1 1 (1986) 81-106
20. Quinlan J.R. Generating production rules from decision trees 30107 (1987)
21. Turney, P., 2000. Types of cost in inductive concept learning. In: Workshop on Cost-Sensitive Learning at the Seventeenth International Conference on Machine Learning, 1511pp.
22. Turney P.D. Cost-sensitive classification: empirical evaluation of a hybrid genetic decision tree induction algorithm. Journal of Artificial Intelligence Research 2 1995 (1995) 369-409
23. Wang, T., Qin, Z., Zhang, S., Cost-sensitive learning - a survey. International Journal of Data Warehousing and Mining (IJDWM), accepted for publication.
24. Witten I.H., and Frank E. Data Mining: Practical Machine Learning Techniques with Java Implementations. second ed. (2000), Morgan Kaufmann Publishers
25. Zadrozny, B., Elkan, C., 2001a. Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In: Proceedings of the 18th International Conference on Machine Learning, pp. 609-616.
26. Zadrozny, B., Elkan, C., 2001b. Learning and making decisions when costs and probabilities are both unknown. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM Press, San Francisco, CA, pp. 204-213.
27. Zadrozny, B., Elkan, C., 2002. Transforming classifier scores into accurate multiclass probability estimates. ACM New York, NY, USA, pp. 694-699.
28. Zadrozny, B., 2005. One-benefit learning: cost-sensitive learning with restricted cost information. In: Proceedings of the First International Workshop on Utility-Based Data Mining, ACM Press, Chicago, IL, pp. 53-58.
29. Zubek, V.B., Dietterich, T.G., 2002. Pruning Improves Heuristic Search for Cost-Sensitive Learning. In: Proceedings of the Nineteenth International Conference on Machine Learning, pp. 27-34.