An empirical comparison of cost-sensitive decision tree induction algorithms

Expert Systems

Volumn 28, Issue 3, 2011, Pages 227-268

  Lomax, Susan ^a   Vadera, Sunil ^a  

^a UNIVERSITY OF SALFORD   (United Kingdom)

Author keywords

cost sensitive learning; data mining; decision trees

Indexed keywords

COST-SENSITIVE; COST-SENSITIVE LEARNING; DECISION TREE INDUCTION; DECISION-TREE ALGORITHM; EMPIRICAL COMPARISON; EMPIRICAL EVALUATIONS; LOOK-AHEAD; MISCLASSIFICATION COSTS; PAIR-WISE COMPARISON; REPRESENTATIVE SAMPLE;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COSTS; DECISION SUPPORT SYSTEMS; DECISION TREES; PLANT EXTRACTS; TREES (MATHEMATICS);

DATA MINING;

EID: 79960260907     PISSN: 02664720     EISSN: 14680394     Source Type: Journal    
DOI: 10.1111/j.1468-0394.2010.00573.x     Document Type: Review

References (40)

1. Abe, N., J. Langford, and, B. Zadrozny, (2004) An iterative method for multi-class cost-sensitive learning conference on knowledge discovery in data, in Proceedings of the 2004 ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Session: Research Track Papers, W. Kim, R. Kohavi, J. Gehrke, and, W. DuMouchel, (eds), NY, USA: ACM Press New York, 3-11. (Pubitemid 40114910)
2. Boddy, M., and, T.L. Dean, (1994) Deliberation scheduling for problem solving in time constrained environments, Artificial Intelligence, 67, 245-285.
3. Bradford, J.P., C. Kunz, R. Kohavi, C. Brunk, and, C.E. Brodley, (1998a) Pruning decision trees with Misclassification Costs. 10th European Conference on Machine Learning (ECML-98), 21-23 April 1998, Chemnitz, Germany, pp. 131-136. (Pubitemid 128067177)
4. Bradford, J.P., C. Kunz, R. Kohavi, C. Brunk, and, C.E. Brodley, (1998b) Pruning decision trees with misclassification costs. Available at (accessed 12 November 2009).
5. Breiman, L., J.H. Friedman, R.A. Olsen, and, C.J. Stone, (1984) Classification and Regression Trees, Belmont: Wadsworth.
6. Domingos, P., (1999) MetaCost: a general method for making classifiers cost-sensitive, in Proceedings of the 5th International Conference on Knowledge Discovery and Data Mining, NY, USA: ACM Press New York, 155-164.
7. Draper, B.A., C.E. Brodley, and, P.E. Utgoff, (1994) Goal-directed classification using linear machine decision trees, IEEE Transactions on Pattern Analysis and Machine Intelligence, 16, 888-893.
8. Esmeir, S., and, S. Markovitch, (2004) Lookahead-based algorithms for anytime induction of decision trees, in ACM International Conference Proceeding Series 21st International Conference on Machine learning Banff, Alberta, Canada, C.E. Brodley (ed.), New York, USA: ACM Press Article No. 33, 257-264. (Pubitemid 40290816)
9. Fan, W., S.J. Stolfo, J. Zhang, and, P.K. Chan, (1999) AdaCost: misclassification cost-sensitive boosting. 16th International Conference on Machine Learning, 27-30 June 1999, Bled, Slovenia, pp. 97-105.
10. Knoll, U., G. Nakhaeizadeh, and, B. Tausend, (1994) Cost-sensitive pruning of decision trees, in Proceedings of the 8th European Conference on Machine Learning ECML-94, Berlin-Germany: Springer-Verlag, 386.
11. Ling, C.X., Q. Yang, J. Wang, and, S. Zhang, (2004) Decision trees with minimal costs, ACM International Conference Proceeding Series 21st international conference on Machine learning Banff, Alberta, Canada, C.E. Brodley, (ed.), New York, USA: ACM Press Article No. 69, 4-8.
12. Margineantu, D., and, T. Dietterich, (2003) A wrapper method for cost-sensitive learning via stratification. Available at (accessed 3 March 2005).
13. Merler, S., C. Furlanello, B. Larcher, and, A. Sboner, (2003) Automatic model selection in cost-sensitive boosting, Information Fusion, 4, 3-10. (Pubitemid 36210677)
14. Murthy, S.K., and, S. Salzberg, (1995) Lookahead and pathology in decision tree induction, in Proceedings of the 14th International Joint Conference on Artificial Intelligence, C.S. Mellish, (ed.), San Francisco, USA: Morgan Kaufmann. Available at (accessed 22 October 2009).
15. Norton, S.W., (1989) Generating better decision trees. Proceedings of the Eleventh International Joint Conference on Artificial Intelligence IJCAI-89, 20-26 August 1989, Detroit, Michigan, USA, 800-805.
16. Núñez, M., (1991) The Use of Background Knowledge in Decision Tree Induction, Machine Learning 6, Boston, USA: Kluwer Academic Publishers, 231-250.
17. Pazzani, M., C. Merz, P. Murphy, K. Ali, T. Hume, and, C. Brunk, (1994) Reducing misclassification costs, Proceedings of the 11th International Conference on Machine Learning, San Francisco, USA: Morgan Kaufmann, 217-225.
18. Quinlan, J.R., (1979) Discovering rules by induction from large collections of examples in Expert Systems in the micro electronic age, D. Michie (ed.), Edinburgh: Edinburgh University Press, 168-201.
19. Quinlan, J.R., (1986) Induction of Decision Trees, Machine Learning 1, Boston, USA: Kluwer Academic Publishers, 81-106.
20. Quinlan, J.R., (1987) Simplifying decision trees, International Journal of Man-Machine Studies, 27, 221-234. (Pubitemid 18089906)
21. Quinlan, J.R., P.J. Compton, K.A. Horn, and, L. Lazarus, (1987) Inductive knowledge acquisition: a case study Based on the Proceedings of the Second Australian Conference on Applications of Expert Systems, Chapter 9, J. Ross Quinlan, (ed.), London: Turing Institute Press, Glasgow, with Addison-Wesley Publishing Company Inc. Boston, MA, USA, 137-156.
22. Sarker, R., H. Abbass, and, C. Newton, (eds.) (2002) Introducing Data Mining and Knowledge Discovery Heuristics and Optimization for Knowledge Discovery (Chapter 1), Toronto, Canada: Idea Group Publishing.
23. Schapire, R., and, Y. Singer, (1998) Improved boosting algorithms using confidence-rated predictions. In Proceedings of the Eleventh Annual Conference on Computational Learning Theory, 24-26 July 1998, University of Wisconsin, USA, pp. 80-91.
24. Tan, M., (1993) Cost-sensitive learning of classification knowledge and its applications in robotics, Machine Learning, 13, 7-33.
25. Tan, M., and, J. Schlimmer, (1989) Cost-sensitive concept learning of sensor use in approach and recognition. Proceedings of the Sixth International Workshop on Machine Learning. ML-89, 26-27 June 1989, Ithaca, New York, pp. 392-395.
26. Tan, M., and, J. Schlimmer, (1990) CSL: a cost-sensitive learning system for sensing and grasping objects. IEEE International Conference on Robotics and Automation, 13-18 May 1990, Cincinnati, Ohio, USA.
27. Ting, K.M., (1998) Inducing cost-sensitive decision trees via instance weighting, in Proceedings of the 2nd European Symposium on Principles of Data Mining and Knowledge Discovery, Berlin: Springer Verlag, 139-147. (Pubitemid 128145830)
28. Ting, K.M., (2000a) An empirical study of MetaCost using boosting algorithms, in Proceedings of ECML, R de Mantaras, and, E Plaza, (eds), Berlin: Springer-Verlag, 983-990.
29. Ting, K.M., (2000b) A comparative study of cost-sensitive boosting algorithms, in Proceedings of ICML-2000, M Kaufmann, (ed.), San Francisco, USA: Morgan Kaufmann, 983-990.
30. Ting, K.M., (2002) An instance-weighting method to induce cost-sensitive decision trees, IEEE Transactions on Knowledge and Data Engineering, 14, 659-665. (Pubitemid 34669622)
31. Ting, K.M., and, Z. Zheng, (1998a) Boosting cost-sensitive trees, in Proceedings of the 1st International Conference on Discovery Science, LNAI-1532, Berlin: Springer-Verlag, 244-255.
32. Ting, K.M., and, Z. Zheng, (1998b) Boosting Decision Trees For Cost-Sensitive Classifications, Machine Learning ECML-98, in 10th European Conference on Machine Learning, C. Nedellec, and, C. Rouveirol, (eds), Chemnitz, Germany: Springer, 190-195.
33. Turney, P.D., (1995) Cost-sensitive classification: empirical evaluation of a hybrid genetic decision tree induction algorithm, Journal of Artificial Intelligence Research, 2, 369-409.
34. thInt. Conference on Machine Learning, pp. 15-21.
35. Vadera, S., (2005) Inducing Safer oblique trees without costs, expert systems: the international journal of knowledge engineering and neural networks, The International Journal of Knowledge Engineering and Neural Networks, 22, 206-221. (Pubitemid 43906931)
36. Vadera, S., and, D. Ventura, (2001) A comparison of cost-sensitive decision tree learning algorithms. Second European Conference in Intelligent Management Systems in Operations, 3-4 July, K. Kobbacy and S. Vadera (eds), Operational Research Society, University of Salford, UK, Birmingham, UK: Operational Research Society, 79-86.
37. Winston, P.H., (1993) Artificial Intelligence, 3rd edn, USA: Addison Wesley.
38. Zadrozny, B., and, C. Elkan, (2001) Learning and making decisions when costs and probabilities are both unknown. Technical Report No CS2001-0664 January 2001. Proceedings of the 7th International Conference on Knowledge Discovery and Data Mining, pp. 203-213.
39. Zadrozny, B., J. Langford, and, N. Abe, (2003a) A Simple Method for Cost-Sensitive Learning, Technical Report RC22666, IBM, (accessed 22 October 2009).
40. Zadrozny, B., J. Langford, and, N. Abe, (2003b) Cost-Sensitive Learning By Cost-Proportionate Example Weighting, Third IEEE International Conference on Data Mining, 19-22 November, Melbourne, FL, 435.