A survey of cost-sensitive decision tree induction algorithms

ACM Computing Surveys

Volumn 45, Issue 2, 2013, Pages

  Lomax, Susan ^a   Vadera, Sunil ^a  

^a UNIVERSITY OF SALFORD   (United Kingdom)

Author keywords

cost sensitive learning; data mining; Decision tree learning

Indexed keywords

COST-SENSITIVE; COST-SENSITIVE LEARNING; DECISION TREE INDUCTION; DECISION TREE LEARNING; MISCLASSIFICATIONS; REFERENCE POINTS;

ALGORITHMS; COSTS; DECISION MAKING; DECISION TREES; SURVEYS;

DATA MINING;

EID: 84875204064     PISSN: 03600300     EISSN: 15577341     Source Type: Journal    
DOI: 10.1145/2431211.2431215     Document Type: Review

References (91)

1. ABE, N., ZADROZNY, B., AND LANGFORD, J. 2004. An iterative method for multi-class cost-sensitive learning. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '04). W. Kim, R. Kohavi, J, J. Gehrke, and W. DuMouchel, Eds., 3
2. AFIFI, A. A. AND CLARK, V. 1996. Computer-Aided Multivariate Analysis, 3rd ED. Chapman & Hall, London.
3. BAUER, E. AND KOHAVI, R. 1999. An empirical comparison of voting classification algorithms: Bagging, boosting and variants. Mach. Learn. 36, 1-2, 105-139
4. BRADFORD, J. P., KUNZ, C., KOHAVI, R., BRUNK, C., AND BRODLEY, C. E. 1998a. Pruning decision trees with misclassification costs. In Proceedings of the 10th European Conference on Machine Learning (ECML '98). 131-136 (Pubitemid 128067177)
5. BRADFORD, J. P., KUNZ, C., KOHAVI, R., BRUNK, C., AND BRODLEY, C. E. 1998b. Pruning decision trees with misclassification costs. http://robotics. stanford.edu/?ronnyk/prune-long.ps.gz
6. BREIMAN, L., FRIEDMAN J. H., OLSEN R. A., AND STONE C. J. 1984. Classification and Regression Trees. Chapman and Hall/CRC, London.
7. BREIMAN, L. 1996. Bagging predictors. Mach. Learn. 24, 2, 123-140 (Pubitemid 126724382)
8. DAVIS, J. V., JUNGWOO, H., AND ROSSBACH, C. J. 2006. Cost-Sensitive decision tree learning for forensic classification. In Proceedings of 17th European Conference on Machine Learning (ECML). Lecture Notes in Computer Science, vol. 4212, Springer, 622-629 (Pubitemid 44618870)
9. DOMINGOS, P. 1999. MetaCost: A general method for making classifiers cost-sensitive. In Proceedings of the 5th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 155-164
10. DONG, M. AND KOTHARI, R. 2001. LOOK-AHEAD BASED FUZZY DECISION TREE INDUCTION. IEEE Trans. Fuzzy Syst. 9, 3, 461-468 (Pubitemid 32636663)
11. DRAPER, B. A., BRODLEY, C. E., AND UTGOFF, P. E. 1994. Goal-Directed classification using linear machine decision trees. IEEE Trans. Pattern Anal. Mach. Intell. 16, 9, 888-893
12. ELKAN, C. 2001. The foundations of cost-sensitive learning. In Proceedings of 17th International Joint Conference on Artificial Intelligence (IJCAI '01). Vol. 2., Morgan Kaufmann, 973-978
13. ESMEIR, S. AND MARKOVITCH, S. 2004. Lookahead-Based algorithms for anytime induction of decision trees. In Proceedings of the 21st International Conference on Machine Learning (ICML '04). C. E. Brodley, Ed., 257-264 (Pubitemid 40290816)
14. ESMEIR, S. AND MARKOVITCH, S. 2007. Anytime induction of cost-sensitive trees. In Proceedings of The 21st Annual Conference on Neural Information Processing Systems (NIPS '07). 1-8
15. ESMEIR, S. AND MARKOVITCH, S. 2008. Anytime induction of low-cost, low-error classifiers: A sampling-based approach. J. Artif. Intell. Res. 33, 1-31
16. ESMEIR, S. AND MARKOVITCH, S. 2010. Anytime algorithms for learning resource-bounded classifiers. In Proceedings of the Budgeted Learning Workshop(ICML '10)
17. ESMEIR, S. AND MARKOVITCH, S. 2011. Anytime learning of anycost classifiers. Mach. Learn. 82, 3, 445-473
18. ESTRUCH, V., FERRI, C., HERNáNDEZ-ORALLO, J., AND RAMíREZ-QUINTANA, M. J. 2002. Re-designing cost-sensitive decision tree learning. In Workshop de Mineria de Datos y Aprendizaje. 33-42
19. FAN, W., STOLFO, S. J., ZHANG, J., AND CHAN, P. K. 1999. AdaCost: Misclassification cost-sensitive boosting. In Proceedings of the 16th International Conference on Machine Learning. 97-105
20. FERRI, C., FLACH, P., AND HERNáNDEZ-ORALLO, J. 2002. Learning decision trees using the area under the roc curve. In Proceedings of the 19th Machine Learning International Workshop then Conference. 139-146
21. FERRI-RAMíREZ, C., HERNáNDEZ, J., AND RAMIREZ, M. J. 2002. Induction of decision multi-trees using levin search. In Proceedings of the International Conference on Computational Science (ICCS '02). Lecture Notes in Computer Science, vol. 2329., Springer, 166-175
22. FRANK, E. AND WITTEN, I. 1998. Reduced-Error pruning with significance tests. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.46.2272
23. FREAN, M. 1990. Small nets and short paths: Optimizing neural computation. Doctoral thesis, Centre for Cognitive Science, University of Edinburgh
24. FREITAS, A., COSTA-PEREIRA, A., AND BRAZDIL, P. 2007. Cost-Sensitive decision trees applied to medical data. In Proceedings of the 9th International Conference on Data Warehousing and Knowledge Discovery. Lecture Notes in Computer Science, vol. 4654., Springer, 303-312
25. FREUND, Y. AND SCHAPIRE, R. E. 1996. Experiments with a new boosting algorithm. In Proceedings of the 13thInternational Machine Learning Workshop then Conference. 148-156
26. FREUND, Y. AND SCHAPIRE, R. E. 1997. A decision-theoretic generalization of on-line learning and an application to boosting, J. Comput. Syst. Sci. 55, 1, 119-139 (Pubitemid 127433398)
27. GREFENSTETTE, J. J. 1990. A User's Guide to GENESIS v5.0. Naval Research Laboratory, Washington, DC. GREINER, R., GROVE, A. J., AND ROTH, D. 2002. Learning class-sensitive active classifiers. Art. Intell. 139, 2, 137-174
28. HART, A. E. 1985. Experience in the use of an inductive system in knowledge engineering. In Research and Development in Expert Systems. M. A. Bramer, Ed., Cambridge University Press
29. HUNT, E. B., MARIN, J., AND STONE, P. J. 1966. Experiments in Induction. Academic Press, New York
30. KNOLL, U., NAKHAEIZADEH, G., AND TAUSEND, B. 1994. Cost-Sensitive pruning of decision trees. In Proceedings of the 8th European Conference on Machine Learning (ECML '94). 383-386
31. KRETOWSKI, M. AND GRZES, M. 2007. Evolutionary induction of decision trees for misclassification cost minimization. In Proceedings of the 8th International Conference on Adaptive and Natural Computing Algorithms (ICANNGA). Lecture Notes in Computer Science, vol. 4431, Springer, 1-10
32. LI, J., LI, X., AND YAO, X. 2005. Cost-Sensitive classification with genetic programming. In Proceedings of the IEEE Congress on Evolutionary Computation. 2114-2121 (Pubitemid 41496111)
33. LIN, F. Y. AND MCCLEAN, S. 2000. The Prediction of financial distress using a cost sensitive approach and prior probabilities. In Proceedings of the 17th International Conference on Machine Learning (ICML '00)
34. LING, C. X., YANG, Q., WANG, J., AND ZHANG, S. 2004. Decision trees with minimal costs. In Proceedings of the ACM International Conference on Machine Learning. ACM Press New York
35. LING, C. X., SHENG, V. S., BRUCKHAUS, T., AND MADHAVJI, N. H. 2006a. Maximum profit mining and its application in software development. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '06). 929
36. LING, C., SHENG, V., AND YANG, Q. 2006b. Test strategies for cost-sensitive decision trees. IEEE Trans. Knowl. Data Engin. 18, 8, 1055-1067 (Pubitemid 44109535)
37. LIU, X. 2007. A new cost-sensitive decision tree with missing values. Asian J. Inf. Technol. 6, 11, 1083-1090
38. LOMAX, S. AND VADERA, S. 2011. An empirical comparison of cost-sensitive decision tree induction algorithms. Expert Syst. J. Knowl. Engin. 28, 3, 227-268
39. LOZANO, A.C. AND ABE, N. 2008. Multi-class cost-sensitive boosting with p-norm loss functions. In Proceeding of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '08). 506
40. MARGINEANTU, D. AND DIETTERICH, T. 2003. A wrapper method for cost-sensitive learning via stratification. http://citeseerx.ist.psu.edu/ viewdoc/summary?doi=10.1.1.27.1102
41. MARGINEANTU, D. 2001. Methods for cost-sensitive learning. Doctoral thesis, Oregon State University
42. MERLER, S., FURLANELLO, C., LARCHER, B., AND SBONER, A. 2003. Automatic model selection in cost-sensitive boosting. Inf. Fusion 4, 1, 3-10 (Pubitemid 36210677)
43. MEASE, D., WYNER, A. J., AND BUJA, A. 2007. Boosted classification trees and class probability/quantile estimation. J. Mach. Learn. Res. 8, 409-439 (Pubitemid 46434120)
44. MEIR, R. AND RäTSCH, G. 2003. An introduction to boosting and leveraging. In Advanced Lectures on Machine Learning, Mendelson, S., Smola, A. Eds., Springer, 119-184
45. MICHALEWICZ, Z. 1996. Genetic Algorithms + Data Structures = Evolution Programs 3rd Ed. Springer
46. MINGERS, J. 1989. An empirical comparison of pruning methods for decision tree induction. Mach. Learn. 4, 227-243
47. MORET, S., LANGFORD, W., ANDMARGINEANTU, D. 2006. Learning to predict channel stability using biogeomorphic features. Ecol. Model. 191, 1, 47-57 (Pubitemid 41735451)
48. MORRISON, D. 1976. Multivariate Statistical Method 2nd Ed. McGraw-Hill, New York
49. MURTHY, S., KASIF, S., AND SALZBERG, S. 1994. A system for induction of oblique decision trees. J. Artif. Intell. Res. 2, 1-32
50. MURTHY, S. AND SALZBERG, S. 1995. Lookahead and pathology in decision tree induction. In Proceedings of the 14th International Joint Conference on Artificial Intelligence. 1025-1033
51. NI, A., ZHANG, S., YANG, S., AND ZHU, X. 2005. Learning classification rules under multiple costs. Asian J. Inf. Technol. 4, 1080-1085
52. NILSSON, N. J. 1965. Learning Machines. McGraw-Hill, New York
53. NORTON, S.W. 1989 Generating better decision trees. In Proceedings of the 11th International Joint Conference on Artificial Intelligence (IJCAI '89). 800-805
54. NúNEZ, M. 1991. The use of background knowledge in decision tree induction. Mach. Learn. 6, 231-250 (Pubitemid 21737707)
55. OMIELAN, A. 2005. Evaluation of a cost-sensitive genetic classifier. MPhil thesis, University of Salford
56. PAZZANI, M., MERZ, C., MURPHY, P., ALI, K., HUME, T., AND BRUNK, C. 1994. Reducing misclassification costs. In Proceedings of the 11th International Conference on Machine Learning. 217-225
57. http://scholar.google.com/scholar?hl=en&btnG=Search&q=intitle: Reducing+misclassification+costs#0
58. QIN, Z., ZHANG, S., AND ZHANG, C. 2004. Cost-Sensitive decision trees with multiple cost scales. In Proceedings of the 17th Austrailian Joint Conference on Artificial Intelligence. G. I. Webb and X. Yu, Eds., Lecture Notes in Artificial Intelligence, vol. 3339, Springer, 380-390 (Pubitemid 41050369)
59. QUINLAN, J. R. 1979. Discovering rules by induction from large collections of examples. In Expert Systems in the Micro Electronic Age, D. Michie, Ed., Edinburgh University Press, 168-201
60. QUINLAN, J. R. 1983. Learning efficient classification procedures and their application to chess end games. In Machine Learning: An Artificial Intelligence Approach, Michalski, Garbonell and Mitchell Eds., Tioga Publishing Company, Palo Alto, CA
61. QUINLAN, J. R. 1986. Induction of decision trees. Mach. Learn. 1, 81-106
62. QUINLAN, J. R. 1987. Simplifying decision trees. Int. J. Man-Mach. Studies 27, 221-234 (Pubitemid 18089906)
63. QUINLAN, J. R. 1993. C4.5: Programs for Machine Learning. Morgan Kaufman, San Mateo, CA
64. QUINLAN, J. R., COMPTON, P. J., HORN, K. A., AND LAZARUS, L. 1987. Inductive knowledge acquisition: A case study. In Application of Expert Systems, J. Ross Quinlan, Ed., Turning Institute Press/Addison-Wesley, 137-156
65. RISSANEN, J. 1978. Modelling by shortest data description. Automatica, 14, 465-471
66. SCHAPIRE, R. E. 1999. A Brief Introduction to Boosting. In Proceedings of the 16th International Joint Conference on Artificial Intelligence (IJCAI99). Vol 2, 1401-1406
67. SCHAPIRE, R. E. AND SINGER, Y. 1999. Improved boosting algorithms using confidence-rated predictions. Machine Learn. 37, 3, 297-336 (Pubitemid 32210620)
68. SHANNON, C. E. 1948. The mathematical theory of communication. Bell Syst. Tech. J. 27, 379-423
69. SHENG, S. AND LING, C. 2005. Hybrid cost-sensitive decision tree. In Proceedings of the 9th European Conference on Principles and Practice of Knowledge Discovery in Databases. Lecture Notes in Computer Science, vol. 3721., Springer, 274-284
70. SHENG, S., LING, C., AND YANG, Q. 2005. Simple test strategies for cost-sensitive decision trees. In 16th European Conference on Machine Learning (ECML' 05). Lecture Notes in Computer Science, vol. 3720. Springer, 365-376
71. SWETS, J., DAWES, R., AND MONAHAN, J. 2000. Better decisions through science. Sci. Amer. 283, 4, 82-87
72. TAN, M. 1993. Cost-sensitive learning of classification knowledge and its applications in robotics. Machine Learn. 13, 7-33
73. TAN, M. AND SCHLIMMER J. 1989. Cost-Sensitive concept learning of sensor use in approach and recognition. In Proceedings of the 6th International Workshop on Machine Learning (ML '89). 392-395
74. TING, K. AND ZHENG, Z. 1998a. Boosting cost-sensitive trees. In Proceedings of the 1st International Conference on Discovery Science. Lecture Notes in Computer Science, vol. 1532., Springer, 244-255
75. TING, K. M. AND ZHENG, Z. 1998b. Boosting trees for cost-sensitive classifications. In Proceedings of the 10th European Conference on Machine Learning. Springer, 190-195 (Pubitemid 128067184)
76. TING, K. M. 1998. Inducing cost-sensitive decision trees via instance weighting. In Proceedings of the 2nd European Symposium on Principles of Data Mining and Knowledge Discovery. Springer, 139-147 (Pubitemid 128145830)
77. TING, K. 2000a. An empirical study of Metacost using boosting algorithms. In Proceedings of the 11th European Conference on Machine Learning. Lecture Notes in Computer Science, vol. 1810., Springer, 413-425
78. TING, K. 2000b. A comparative study of cost-sensitive boosting algorithms. In Proceedings of the 17th International Conference on Machine Learning. 983-990
79. TING, K. M. 2002. An instance-weighting method to induce cost-sensitive decision trees. IEEE Trans. Knowl. Data Engin. 14, 3, 659-665 (Pubitemid 34669622)
80. TURNEY, P. D. 1995. Cost-Sensitive classification: Empirical evaluation of a hybrid genetic decision tree induction algorithm. J. Artif. Intell. Res. 2, 369-409
81. VADERA, S. 2005a. Inducing cost-sensitive non-linear decision trees. Tech. rep. http://scholar.google.com/scholar?hl=en&btnG=Search&q= intitle:Inducing+cost-sensitive+non-linear+decision+trees#0
82. VADERA, S. 2005b. Inducing safer oblique trees without costs. Expert Syst: Int. J. Knowl. Engin. Neural Netw. 22, 4, 206-221 (Pubitemid 43906931)
83. VADERA, S. 2010. CSNL: A cost-sensitive non-linear decision tree algorithm. ACM Trans. Knowl. Discov. Data 4, 2, 1-25
84. VON NEUMANN, J. 1951. Various techniques used in connection with random digits. Monte carlo methods. Nat. Bureau Standards 12, 36-38
85. WINSTON, P. H. 1993. Artificial Intelligence 3rd Ed. Addison Wesley.
86. ZADROZNY, B., LANGFORD, J., ANDABE, N. 2003a. Asimple method for cost-sensitive learning. Tech. rep. RC22666
87. http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.7.7947
88. ZADROZNY, B., LANGFORD, J., AND ABE, N. 2003b. Cost-Sensitive learning by cost-proportionate example weighting. In Proceedings of the 3rd IEEE International Conference on Data Mining. 435
89. ZHANG, S., QIN, Z., LING, C., AND SHENG, S. 2005. Missing is useful: Missing values in cost-sensitive decision trees. IEEE Trans. Knowl. Data Engin. 17, 12, 1689-1693 (Pubitemid 43060418)
90. ZHANG, S., ZHU, X., ZHANG, J., AND ZHANG, C. 2007. Cost-Time sensitive decision tree with missing values. Knowl. Sci., Engin. Manag. 4798, 447-459
91. ZHANG, S. 2010. Cost-Sensitive classification with respect to waiting cost. Knowl Based Syst. 23, 5, 369-378.