CSNL: A cost-sensitive non-linear decision tree algorithm

ACM Transactions on Knowledge Discovery from Data

Volumn 4, Issue 2, 2010, Pages

  Vadera, Sunil ^a  

^a UNIVERSITY OF SALFORD   (United Kingdom)

Author keywords

Cost sensitive learning; Decision tree learning

Indexed keywords

COST-SENSITIVE; COST-SENSITIVE ALGORITHM; COST-SENSITIVE LEARNING; DATA SETS; DECISION TREE LEARNING; DECISION TREE LEARNING ALGORITHM; MISCLASSIFICATIONS; MULTIPLE TREES; NON-LINEAR;

COSTS; DECISION TREES; DISCRIMINANT ANALYSIS; LEARNING ALGORITHMS;

COST BENEFIT ANALYSIS;

EID: 77953193051     PISSN: 15564681     EISSN: 1556472X     Source Type: Journal    
DOI: 10.1145/1754428.1754429     Document Type: Article

References (59)

1. ABE, N., ZADROZNY, B., AND LANGFORD, J. 2004. An iterative method for multi-class cost-sensitive learning. In Proceedings of the International Conference on Knowledge Discovery and Data Mining (KDD). 3-11.
2. ALLWEIN, E. L., SCHAPIRE, R. E., AND SINGER, Y. 2000. Reducing multiclass to binary: A unifying approach for margin classifiers. In Proceedings of the 17th International Conference on Machine Learning. Morgan Kaufmann, San Francisco, 9-16.
3. ALTHOFF, K., AURIOL, E., BARLETTA, R., AND MANAGO, M. 1995. A Review of Industrial Case-Based Reasoning Tools. AI Intelligence, Oxford.
4. BAUER, E. AND KOHAVI, R. 1999. An empirical comparison or voting classification algorithms: Bagging, boosting, and variants. Mach. Learn. 36, 1-2, 105-139.
5. BENNETT, K. P. 1999. On mathmatical programming methods and support vector machines. In Advances in Kernel Methods-Support Vector Machines, A. Schoelkopf, C. Burges, and A. Smola, Eds. MIT Press, Cambridge, MA, Chapter 19, 307-326.
6. BERRY, M. AND LINOFF, G. 2004. Data Mining Techniques, 2nd Ed. Wiley, New York.
7. BLAKE, C. AND MERZ, C. 1998. UCI repository of machine learning databases. Department of Information and Computer Science, University of California, Irvine. http://www.ics.uci.edu/mlearn/ MLRepository.html.
8. BRADFORD, J., KUNZ, C., KOHAVI, R., BRUNK, C., AND BRODLEY, C. 1998. Pruning decision trees with misclassification costs. In Proceedings of the 10th European Conference on Machine Learning. Lecture Notes in Computer Science, No.398. Springer-Verlag, Heidelberg, Berlin, 131-136.
9. BREIMAN, L. 1996. Bagging predictors. Mach. Learn. 24, 2, 123-140.
10. BREIMAN, L., FRIEDMAN, J. H., OLSEN, R. A., AND STONE, C. J. 1984. Classification and Regression Trees. Wadsworth, Belmont.
11. BRESLOW, L. AND AHA, D. 1997a. Comparing tree-simplification procedures. In Proceedings of the 6th International Workshop on Artificial Intelligence and Statistics. 67-74.
12. BRESLOW, L. AND AHA, D. 1997b. Simplifying decision trees: A survey. Knowl. Engin. Rev. 12, 1-40.
13. BROWN, G. 2009. Feature selection by filters: a unifying perspective. In Proceedings of the 5th UK Symposium on Knowledge Discovery and Data Mining. S. Vadera, Ed., 34-43.
14. BUNTINE, W. AND NIBLETT, T. 1992. A further comparison of splitting rules for decision-tree induction. Mach. Learn. 8, 75-85.
15. DASH, M. AND LIU, H. 1997. Feature selection for classification. Intell. Data Anal. 1, 3, 131-156.
16. DOMINGOS, P. 1999. MetaCost: A general method for making classifiers cost-sensitive. In Proceedings of the 5th International Conference on Knowledge Discovery and DataMining. 155-164.
17. DRAPER, B., BRODLEY, C. E., AND UTGOFF, P. E. 1994. Goal-directed classification using linear machine decision trees. IEEE Trans. Patt. Anal. Mach. Intel. 16, 9, 888-893.
18. DRUMMOND, C. AND HOLTE, R. 2006. Cost curves: An improved method for visualizing classifier performance. Mach. Learn. 65, 1, 95-130.
19. ELKAN, C. 2001. The foundations of cost-sensitive learning. In Proceedings of the 17th International Joint Conference on Artificial Intelligence. Morgan Kaufmann, San Francisco, 973-978.
20. ESMEIR, S. AND MARKOVITCH, S. 2008. Anytime induction of low-cost, low-error classifiers: a sampling-based approach. J. Artif. Intel. Resear. 33, 1-31.
21. ESPOSITO, F.,MALERBA, D., AND SEMERARO, G. 1997. A compartive analysis of methods for pruning decision trees. IEEE Trans. Patt. Anal. Mach. Intel. 19, 5, 476-491.
22. FAN, W., STOLFO, S., ZHANG, J., AND CHAN, P. 1999. AdaCost: Misclassification cost-sensitive boosting. In Proceedings of the 16th International Conference on Machine Learning. Morgan Kaufmann, San Francisco, 97-105.
23. FISHER, R. 1936. The use of multiple measurements in taxonomic problems. Annal. Eugen. 8, 179-188.
24. FRANK, E. AND WITTEN, I. 1998. Reduced-error pruning with significance tests. http://citeseer.ist.psu.edu/frank98reducederror.html.
25. GREFENSTETTE, J. 1986. Optimization of control parameters for genetic algorithms. IEEE Trans. Syst. Man, Cybern. 16, 122-128.
26. HONG, S. J. 1997. Use of contextual information for feature ranking and discretization. IEEE Trans. Knowl. Data Engin. 9, 718-730.
27. JI, S. AND CARIN, L. 2007. Cost-sensitive feature acquisition and classification. Patt. Recog. 40, 5, 1474-1485.
28. JOHNSON, R. AND WICHERN, D. 1998. Applied Multivariate Statistical Analysis 4th Ed. Prentice Hall, Englewood Cliffs.
29. KANANI, P. AND MELVILLE, P. 2008. Prediction-time active feature-value acquisition for customer targeting. In Proceedings of the Workshop on Cost Sensitive Learning. http://www. cs.iastate.edu/oksayakh/csl/accepted papers/kanani.pdf.
30. KNOLL, U., NAKHAEIZADEH, G., AND TAUSEND, B. 1994. Cost-sensitive pruning of decision trees. In Proceedings of the 8th European Conference on Machine Learning. Vol.2. Springer-Verlag, Berlin, 383-386.
31. KOHAVI, R. AND JOHN, G. H. 1997. Wrappers for feature subset selection. Artif. Intel. 97, 1-2, 273-324.
32. LING, C., SHENG, V., AND YANG, G. 2006. Test strategies for cost-sensitive decision trees. IEEE Trans. Knowl. Data Engin. 18, 8, 1055-1067.
33. MARGINEANTU, D. 2001. Methods for cost-sensitive learning. Ph.D. thesis, School of Electrical Engineering and Computer Science, Oregan State University, Corvallis, OR.
34. MARTIN, A., DODDINGTON, G., KAMM, T., ORDOWSKI, M., AND PRZYBOCKI, M. 1997. The DET curve in assessment of detection task performance. In Proceedings of Eurospeech'97. International Speech Communications Association, 1895-1898.
35. MASNADI-SHIRAZI, H. AND VASCONCELOS, N. 2007. Asymmetric boosting. In Proceedings of the 24th International Conference on Machine Learning. 609-616. (Pubitemid 350094030)
36. MASNADI-SHIRAZI, H. AND VASCONCELOS, N. 2008. On the design of loss functions for classification: Theory, robustness to outliers, and SavageBoost. In Proceedings of Neural Information Processing Systems. D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, Eds. 1049-1056.
37. MERLER, S., FURLANELLO, C., LARCHER, B., AND SBONER, A. 2003. Automatic model selection in cost-sensitive boosting. Inform. Fusion 4, 3-10.
38. MINGERS, J. 1989. An empirical comparison of pruning methods for decision tree induction. Mach. Learn. 4, 227-243.
39. MURTHY, S., KASIF, S., AND SALZBERG, S. 1994. A system for induction of oblique decision trees. J. Artif. Intel. Resear. 2, 1-32.
40. NILSSON, N. 1965. Learning Machines. McGraw-Hill, New York.
41. ŃUÑEZ, M. 1991. The use of background knowledge in decision tree induction. Mach. Learn. 6, 231-250.
42. PAZZANI, M., MERZ, C., MURPHY, P., ALI, K., HURNE, T., AND BRUNK, C. 1994. Reducing misclassification costs: Knowledge-intensive approaches to learning from noisy data. In Proceedings of the 11th International Conference on Machine Learning. Morgan Kaufmann, San Francisco, CA, 217-225.
43. PAZZANI, M. J. 2000. Knowledge discovery from data? IEEE Intel. Syst. 15, 2, 10-13.
44. PROVOST, F. J. AND BUCHANAN, B. G. 1995. Inductive policy: The pragmatics of bias selection. Mach. Learn. 20, 35-61.
45. PROVOST, F. J., FAWCETT, T., AND KOHAVI, R. 1998. The case against accuracy estimation for comparing induction algorithms. In Proceedings of the 15th International Conference on Machine Learning. 445-553.
46. QUINLAN, J. R. 1987. Simplifying decision trees. Int. J. Man-Mach. Stud. 27, 221-234.
47. QUINLAN, J. R. 1993. C4.5: Programs for Machine Learning. Morgan Kauffman, California.
48. SAVAGE, L. 1971. The elicitation of personal probabilities and expectations. J. Am. Statist. Ass. 66, 783-801.
49. SWETS, J. 1964. Signal Detection and Recognition by Human Observers. John Wiley & Sons Inc.
50. TAN,M. 1993. Cost-sensitive learning of classification knowledge and its applications in robotics. Mach. Learn. 13, 7-33.
51. TING, K. 2000. A comparative study of cost-sensitive boosting algorithms. In Proceedings of the 17th International Conference on Machine Learning. Morgan Kaufmann, San Francisco, 983-990.
52. TING, K. AND ZHENG, Z. 1998. Boosting trees for cost-sensitive classifications. In Proceedings of the 10th European Conference on Machine Learning. 190-195.
53. TURNEY, P. 1995. Cost sensitive classification: Empirical evaluation of a hybrid genetic decision tree induction algorithm. J. Artif. Intel. Resear. 2, 369-409.
54. TURNEY, P. 2000. Types of cost in inductive concept learning. In Proceedings of the Workshop on Cost-Sensitive Learning 7th International Conference on Machine Learning. 15-21.
55. VADERA, S. 2005a. Inducing cost-sensitive nonlinear decision trees, Tech. rep. School of Computing, Science and Engineering, University of Salford.
56. VADERA, S. 2005b. Inducing safer oblique trees without costs. Int. J. Knowl. Engin. Neural Netw. 22, 4, 206-221.
57. VADERA, S. AND VENTURA, D. 2001. A comparison of cost-sensitive decision tree learning algorithms. In Proceedings of the 2nd European Conference on Intelligent Management Systems in Operations. Operational Research Society, 79-86.
58. ZADROZNY, B., LANGFORD, J., AND ABE, N. 2003. Cost-sensitive learning by cost-proportionate example weighting. In Proceedings of the 3rd IEEE International Conference on Data Mining. 435-442.
59. ZHU, X.,WU, X., KHOSHGOFTAAR, T., AND SHI, Y. 2007. An empirical study of noise impact on costsensitive learning. In Proceedings of 20th International Joint Conference on Artificial Intelligence. 1168-1174.