A multi-objective genetic programming approach to developing Pareto optimal decision trees

Decision Support Systems

Volumn 43, Issue 3, 2007, Pages 809-826

  Zhao, Huimin ^a  

^a UNIVERSITY OF WISCONSIN MILWAUKEE   (United States)

Author keywords

Binary classification; Cost sensitive classification; Data mining; Decision tree; Genetic programming; Multi objective optimization; Pareto optimality

Indexed keywords

DATA MINING; DECISION TREES; PARETO PRINCIPLE; PATTERN RECOGNITION; PROBLEM SOLVING; SMART CARDS;

BINARY CLASSIFICATION; COST SENSITIVE CLASSIFICATION; MULTI OBJECTIVE OPTIMIZATION; PARETO OPTIMALITY;

GENETIC PROGRAMMING;

EID: 33947502269     PISSN: 01679236     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.dss.2006.12.011     Document Type: Article

References (65)

1. Anandarajan M. Profiling Web usage in the workplace: a behavior-based artificial intelligence approach. Journal of Management Information Systems 19 1 (2002) 243-266
2. Banzhaf W., Nordin P., Keller R.E., and Francone F.D. Genetic Programming: An Introduction (1998), Morgan Kaufman, San Francisco, CA
3. Ben-Tal A. Characterization of Pareto and lexicographic optimal solutions. Multiple Criteria Decision Making, Theory and Applications. Lecture Notes in Economics and Mathematical Systems vol. 177 (1980), Springer-Verlag, Berlin, Germany 1-11
4. Bleuler S., Brack M., Thiele L., and Zitzler E. Multiobjective genetic programming: reducing bloat using SPEA2. Proceedings of the 2001 Congress on Evolutionary Computation (2001) 536-543
5. Bot M.C.J., and Langdon W.B. Application of genetic programming to induction of linear classification trees. Proceedings of the Third European Conference on Genetic Programming (2000) 247-258
6. Domingos P. MetaCost: A general method for making classifiers cost sensitive. Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (1999) 155-164
7. Domingos P., and Pazzani M. Beyond independence: conditions for the optimality of the simple Bayesian classifier. Proceedings of the Thirteenth International Conference on Machine Learning (1996) 105-112
8. Dworman G., Kimbrough S.O., and Laing J.D. On automated discovery of models using genetic programming: bargaining in a three-agent coalitions game. Journal of Management Information Systems 12 3 (1996) 97-125
9. Elkan C. The foundations of cost-sensitive learning. Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence (2001) 973-978
10. Fan W., Gordon M.D., and Pathak P. Genetic programming-based discovery of ranking functions for effective Web search. Journal of Management Information Systems 21 4 (2005) 37-56
11. Fan W., Pathak P., and Wallace L. Nonlinear ranking function representations in genetic programming-based ranking discovery for personalized search. Decision Support Systems 42 3 (2006) 1338-1349
12. Fawcett T. ROC Graphs: Notes and Practical Considerations for Data Mining Researchers, HPL-2003-4, Intelligent Enterprise Technologies Lab (2003), Hewlett-Packard, Palo Alto, CA
13. Folino G., Pizzuti C., and Spezzano G. Improving induction decision trees with parallel genetic programming. Proceedings of the Tenth Euromicro Workshop on Parallel, Distributed and Network-based Processing (2002) 181-187
14. Fonseca C.M., and Fleming P.J. Multiobjective optimization and multiple constraints handling with evolutionary algorithms. Part 1: A unified formulation. IEEE Transactions on Systems, Man, and Cybernetics 28 1 (1998) 26-37
15. C. Gathercole, An Investigation of Supervised Learning in Genetic Programming, PhD Dissertation, University of Edinburgh, 1998.
16. Holland J. Adaptation in Natural and Artificial Systems (1975), MIT Press, Cambridge, MA
17. Huang J., and Ling C.X. Using AUC and accuracy in evaluating learning algorithms. IEEE Transactions on Knowledge and Data Engineering 17 3 (2005) 299-310
18. Huang Z., Chen H., Hsu C., Chen W., and Wu S. Credit rating analysis with support vector machines and neural networks: a market comparative study. Decision Support Systems 37 4 (2004) 543-558
19. Hyafil L. Construction optimal binary decision trees is NP-complete. Information Processing Letters 5 1 (1976) 15-17
20. Jain H., Zhao H., and Chinta N.R. Web service identification - a multi-objective genetic algorithm based approach. Proceedings of the Second Workshop on e-Business (2003) 351-361
21. Keerthi S.S., Shevade S.K., Bhattacharyya C., and Murthy K.R.K. Improvements to Platt's SMO Algorithm for SVM Classifier Design. Technical Report vol. CD-99-14 (1999), Control Division, Department of Mechanical and Production Engineering, National University of Singapore
22. Kiang M.Y. A comparative assessment of classification methods. Decision Support Systems 35 4 (2003) 441-454
23. Kim E., Kim W., and Lee Y. Combination of multiple classifiers for the customer's purchase behavior prediction. Decision Support Systems 34 2 (2003) 167-175
24. Kohavi R. A study of cross-validation and bootstrap for accuracy estimation and model selection. Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence (1995) 1137-1143
25. Koza J.R. Concept formation and decision tree induction using the genetic programming paradigm. Proceedings of the First Workshop on Parallel Problem Solving from Nature (1991) 124-128
26. Koza J.R. Genetic Programming: On the Programming of Computers by Means of Natural Selection (1992), The MIT Press, Cambridge, MA
27. J. Li, FGP: a Genetic Programming Based Tool for Financial Forecasting, PhD Thesis, University of Essex, 2001.
28. Li X.-B. A scalable decision tree system and its application in pattern recognition and intrusion detection. Decision Support Systems 41 1 (2005) 112-130
29. Li J., Li X., and Yao X. Cost-sensitive classification with genetic programming. Proceedings of the 2005 Congress on Evolutionary Computation vol. 3 (2005) 2114-2121
30. Liu Y., Ram S., and Lusch R. A unified market segmentation method for generating Pareto optimal solution sets. Proceedings of the Fifteenth Workshop on Information Technologies and Systems (2005) 219-224
31. Loveard T., and Ciesielski V. Representing classification problems in genetic programming. Proceedings of the 2001 Congress on Evolutionary Computation (2001) 1070-1077
32. Loveard T., and Ciesielski V. Employing nominal attributes in classification using genetic programming. Proceedings of the Fourth Asia-Pacific Conference on Simulated Evolution and Learning (2002) 487-491
33. Mangiameli P., West D., and Rampal R. Model selection for medical diagnosis decision support systems. Decision Support Systems 36 3 (2004) 247-259
34. Mirrazavi S.K., Jones D.F., and Tamiz M. MultiGen: an integrated multiple-objective solution system. Decision Support Systems 36 2 (2003) 177-187
35. Montana D.J. Strongly typed genetic programming. Evolutionary Computation 3 2 (1995) 199-230
36. Murthy S.K. Automatic construction of decision trees from data: A multi-disciplinary survey. Data Mining and Knowledge Discovery 2 4 (1998) 345-389
37. Newman D.J., Hettich S., Blake C.L., and Merz C.J. UCI Repository of Machine Learning Databases (1998). http://www.ics.uci.edu/~mlearn/MLRepository.html
38. Platt J. Fast training of support vector machines using sequential minimal optimization. In: Schölkopf B., Burges C., and Smola A. (Eds). Advances in Kernel Methods - Support Vector Learning (1998), MIT Press
39. Provost F., Fawcett T., and Kohavi R. The case against accuracy estimation for comparing induction algorithms. Proceedings of the Fifteenth International Conference on Machine Learning (1998) 445-453
40. Quinlan J.R. Induction of decision trees. Machine Learning 1 1 (1986) 81-106
41. Quinlan J.R. Simplifying decision trees. International Journal of Man-Machine Studies 27 (1987) 221-234
42. Quinlan J.R. C4.5: Programs for Machine Learning (1993), Morgan Kaufmann, San Francisco
43. Rodríguez-Vázquez K., Fonseca C.M., and Fleming P.J. Identifying the structure of nonlinear dynamic systems using multiobjective genetic programming. IEEE Transactions on Systems, Man and Cybernetics: Part A. Systems and Humans 34 4 (2004) 531-545
44. Rumelhart D.E., Hinton G.E., and William R.J. Learning representations by back-propagating errors. Nature 323 (1986) 533-536
45. Ryu Y.U., and Yue W.T. Firm bankruptcy prediction: experimental comparison of isotonic separation and other classification approaches. IEEE Transactions on Systems, Man and Cybernetics: Part A. Systems and Humans 35 5 (2005) 727-737
46. Sarkar S., and Sriram R.S. Bayesian models for early warning of bank failures. Management Science 47 11 (2001) 1457-1475
47. Shirasaka M., Zhao Q., Hammarmi O., Kuroda K., and Saito K. Automatic design of binary decision trees based on genetic programming. Proceedings of the Second Asia-Pacific Conference on Simulated Evolution and Learning (1998)
48. Sinha A.P., and May J.H. Evaluating and tuning predictive data mining models using receiver operating characteristic curves. Journal of Management Information Systems 21 3 (2005) 249-280
49. Smith J.W., Everhart J.E., Dickson W.C., Knowler W.C., and Johannes R.S. Using the ADAP learning algorithm to forecast the onset of diabetes mellitus. Proceedings of the Symposium on Computer Applications and Medical Care (1988) 261-265
50. Srinivas N., and Deb K. Multiobjective optimization using nondominated sorting in genetic algorithms. Evolutionary Computation 2 3 (1994) 221-248
51. Sung T.K., Chang N., and Lee G. Dynamics of modeling in data mining: interpretive approach to bankruptcy prediction. Journal of Management Information Systems 16 1 (1999) 63-86
52. Thomassey S., and Fiordaliso A. A hybrid sales forecasting system based on clustering and decision trees. Decision Support Systems 42 1 (2006) 408-421
53. Ting K.M. An Instance-weighting method to induce cost-sensitive trees. IEEE Transactions on Knowledge and Data Engineering 14 3 (2002) 659-665
54. Tsang E., Yung P., and Li J. EDDIE-automation, a decision support tool for financial forecasting. Decision Support Systems 37 4 (2004) 559-565
55. Tür G., and Güvenir H.A. Decision tree induction using genetic programming. Proceedings of the Fifth Turkish Symposium on Artificial Intelligence and Neural Networks (1996)
56. Weiss S.M., and Kulikowski C.A. Computer Systems That Learn - Classification and Prediction Methods from Statistics. Neural Nets, Machine Learning, and Expert System (1991), Morgan Kaufmann
57. Witten I.H., and Frank E. Data Mining: Practical Machine Learning Tools and Techniques. 2nd Edition (2005), Morgan Kaufmann
58. Zadrozny B., Langford J., and Abe N. Cost-sensitive learning by cost-proportionate example weighting. Proceedings of the IEEE International Conference on Data Mining (2003) 435-442
59. Zhao H., and Ram S. Constrained cascade generalization of decision trees. IEEE Transactions on Knowledge and Data Engineering 16 6 (2004) 727-739
60. Zhao Q., and Shirasaka M. A study on evolutionary design of binary decision trees. Proceedings of the 1999 Congress on Evolutionary Computation (1999) 1988-1993
61. Zhao H., and Sinha A.P. An efficient algorithm for generating generalized decision forests. IEEE Transactions on Systems, Man, and Cybernetics: Part A. Systems and Humans 35 5 (2005) 754-762
62. Zhou L., Burgoon J.K., Twitchell D.P., Qin T., and Nunamaker J. A comparison of classification methods for predicting deception in computer-mediated communication. Journal of Management Information Systems 20 4 (2004) 139-165
63. Zhu D., Premkumar G., Zhang X., and Chu C. Data mining for network intrusion detection: A comparison of alternative methods. Decision Sciences 32 4 (2001) 635-660
64. E. Zitzler, Evolutionary Algorithms for Multiobjective Optimization: Methods and Applications, PhD thesis, ETH Zurich, Switzerland, 1999.
65. Zitzler E., Deb K., and Thiele L. Comparison of multiobjective evolutionary algorithms: empirical results. Evolutionary Computation 8 2 (2000) 173-195