Evolving decision trees with beam search-based initialization and lexicographic multi-objective evaluation

Information Sciences

Volumn 258, Issue , 2014, Pages 160-181

  Basgalupp, Márcio P ^a   Barros, Rodrigo C ^b   De Carvalho, André C P L F ^b   Freitas, Alex A ^c  

^a FEDERAL UNIVERSITY OF SÃO PAULO   (Brazil)

^b UNIVERSITY OF SÃO PAULO   (Brazil)

^c UNIVERSITY OF KENT   (United Kingdom)

Author keywords

Decision tree; Lexicographic optimization; Machine learning; Multi objective evolutionary algorithm

Indexed keywords

DECISION-TREE INDUCTION; EVOLUTIONARY ASPECTS; FITNESS FUNCTIONS; GREEDY APPROACHES; LEXICOGRAPHIC OPTIMIZATION; MULTI OBJECTIVE EVOLUTIONARY ALGORITHMS; MULTI-OBJECTIVE EVALUATIONS; MULTI-OBJECTIVE GENETIC ALGORITHM;

DATA MINING; EVOLUTIONARY ALGORITHMS; LEARNING SYSTEMS; LINEAR PROGRAMMING;

DECISION TREES;

EID: 84889082685     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2013.07.025     Document Type: Article

References (72)

1. K. Bache, M. Lichman, UCI machine learning repository, 2013.
2. R.C. Barros, M.P. Basgalupp, A.C.P.L. de Carvalho, and A.A. Freitas A survey of evolutionary algorithms for decision-tree induction IEEE Trans. Syst. Man. Cybern. C Appl. Rev. 42 2012 291 312
3. R.C. Barros, M.P. Basgalupp, A.C.P.L.F. de Carvalho, and A.A. Freitas Towards the automatic design of decision tree induction algorithms Proceedings of the 13th Annual Conference Companion on Genetic and Evolutionary Computation, GECCO '11 2011 ACM New York, NY, USA 567 574
4. R.C. Barros, M.P. Basgalupp, A.C.P.L.F. de Carvalho, and A.A. Freitas A hyper-heuristic evolutionary algorithm for automatically designing decision-tree algorithms Proceedings of the 14th International Conference on Genetic and Evolutionary Computation Conference, GECCO '12 2012 ACM New York, NY, USA 1237 1244
5. R.C. Barros, M.P. Basgalupp, A.C.P.L.F. de Carvalho, A.A. Freitas, Automatic design of decision-tree algorithms with evolutionary algorithms, Evol. Comput. (2013). (in press). http://www.mitpressjournals.org/doi/abs/10.1162/ EVCO-a-00101
6. R.C. Barros, M.P. Basgalupp, D.D. Ruiz, A.C.P.L.F. de Carvalho, and A.A. Freitas Evolutionary model tree induction SAC '10: Proceedings of the 2010 ACM Symposium on Applied Computing 2010 ACM New York, NY, USA 1131 1137
7. R.C. Barros, D.D. Ruiz, and M.P. Basgalupp Evolutionary model trees for handling continuous classes in machine learning Inf. Sci. (Ny) 181 2011 954 971
8. R.C. Barros, A.T. Winck, K.S. Machado, M.P. Basgalupp, A.C.P.L.F. de Carvalho, D.D. Ruiz, and O.N. de Souza Automatic design of decision-tree induction algorithms tailored to flexible-receptor docking data BMC Bioinform. 13 2012 310
9. M.P. Basgalupp, R.C. Barros, A.C.P.L.F. de Carvalho, A.A. Freitas, and D.D. Ruiz Legal-tree: a lexicographic multi-objective genetic algorithm for decision tree induction Proceedings of the 2009 ACM Symposium on Applied Computing, SAC'09 2009 ACM Press 1085 1090
10. M.P. Basgalupp, R.C. Barros, and D.D. Ruiz Predicting software maintenance effort through evolutionary-based decision trees Proceedings of the 2012 ACM Symposium on Applied Computing, SAC '12 2012 ACM New York, NY, USA 1209 1214
11. M.P. Basgalupp, R.C. Barros, T.S. da Silva, and A.C.P.L.F. de Carvalho Software effort prediction: a hyper-heuristic decision-tree based approach Proceedings of the 2013 ACM Symposium on Applied Computing, SAC'13 2013 ACM Press 1109 1116
12. M.P. Basgalupp, A.C.P.L.F. de Carvalho, R.C. Barros, D.D. Ruiz, and A.A. Freitas Lexicographic multi-objective evolutionary induction of decision trees Int. J. Bioinspired Comput. 1 2009 105 117
13. K. Bennett Global tree optimization: a non-greedy decision tree algorithm Comput. Sci. Stat. 1994 156 160
14. M.C. Bot, and W.B. Langdon Application of genetic programming to induction of linear classification trees R. Poli, W. Banzhaf, W.B. Langdon, J. Miller, P. Nordin, T.C. Fogarty, Genetic Programming Lecture Notes in Computer Science vol. 1802 2004 Springer Berlin/Heidelberg 247 258
15. L. Breiman, J.H. Friedman, R.A. Olshen, and C.J. Stone Classification and Regression Trees 1984 Wadsworth
16. W. Buntine Learning classification trees Stat. Comput. 2 1992 63 73
17. A. Cano, A. Zafra, and S. Ventura An interpretable classification rule mining algorithm Inf. Sci. (Ny) 240 2013 1 20
18. P. Chou Optimal partitioning for classification and regression trees IEEE Trans. Pattern Anal. Mach. Intell. 13 1991 340 354
19. C.A.C. Coello, G.B. Lamont, and D.A.V. Veldhuizen Evolutionary Algorithms for Solving Multi-Objective Problems (Genetic and Evolutionary Computation) 2006 Springer-Verlag New York, Inc. Secaucus, NJ, USA
20. J. Demšar Statistical comparisons of classifiers over multiple data sets J. Mach. Learn. Res. 7 2006 1 30
21. M. Dong, and R. Kothari Look-ahead based fuzzy decision tree induction IEEE Trans. Fuzzy Syst. 9 2001 461 468
22. J. Eggermont, J. Kok, W. Kosters, Genetic programming for data classification: refining the search space, in: Proceedings of the 15th Belgium/Netherlands Conference on Artificial Intelligence, BNAIC'03, 2003, pp. 123-130.
23. J. Eggermont, J.N. Kok, and W.A. Kosters Detecting and pruning introns for faster decision tree evolution X. Yao, E. Burke, J.A. Lozano, J. Smith, J.J. Merelo-Guervos, J.A. Bullinaria, J. Rowe, P. Tino, A. Kaban, H.P. Schwefel, Parallel Problem Solving from Nature - PPSN VIII Lecture Notes in Computer Science vol. 3242 2004 Springer Berlin/Heidelberg 1071 1080
24. J. Eggermont, J.N. Kok, and W.A. Kosters Genetic programming for data classification: partitioning the search space Proceedings of the 2004 ACM Symposium on Applied Computing, SAC'04 2004 ACM New York, NY, USA 1001 1005
25. S. Esmeir, and S. Markovitch Anytime learning of decision trees J. Mach. Learn. Res. 8 2007 891 933
26. J.K. Estrada-Gil, J.C. Fernández-López, E. Hernández-Lemus, I. Silva-Zolezzi, A. Hidalgo-Miranda, G. Jiménez-Sánchez, and E.E. Vallejo-Clemente Gpdti: a genetic programming decision tree induction method to find epistatic effects in common complex diseases Bioinformatics 23 2007 i167 i174
27. G. Folino, C. Pizzuti, and G. Spezzano Genetic programming and simulated annealing: a hybrid method to evolve decision trees Proceedings of the European Conference on Genetic Programming 2000 Springer-Verlag London, UK 294 303
28. G. Folino, C. Pizzuti, and G. Spezzano Parallel genetic programming for decision tree induction Proceedings of the 13th IEEE International Conference on Tools with Artificial Intelligence, ICTAI '01 2001 IEEE Computer Society Washington, DC, USA 129
29. G. Folino, C. Pizzuti, and G. Spezzano Improving induction decision trees with parallel genetic programming Proceedings of the Euromicro Conference on Parallel, Distributed, and Network-Based Processing 2002 IEEE Computer Society Los Alamitos, CA, USA 1 7
30. A.A. Freitas Data Mining and Knowledge Discovery with Evolutionary Algorithms 2002 Springer-Verlag New York, Inc. Secaucus, NJ, USA
31. A.A. Freitas A critical review of multi-objective optimization in data mining: a position paper SIGKDD Explor. Newsl. 6 2004 77 86
32. Y. Freund, and L. Mason The alternating decision tree learning algorithm ICML '99: Proceedings of the Sixteenth International Conference on Machine Learning 1999 Morgan Kaufmann Publishers Inc. San Francisco, CA, USA 124 133
33. M. Friedman A comparison of alternative tests of significance for the problem of m rankings Ann. Mathe. Stat. 11 1940 86 92
34. Z. Fu, B.L. Golden, S. Lele, S. Raghavan, and E. Wasil Diversification for better classification trees Comput. Oper. Res. 33 2006 3185 3202
35. Z. Fu, F. Mae, A computational study of using genetic algorithms to develop intelligent decision trees, in: Proceedings of the 2001 IEEE Congress on Evolutionary Computation, CEC'01, 2001, pp. 1382-1387.
36. A. Garcia-Almanza, E. Tsang, Simplifying decision trees learned by genetic programming, in: Proceedings of the 2006 IEEE Congress on Evolutionary Computation, CEC'06, 2006, pp. 2142-2148.
37. D. Haizhou, and M. Chong Study on constructing generalized decision tree by using dna coding genetic algorithm Proceedings of the 2009 International Conference on Web Information Systems and Mining, WISM '09 2009 IEEE Computer Society Washington, DC, USA 163 167
38. J. Han Data Mining: Concepts and Techniques 2001 Morgan Kaufmann Publishers Inc. San Francisco, CA, USA
39. U. Johansson, L. Niklasson, Evolving decision trees using oracle guides, in: Proceedings of the 2009 IEEE Symposium on Computational Intelligence and Data Mining, CIDM'09, 2009, pp. 238-244.
40. D. Kim Structural risk minimization on decision trees using an evolutionary multiobjective optimization M. Keijzer, U.M. O'Reilly, S.M. Lucas, E. Costa, T. Soule, Genetic Programming Lecture Notes in Computer Science vol. 3003 2004 Springer Berlin/Heidelberg 338 348
41. P. Kokol, S. Pohorec, G. Štiglic, and V. Podgorelec Evolutionary design of decision trees for medical application Wiley Interdisciplinary Rev.: Data Min. Knowledge Discovery 2 2012 237 254
42. J.R. Koza Concept formation and decision tree induction using the genetic programming paradigm PPSN I: Proceedings of the 1st Workshop on Parallel Problem Solving from Nature 1991 Springer-Verlag London, UK 124 128
43. M. Kretowski, and M. Grzes Evolutionary induction of cost-sensitive decision trees F. Esposito, Z. Ras, D. Malerba, G. Semeraro, Foundations of Intelligent Systems Lecture Notes in Computer Science vol. 4203 2006 Springer Berlin/Heidelberg 121 126
44. M. Kretowski, M. Grzes, Mixed decision trees: An evolutionary approach, in: Proceedings of the 8th International Conference on Data Warehousing and Knowledge Discovery, DaWaK'06, Krakow, Poland, 2006, pp. 260-269.
45. M. Kretowski, and M. Grzes Evolutionary induction of decision trees for misclassification cost minimization Proceedings of the 8th International Conference on Adaptive and Natural Computing Algorithms, Part I, ICANNGA '07 2007 Springer-Verlag Berlin, Heidelberg 1 10
46. M. Kretowski, and M. Grzes Evolutionary induction of decision trees for misclassification cost minimization B. Beliczynski, A. Dzielinski, M. Iwanowski, B. Ribeiro, Adaptive and Natural Computing Algorithms Lecture Notes in Computer Science vol. 4431 2007 Springer Berlin/Heidelberg 1 10
47. P. Larrañaga, H. Karshenas, C. Bielza, and R. Santana A review on evolutionary algorithms in Bayesian network learning and inference tasks Inf. Sci. (Ny) 233 2013 109 125
48. X. Llora, and J.M. Garrell Evolution of decision trees Proceedings of the 4th Catalan Conference on Artificial Intelligence 2001 ACIA Press 115 122
49. T. Loveard, V. Ciesielski, Representing classification problems in genetic programming, in: Proceedings of the 2001 IEEE Congress on Evolutionary Computation, vol. 2, 2001, pp. 1070-1077.
50. T. Loveard, V. Ciesielski, Employing nominal attributes in classification using genetic programming, in: L. Wang, K.C. Tan, T. Furuhashi, J.H. Kim, X. Yao (Eds.), Proceedings of the 4th Asia-Pacific Conference on Simulated Evolution And Learning, SEAL'02, Orchid Country Club, Singapore, 2002, pp. 487-491.
51. M.A. Munson A study on the importance of and time spent on different modeling steps SIGKDD Explorat. 13 2011 65 71
52. S.K. Murthy, and S. Salzberg Lookahead and pathology in decision tree induction Proceedings of the 14th International Joint Conference on Artificial Intelligence 1995 Morgan Kaufmann 1025 1031
53. C. Nadeau, and Y. Bengio Inference for the generalization error Mach. Learn. 52 2003 239 281
54. P. Nemenyi, Distribution-Free Multiple Comparisons, Ph.D. Thesis, 1983.
55. S.W. Norton Generating better decision trees Proceedings of the 11th International Joint Conference on Artificial Intelligence, IJCAI'89 1989 Morgan Kaufmann Publishers Inc. San Francisco, CA, USA 800 805
56. D. Page, and S. Ray Skewing: an efficient alternative to lookahead for decision tree induction Proceedings of the 18th International Joint Conference on Artificial Intelligence, IJCAI'03 2003 Morgan Kaufmann Publishers Inc. 601 607
57. A. Papagelis, and D. Kalles Ga tree: genetically evolved decision trees Proceedings of the 2000 IEEE International Conference on Tools with Artificial Intelligence, ICTAI'00 2000 IEEE Computer Society Los Alamitos, CA, USA 203 206
58. A. Papagelis, and D. Kalles Breeding decision trees using evolutionary techniques Proceedings of the 2001 International Conference on Machine Learning, ICML '01 2001 Morgan Kaufmann Publishers Inc. San Francisco, CA, USA 393 400
59. V. Podgorelec, P. Kokol, Self-adapting evolutionary decision support model, in: Proceedings of the IEEE International Symposium on Industrial Electronics, ISIE '99, vol. 3, 1999, pp. 1484-1489.
60. V. Podgorelec, M. Šprogar, and S. Pohorec Evolutionary design of decision trees Wiley Interdisciplinary Rev.: Data Min. Knowledge Discovery 3 2013 63 82
61. J.R. Quinlan Induction of decision trees Mach. Learn. 1 1986 81 106
62. J.R. Quinlan C4.5: Programs for Machine Learning 1993 Morgan Kaufmann Publishers Inc. San Francisco, CA, USA
63. S. Rouwhorst, A. Engelbrecht, Searching the forest: using decision trees as building blocks for evolutionary search in classification databases, in: Proceedings of the 2000 IEEE Congress on Evolutionary Computation, CEC'00, 2000, pp. 633 -638.
64. S.J. Russell Norvig, Artificial Intelligence: A Modern Approach second ed. 2003 Prentice Hall
65. P. Shelokar, A. Quirin, and Ó. Cordón A multiobjective evolutionary programming framework for graph-based data mining Inf. Sci. (Ny) 237 2013 118 136
66. M. Shirasaka, Q. Zhao, O. Hammami, K. Kuroda, K. Saito, Automatic design of binary decision trees based on genetic programming, in: C. Newton (Ed.), Proceedings of the 2nd Asia-Pacific Conference on Simulated Evolution and Learning, SEAL'98, Australian Defence Force Academy, Canberra, Australia, 1998.
67. K. Sorensen, and G.K. Janssens Data mining with genetic algorithms on binary trees Eur. J. Oper. Res. 151 2003 253 264
68. T. Tanigawa, and Q. Zhao A study on efficient generation of decision trees using genetic programming Proceedings of the Genetic and Evolutionary Computation Conference, GECCO'00 2000 Morgan Kaufmann Las Vegas 1047 1052
69. P.E. Utgoff, N.C. Berkman, and J.A. Clouse Decision tree induction based on efficient tree restructuring Mach. Learn. 29 1997 5 44
70. I.H. Witten, and E. Frank Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations 1999 Morgan Kaufmann
71. H. Zhao A multi-objective genetic programming approach to developing pareto optimal decision trees Decis. Support Syst. 43 2007 809 826
72. Q. Zhao, and M. Shirasaka A study on evolutionary design of binary decision trees Proceedings of the 1999 IEEE Congress on Evolutionary Computation, CEC'99 vol. 3 1999 IEEE Press Mayflower Hotel, Washington, DC, USA 1988 1993