A heuristic for learning decision trees and pruning them into classification rules

AI Communications

Volumn 16, Issue 2, 2003, Pages 71-87

  Ranilla, José ^a   Luaces, Oscar ^a   Bahamonde, Antonio ^a  

^a UNIVERSITY OF OVIEDO   (Spain)

Author keywords

Classification rules; Decision trees; Impurity level; Machine learning; Pruning

Indexed keywords

ALGORITHMS; DECISION THEORY; TREES (MATHEMATICS);

DECISION TREES;

LEARNING SYSTEMS;

EID: 0038340808     PISSN: 09217126     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (36)

1. D. Aha, A Study of instance-based algorithms for Supervised Learning Tasks: Mathematical. Empirical, and Psychological Evaluations, PhD thesis, University of California at Irvine, 1990.
2. D. Aha, D.W. Kibler and M.K. Albert, Instance-based learning algorithms, Machine Learning 6 (1991), 37-66.
3. C.L. Blake and C.J. Merz, UCI repository of machine learning databases. [http://www.ics.uci.edu/̃mlearn/MLRepository. html]. University of California, Irvine, Dept. of Information and Computer Sciences, 1998.
4. C. Borgelt, J. Gebhardt and R. Kruse, Concepts for probabilistic and possibilistic induction of decision trees on real world data, in: Proc. of the EUFIT '96, Volume 3, 1996, pp. 1556-1560.
5. L. Breiman, J,H. Friedman, R.A. Olshen and C.J. Stone, Classification and Regression Trees, Wadsworth International Group, Belmont, CA, 1984.
6. L.A. Breslow and D.W. Aha, Simplifying decision trees: a survey, Knowledge Engineering Review 12(1) (1997), 1-40.
7. W.W. Cohen, Efficient pruning methods for separate-and-conquer rule learning systems, in: Proceedings of the 13th International Joint Conference on Artificial Intelligence, 1993, pp. 988-994.
8. W.W. Cohen, Fast effective rule induction, in: International Conference on Machine Learning, 1995, pp. 115-123.
9. J.J. del Coz, O. Luaces, J.R. Quevedo, J. Alonso, J. Ranilla and A. Bahamonde, Self-organizing cases to find paradigms, in: Proceedings of the International Work-Conference on Artificial and Natural Neural Networks, IWANN '99, Volume 1606 of LNCS, Springer-Verlag, 1999, pp. 527-536.
10. J.J. del Coz, BETS: Sistema de aprendizaje basado en la selección de ejemplos paradigmáticos, PhD thesis, Dept. of Computer Science, University of Oviedo at Gijón, 2000.
11. T. Dietterich, M. Kearns and Y. Mansour, Applying the weak learning framework to understand and improve C4.5, in: Proc. 13th International Conference on Machine Learning, Morgan Kaufmann, 1996, pp. 96-104.
12. P. Domingos, Unifying instance-based and rule-based induction, Machine Learning 24 (1996), 141-168.
13. J. Fürnkranz, Pruning algorithms for rule learning, Machine Learning 27(2) (1997), 139-171.
14. J. Fürnkranz and G. Widmer, Incremental reduced error pruning, in: International Conference on Machine Learning, 1994, pp. 70-77.
15. R.C. Holte, Very simple classification rules perform well on most commonly used datasets, Machine Learning 11 (1993), 63-91.
16. E.B. Hunt, J. Marin and P.T. Stone, Experiments in Induction, Academic Press, New York, NY, 1966.
17. L. Hyafil and R.L. Rivest, Constructing optimal binary decision trees is NP-complete, Information Processing Letters 5(1) (1976), 15-17.
18. ++, in: Proceedings of the Sixth International Conference on Tools with Artificial Intelligence, IEEE Computer Society Press, 1994, pp. 740-743.
19. R. Kohavi, A study of cross-validation and Bootstrap for accuracy estimation and model selection, in: IJCAI, 1995, pp. 1137-1145.
20. O. Luaces, Un Sistema de aprendizaje de reglas explicitas mediante la Generalization de Instancias, PhD thesis, Department of Computer Science, University of Oviedo at Gijón, 1999.
21. O. Luaces and A. Bahamonde, Inflating examples to obtain rules, Technical report, Artificial Intelligence Center, University of Oviedo at Gijón, 2000.
22. O. Luaces, J.J. del Coz, J.R. Quevedo, J. Alonso, J. Ranilla and A. Bahamonde, Autonomous clustering for machine learning, in: Proceedings of the International Work-Conference on Artificial and Natural Neural Networks, IWANN '99, Volume 1606 of Lecture Notes in Computer Sciences, Alicante, Spain, Springer-Verlag, 1999, pp. 497-506.
23. M. Matsumoto and T. Nishimura. Mersenne twister: A 623-dimensionally equidistributed uniform pseudorandom number generator, ACM Transactions on Modeling and Computer Simulation 8(1) (1998), 3-30.
24. J.R. Quinlan, Learning efficient classification procedures and their application to chess end games, in: Machine Learning: An Artificial Intelligence Approach, R.S. Michalski, J.G. Carbonell and T.M. Mitchell, eds, Tioga, Palo Alto, 1983, pp. 463-482.
25. J.R. Quinlan, C4.5: Programs for Machine Learning, Morgan Kaufmann Publishers, San Mateo, California, 1993.
26. J.R. Quinlan, Improved use of continuous attributes in C4.5, Journal of Artificial Intelligence Research 4 (1996), 77-90.
27. J. Ranilla and A. Bahamonde, FAN: Finding Accurate iNductions, International Journal of Human Computer Studies 56(4) (2002), 445-474.
28. J. Ranilla, R. Mones and A. Bahamonde, El nivel de impureza de una regla de clasincación aprendida a partir de ejemplos, Novática 131 (1998), 37-43.
29. J. Ranilla, ABANICO: Aprendizaje Basado en Agrupación numérica en intervalos continuos, PhD thesis, Department of Computer Science, University of Oviedo at Gijón, 1998.
30. L. Rendell, A general framework for induction and a study of selective induction, Machine Learning 1(2) (1986), 177-226.
31. J. Rissanen, A universal prior for integers and estimation by minimum description length, The Annals of Statistics 11(2) (1983), 416-431.
32. S. Salzberg, Learning with Nested Generalized Exemplars, Kluwer Academic Publishers, Boston, MA, 1990.
33. C.E. Shannon, A mathematical theory of communication, Bell Systems Technical Journal 27 (1948), 379-423 (Part I), 623-656 (Part II).
34. L. Todorovski, P. Flach and N. Lavrač, Predictive performance of weighted relative accuracy, in: 4th European Conference on Principles of Data Mining and Knowledge Discovery (PKDD2000), D.A. Zighed, J. Komorowski and J. Zytkow, eds, Springer-Verlag, 2000, pp. 255-264.
35. D. Wettschereck and T.G. Dietterich, An experimental comparison of the nearest-neighbor and nearest-hyperrectangle algorithms, Machine Learning 19 (1995), 5-27.
36. D.R. Wilson and T.R. Martínez, Improved heterogeneous distance functions, Journal of Artificial Intelligence Research 6(1) (1997), 1-34.