A Survey of Decision Tree Classifier Methodology

IEEE Transactions on Systems, Man and Cybernetics

Volumn 21, Issue 3, 1991, Pages 660-674

  Safavian, S Rasoul ^a   Landgrebe, David ^a  

^a PURDUE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

INFORMATION THEORY; NEURAL NETWORKS; OPTIMIZATION; PROBABILITY;

DECISION TREE CLASSIFIERS; ENTROPY REDUCTION; INCREMENTAL TREE DESIGN; MISSING VALUE PROBLEM;

DECISION THEORY AND ANALYSIS;

EID: 0026154509     PISSN: 00189472     EISSN: 21682909     Source Type: Journal    
DOI: 10.1109/21.97458     Document Type: Article

References (99)

1. J. Aczel and J. Daroczy, On Measures of Information and Their Characterizations. New York: Academic, 1975.
2. R. Ahlsmede and I. Wegeru, Search Problems. New York: Wiley-Interscience. 1987.
3. A. V. Aho, J. E. Hopcroft, and J. D. Ullmann, “The Design and Analysis of Computer Algorithm”. Reading, MA: Addison-Wesley, 1974.
4. P. Argentiero, R. Chin, and P. Beaudet, “An automated approach to the design of decision tree classifiers,” IEEE Trans. Patt. Analy. Mach. Intell., vol. PAMI-4, pp. 51-57, 1982.
5. L. Atlas et al., “A performance comparison of trained multilayer perceptrons and trained classification trees,” Proc. IEEE, vol. 78, pp. 1614-1619, 1990.
6. L. A. Bartolucci, P. H. Swain, and C. Wu, “Selective radiant temperature mapping using a layered classifier,” IEEE Trans. Geosci. Electron., vol. GE-14, pp. 101-106, 1976.
7. L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, Classification and Regression Trees. Belmont, CA: Wadsworth Int., 1984.
8. R. C. Casey and G. Nagy, “Decision tree design using a probabilistic model,” IEEE Trans. Inform. Theory, vol. IT-30, pp. 93-99, 1984.
9. R. L. P. Chang, “Application of fuzzy decision techniques to pattern recognition and curve fitting,” Ph.D. dissertation, Dep. of EECS, Princeton Univ., Princeton, NJ, 1976.
10. R. L. P. Chang and T. Pavlidis, “Fuzzy decision tree algorithms,” IEEE Trans. Syst., Man, Cybern., vol. SMC-7, pp. 28-35, 1977.
11. P. A. Chou and R. M. Gray, “On decision trees for pattern recognition,” in Proc. IEEE Symp. Inform. Theory, Ann Arbor, MI, p. 69, 1986.
12. M. Chou, T. Lookabaugh, and R. M. Gray, “Optimal pruning with applications to tree structured source coding and modeling,” IEEE Trans. Inform. Theory, vol. T35, pp. 299-315, 1989.
13. G. R. Dattatreya and L. N. Kanal, “Adaptive pattern recognition with random costs and its application to decision trees,” IEEE Trans. Syst., Man, Cybern., vol. SMC-16, pp. 208-218, 1986.
14. G. R. Dattatreya and L. N. Kanal, “Decision trees in pattern recognition,” in Progress in Pattern Recognition 2, Kanal and Rosenfeld, Eds., Elsevier Science, pp. 189-239, 1985.
15. G. R. Dattatreya and V. V. S. Sarma, “Decision tree design for pattern recognition including feature measurement cost,” in Proc. 5th Int. Conf. Pattern Recognition, vol. II, pp. 1212-1214, 1980.
16. G. R. Dattatreya and V. V. S. Sarma, “Bayesian and decision tree approaches for pattern recognition including feature measurement costs,” IEEE Trans. Patt. Anal. Mach. Intell., vol. PAMI-3, pp. 293-298, 1981.
17. E. Diday and J.V. Moreau, “Learning hierarchical clustering from examples-Applications to the adaptive construction of dissimilarity indices,” Pattern Recognition Lett., pp. 223-230, 1986.
18. R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis. New York: Wiley-Interscience, 1973.
19. P. Fletcher and M.J.D. Powell, “A rapid decent method for minimization,” Computer J., vol. 6, ISS.2, pp. 163-168, 1963.
20. J. H. Friedman, “A variable metric decision rule for nonparametric classification,” Stanford Linear Accelerator Center-PUB-1573, CS-75-487, Apr. 1975.
21. “A recursive partitioning decision rule for nonparametric classifer,” IEEE Trans. Comput., vol. C-26, pp. 404-408, 1977.
22. K. S. Fu, Sequential Methods in Pattern Recognition and Machine Learning. New York: Academic, 1968.
23. K. Fukunaga, Introduction to Statistical Pattern Recognition. New York: Academic, 1972.
24. K. Fukunaga and P. M. Narendra, “A branch and bound algorithm for computing k-nearest neighbors,” IEEE Trans. Comput., vol. C-24, pp. 750-753, 1975.
25. S.B. Gelfand, C.S. Ravishankar, and E.J. Delp, “An iterative growing and pruning algorithm for classification tree design,” IEEE Trans. Patt. Anal. Mach. Intell., pp. 163-174, 1991.
26. B. V. Gnedenko and V. S. Koreljuk, “On the maximum discrepancy between two empirical distributions,” Dokl. Acad. Nauk., vol. SSSR 4, pp. 525-528, 1951.
27. M. Golea and M. Marchand, “A growth algorithm for neural networks,” Europhys. Lett., vol. 12, no. 3, pp. 205 - 210, 1990.
28. S. W. Golomb and L. D. Bavmert, “Backtrack programming,” J. Ass. Comput. Mach., vol. 12, pp. 516-524, 1965.
29. R. M. Goodman and P. Smyth, “Decision tree design from a communication theory standpoint,” IEEE Trans. Inform. Theory, vol. IT-34, pp. 979-994, 1988.
30. Y. X. Gu, Q. R. Wang, and C. Y. Suen, “Application of multilayer decision tree in a computer recognition of Chinese characters,” IEEE Trans. Patt. Analy. Mach. Intell., vol. PAMI-5, pp. 83-89, 1983.
31. S. Gelfand and H. Guo, “Tree classifiers with multilayer perceptron feature extraction,” Ph. D. dissertation, School of Elec. Eng., Purdue Univ., W. Lafayette, IN, May 1991.
32. D. E. Gustafson, S. B. Gelfand, and S. K. Mitter, “A nonparametric multiclass partitioning methods for classification,” in Proc. 5th Int. conf. pattern Recognition, pp. 654-659, 1980.
33. P. A. V. Hall, “Branch-and-Bound and Beyond,” in Proc. 2nd Joint Int. Conf. Artificial Intelligence, 1971.
34. R. M. Haralick, “The table lookup rule,” in Proc. Conf. Pattern Recognition, p. 447, 1976.
35. P. E. Hart, “Searching probabilistic decision trees,” AI Group Tech. Note No. 2, SRI project 7494, SIR, Stanford, CA, 1969.
36. C. R. P. Hartmann, P. K. Varshney, K. G. Mehrotra, and C. L. Gerberich, “Application of information theory to the construction of efficient decision trees,” IEEE Trans. Inform. Theory, vol. IT-28, pp. 565-577, 1982.
37. E. G. Henrichon, Jr. and K. S. Fu, “A nonparametric partitioning procedure for pattern classification,” IEEE Trans. Comput., vol. C-18, pp. 604-624, 1969.
38. G. F. Hughes, “On the mean accuracy of statistical pattern recognizers,” IEEE Trans. Inform. Theory, vol. IT-14, pp. 55-63, 1968.
39. L. Hyafil and R. L. Rivest, “Constructing optimal binary decision trees is NP-complete,” Inform. Processing Lett., vol. 5, no. 1, pp. 15-17, 1976.
40. L. N. Kanal, “problem-Solving methods and search strategies for pattern recognition,” IEEE Trans. Patt. Anal. Mach. Intell., vol. PAMI-1, pp. 193-201, 1979.
41. B. Kim and D. A. Landgrebe, “Hierarchical decision tree classifiers in high-dimensional and large class data,” Ph.D. dissertation and Tech. Rep. TREE-90-47, School of Elec. Eng., Purdue Univ., W. Lafayette, IN, 1990.
42. P. R. Krishnaiah, Ed., On Hierarchical Classifier and Interactive Design, in Applications of Statistics. Amsterdam, The Netherlands: North Holland, 1971, pp. 301-321.
43. D.E. Knuth, “Optimum binary search trees,” ACTA Inform., vol. 1, pp. 14-25, 1971.
44. D.E. Knuth, “The art of computer programming, 1: Fundamental algorithms,” Reading, MA: Addison-Wesley, 1968.
45. A. V. Kulkarni and L. N. Kanal, “An optimization approach to hierarchical classifier design,” in Proc. 3rd Int. Joint Conf. Pattern Recognition, San Diego, CA, 1976.
46. A.V. Kulkarni and L.N. Kanal, “Admissible search strategies for parametric and non-parametric hierarchical classifiers,” in Proc. 4th Int. Conf. Pattern Recognition, Kyoto, Japan, 1978.
47. A. V. Kulkarni, “On the mean accuracy of hierarchical classifiers,” IEEE Trans. Comput., vol. C-27, pp. 771-776, 1978.
48. A. V. Kulkarni, “Optimal and heuristic synthesis of hierarchical classifiers,” Ph.D. dissertation, Univ. Maryland, College Park, Comput. Sci. Tech. Rep. TR-469, 1976.
49. M. W. Kurzynski, “Decision rules for a hierarchical classifier,” Pattern Recog. Lett, vol. 1, pp. 305-310, 1983.
50. M. W. Kurzynski, “The optimal strategy of a tree classifier,” Pattern Recog., vol. 16, pp. 81-87, 1983.
51. E.L. Lawler and D.E. Wood, “Brand-and-bound: A survey,”Operation Res., vol. 14, 1966.
52. G. Landeweerd, T. Timmers, E. Gersema, M. Bins and M. Halic, “Binary tree versus single level tree classification of white blood cells,” Pattern Recog., vol. 16, pp. 571-577, 1983.
53. X. Li and R. C. Dubes, “Tree classifier design with a Permutation statistic,” Pattern Recog., vol. 19, pp. 229-235, 1986.
54. Y. K. Lin and K. S. Fu, “Automatic classification of cervical cell using a binary tree classifier,” Pattern Recog., vol. 16, pp. 69-80, 1983.
55. R. P. Lippmann, “An introduction to computing with neural nets,” IEEE ASSP Mag., pp. 4-22, Apr. 1987.
56. C. McMillan, Jr., Mathematical Programming. New York: Wiley, 1975, ch. 8.
57. W. S. Meisel and D. A. Michalopoulos, “A Partitioning algorithm with application in pattern classification and optimization of decision trees,” IEEE Trans. Comput., vol. C-22, pp. 93-103, Jan. 1973.
58. J. Mui and K. S. Fu, “Automated classification of nucleated blood cells using a binary tree classifier,” IEEE Trans. Patt. Anal. Mach. Intell., vol. PAMI-2, pp. 429-443, 1980.
59. M. Nadler, “Error and reject rates in a hierarchical pattern recognizer,” IEEE Trans. Comput., vol. C-19, pp. 1598-1601, 1970.
60. N. J. Nilsson, problem-Solving Methods in Artificial Intelligence. New York: McGraw-Hill, 1971.
61. H. Payne and W. Meisel, “An algorithm for constructing optimal binary decision trees,” IEEE Trans. Comput., vol. C-26, pp. 905-916, 1977.
62. J.R. Ouinlan, “Induction of decision trees,” Machine Learning, vol. 1, pp. 81-106, 1986.
63. “Decision trees and decision-making,” IEEE Trans. Syst., Man., Cybern., vol. SMC-20, pp. 339-346, 1990.
64. J.R. Quinlan and R.L. Rivest, “Inferring decision trees using the minimum description length principle,” Information and Computation, vol. 80, no. 3, pp. 227-248, 1989.
65. E. Rounds, “A combined non-parametric approach to feature selection and binary decision tree design,” Pattern Recog., vol. 12, pp. 313-317, 1980.
66. E. Rounds, “Computation of two-sample Kolmogorov-Smirnov statistic,” Project Memo TSC-PM-A 142-5, Technology Service Corp., Santa Monica, CA, 1978.
67. D. E. Rumelhart and J. L. McClelland, Eds., Parallel Distributed Processing, vol. 1. Cambridge, MA:, MIT Press, 1986.
68. J. Schuermann and W. Doster, “A decision-theoretic approach in hierarchical classifier design,” Pattern Recog., vol. 17, pp. 359-369, 1984.
69. I. K. Sethi, “Entropy nets: From decision trees to neural networks,” Proc. IEEE, vol. 78, pp. 1605-1613, Oct. 1990.
70. “Layered neural net design through decision trees,” in Proc. Int. Symp. Circuits and Syst., New Orleans, LA, May 1 -3, 1990.
71. I. K. Sethi and B. Chatterjee, “Efficient decision tree design for discrete variable pattern recognition problems,” Pattern Recog., vol. 19, pp. 197-206, 1977.
72. I. K. Sethi and M. Otten, “Comparison between entropy net and decision tree classifiers,” in Proc. Int. Joint Conf. Neural Net. IJCNN, San Diego, CA 1989.
73. I. K. Sethi and G. Sarvarayudu, “Hierarchical classifier design using mutual information,” IEEE Trans Patt. Anal. Mach. Intell., vol. PAMI-4, pp. 441-445, 1982.
74. Q. Y. Shi, “A method for the design of binary tree classifiers,” in Proc. IEEE Conf. Image Processing and Pattern Recog., pp. 21-26, 1981.
75. C. R. Smith and W. T. Grandy, Jr., eds., Maximum-Entropy and Bayesian Methods in Inverse Problems. The Netherlands: Reidel, 1985.
76. C.Y. Suen and Q.R. Wang, “ISOETRP-An interactive clustering algorithm with new objectives,” Pattern Recog., vol. 17, pp. 211-219, 1984.
77. P. Swain and H. Hauska, “The decision tree classifier design and potential,” IEEE Trans. Geosci. Electron, vol. GE-15, pp. 142-147, 1977.
78. J. L. Talmon, “A multiclass nonparametric partitioning algorithm,” in E. S. Gelsema and L. N. Kanal, Eds., Pattern Recog. in Practice II. Amsterdam, The Netherlands: Elsevier Science, 1986.
79. P. E. Utgoff, “Incremental induction of decision trees,” Machine Learning, vol. 4, pp. 161-186, 1989.
80. A. Wald, Sequential Analysis. New York: Wiley, 1947.
81. Q.R. Wang and C.Y. Suen, “Large tree classifier with heuristic search and global training,” IEEE Trans. Patt. Anal. Mach. Intell., vol. PAMI-9, pp. 91-102, 1987.
82. Q. R. Wang and C. Y. Suen, “Analysis and design of a decision tree based on entropy reduction and its application to Large character set recognition,” IEEE Trans. Patt. Anal. Mach. Intell., vol. PAMI-6, pp. 406-417, 1984.
83. Q.R. Wang and C.Y. Suen, “Classification of Chinese characters by phase feature and fuzzy logic search,” in Proc. Int. Conf. Chinese Inform. Processing, vol. 1, Beijing, China, pp. 133-155, 1983.
84. Q.R. Wang, Y.X. Gu, and C.Y. Suen, “A preliminary study on computer recognition of Chinese characters printed in different fonts,” in Proc. Int. Conf. Chinese Lang. Comput. Soc., pp. 344-351, 1982.
85. Watanabe, “Pattern recognition as conceptual morphogenesis,” IEEE Trans. Patt. Anal. Mach. Intell., vol. PAMI-2, pp. 161-165, 1980.
86. S. Watanabe, Knowing and Guessing. New York: Wiley, 1969.
87. Pattern Recognition: Human and Machine. New York: Wiley-Interscience, 1985.
88. “Learning process and inverse H-theorem,” IRE Trans. Inform. Theory, vol. IT-8, p. 246, 1962.
89. “Information-theoretical aspects of inductive and deductive inference,” IBM J. Res. Develop., vol. 4, p. 208, 1960.
90. “Application of dynamical coalescence model of clustering,” in Proc. 3rd Int. Joint Conf. Pattern Recog., San Diego, CA, p. 176, 1976.
91. “Karhunen-Loeve expansion and factor analysis,” in Trans. 4th Prague Conf. Inform. Theory, 1965.
92. S. Watanabe and E. T. Harada, “A dynamical model of clustering,” in Proc. 2nd Int. Joint Conf. Pattern Recog., Copenhagen, Denmark, p. 413, 1974.
93. S. Watanabe, “Pattern recognition as a quest for minimum entropy,” Pattern Recog., vol. 13, p. 381-387, 1981.
94. R. Winston, “A heuristic program that constructs decision trees,” MIT project MAC, Memo #173, 1969.
95. C. Wu, D. Landgrebe, and P. Swain, “The decision tree approach to classification,” School Elec. Eng., Purdue Univ., W. Lafayette, IN, Rep. RE-EE 75-17, 1975.
96. S. S. Yau and J. M. Garnet, “Least mean square approach to pattern classification,” in Frontier of Pattern Recognition, New York: Academic, 1972, pp. 575-588.
97. K. C. You and K. S. Fu, “An approach to the design of a linear binary tree classifier,” in Proc. 3rd Symp. Machine Processing of Remotely Sensed Data, Purdue Univ., W. Lafayette, IN, 1976.
98. S. Q. Yun and K. S. Fu, “A method for the design of binary tree classifiers,” Pattern Recog., vol. 16, pp. 593-603, 1983.
99. L. A. Zadeh, “Fuzzy sets,” Inform. Contr., pp. 338-353, 1965.