An Exact Probability Metric for Decision Tree Splitting and Stopping

Machine Learning

Volumn 28, Issue 2-3, 1997, Pages 257-291

  Martin, J Kent ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Fisher's Exact Test; Hypergeometric; Post pruning; Pre pruning

Indexed keywords

APPROXIMATION THEORY; COMPUTATIONAL COMPLEXITY; DECISION THEORY; HEURISTIC METHODS; PROBABILITY; TREES (MATHEMATICS);

BAYESIAN STATISTICS; DECISION TREE; FISHERS EXACT TEST; HYPERGEOMETRIC DISTRIBUTION;

LEARNING ALGORITHMS;

EID: 0031211834     PISSN: 08856125     EISSN: None     Source Type: Journal    
DOI: 10.1023/a:1007367629006     Document Type: Article

References (44)

1. Agresti, A. (1990). Categorical data analysis. New York: John Wiley & Sons.
2. Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and regression trees. Pacific Grove, CA: Wadsworth & Brooks.
3. Buntine, W. L. (1990). A theory of learning classification rules. PhD thesis. University of Technology, Sydney.
4. Buntine, W. & Niblett, T. (1992). A further comparison of splitting rules for decision-tree induction. Machine Learning, 8, 75-85.
5. Cestnik, B., Kononenko, I. & Bratko, I. (1987). ASSISTANT 86: A knowledge-elicitation tool for sophisticated users. In Progress in Machine Learning, EWSL-87. Wilmslow: Sigma Press.
6. Cestnik, B. & Bratko, I. (1991). On estimating probabilities in tree pruning. In Machine Learning, EWSL-91. Berlin: Springer-Verlag.
7. 2 tests. Biometrics, 10, 417-451.
8. Elder, J. F. (1995). Heuristic search for model structure. In Fisher, D. & Lenz, H-J. (Eds.) Learning from Data: Artificial Intelligence and Statistics V, Lecture Notes in Statistics, v. 112 (pp. 131-142). New York: Springer.
9. Fayyad, U. M., & Irani, K. B. (1992a). The attribute selection problem in decision tree generation. Proceedings of the 10th National Conference on Artificial Intelligence, AAAI-92 (pp. 104-110). Cambridge, MA: MIT Press.
10. Fayyad, U. M., & Irani, K. B. (1992b). On the handling of continuous-valued attributes in decision tree generation. Machine Learning, 8, 87-102.
11. Fayyad, U. M., & Irani, K. B. (1993). Multi-interval discretization of continuous-valued attributes for classification learning. Proceedings of the 13th International Joint Conference on Artificial Intelligence (IJCAI-93) (pp. 1022-1027). San Mateo, CA: Morgan Kaufmann.
12. Fisher, D. H. (1987). Knowledge acquisition via incremental conceptual clustering. Machine Learning, 2, 139-172.
13. Fisher, D. H. (1992). Pessimistic and optimistic induction. Technical Report CS-92-22, Department of Computer Science, Vanderbilt University, Nashville, TN.
14. Fisher, D. H., & Schlimmer, J. C. (1988). Concept simplification and prediction accuracy. Proceedings of the 5th International Conference on Machine Learning (ML-88) (pp. 22-28). San Mateo, CA: Morgan-Kaufmann.
15. Fulton, T., Kasif, S., & Salzberg, S. (1995). Efficient algorithms for finding multi-way splits for decision trees. Machine Learning: Proceedings of the 12th International Conference (ML-95) (pp. 244-251). San Francisco: Morgan Kaufmann.
16. Gluck, M. A., & Corter, J. E. (1985). Information, uncertainty, and the utility of categories. Proceedings of the 7th Annual Conference of the Cognitive Society (pp. 283-287). Hillsdale, NJ: Lawrence Erlbaum.
17. John, G. H. (1995). Robust linear discriminant trees. In Fisher, D. & Lenz, H-J. (Eds.) Learning from Data: Artificial Intelligence and Statistics V, Lecture Notes in Statistics, v. 112 (pp. 375-386). New York: Springer.
18. Kira, K. & Rendell, L. A. (1992). A practical approach to feature selection. Machine Learning: Proceedings of the 9th International Conference (ML-92) (pp. 249-256). San Mateo, CA: Morgan Kaufmann.
19. Kononenko, I. (1994). Estimating attributes: Analysis and extensions of RELIEF. Proceedings of the European Conference on Machine Learning (ECML-94), (pp. 171-182). Berlin: Springer.
20. Liu, W. Z., & White, A. P. (1994). The importance of attribute-selection measures in decision tree induction. Machine Learning, 15, 25-41.
21. Lopez de Mantaras, R. (1991). A distance-based attribute selection measure for decision tree induction. Machine Learning, 6, 81-92.
22. Martin, J. K. (1995). An exact probability metric for decision tree splitting and stopping. Technical Report 95-16, Department of Information & Computer Science, University of California, Irvine, CA.
23. Martin, J. K. & Hirschberg, D. S. (1996a). On the complexity of learning decision trees. Proceedings Fourth International Symposium on Artificial Intelligence and Mathematics, AI/MATH-96 (pp. 112-115). Fort Lauderdale, FL.
24. Martin, J. K. & Hirschberg, D. S. (1996b). Small sample statistics for classification error rates I: Error rate measurements. Technical Report 96-21, Department of Information & Computer Science, University of California, Irvine, CA.
25. Martin, J. K. & Hirschberg, D. S. (1996c). Small sample statistics for classification error rates II: Confidence intervals and significance tests. Technical Report 96-22, Department of Information & Computer Science, University of California, Irvine, CA.
26. Mingers, J. (1987). Expert systems - rule induction with statistical data. Journal of the Operational Research Society, 38, 39-47.
27. Mingers, J. (1989a). An empirical comparison of pruning measures for decision tree induction. Machine Learning, 4, 227-243.
28. Mingers, J. (1989b). An empirical comparison of selection measures for decision tree induction. Machine Learning, 3, 319-342.
29. Murphy, P. M., & Aha, D. W. (1995). UCI Repository of Machine Learning Databases. (machine-readable data depository). Department of Information & Computer Science, University of California, Irvine, CA.
30. Murphy P. M. & Pazzani, M. J. (1991). ID2-of-3: Constructive induction of M-of-N concepts for discriminators in decision trees. Machine Learning: Proceedings of the 8th International Workshop (ML-91) (pp. 183-187). San Mateo, CA: Morgan Kaufmann.
31. Murthy, S., Kasif, S., Salzberg, S., & Beigel, R. (1993). OC-1: Randomized induction of oblique decision trees. Proceedings of the 11th National Conference on Artificial Intelligence (AAAI-93) (pp. 322-327). Menlo Park, CA: AAAI Press.
32. Murthy, S. & Salzberg, S. (1995). Lookahead and pathology in decision tree induction. Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI-95) (pp. 1025-1031). San Mateo, CA: Morgan Kaufmann.
33. Niblett, T. (1987). Constructing decision trees in noisy domains. In Progress in Machine Learning, EWSL-87. Wilmslow: Sigma Press.
34. Niblett, T., & Bratko, I. (1986). Learning decision rules in noisy domains. In Proceedings of Expert Systems 86. Cambridge: Cambridge University Press.
35. Park, Y. & Sklansky, J. (1990). Automated design of linear tree classifiers. Pattern Recognition, 23, 1393-1412.
36. Quinlan, J. R. (1986). Induction of decision trees. Machine Learning, 1, 81-106.
37. Quinlan, J. R. (1988). Simplifying decision trees. In B. R. Gaines & J. H. Boose (Eds.). Knowledge Acquisition for Knowledge-Based Systems. San Diego: Academic Press.
38. Quinlan, J. R. (1993). C4.5: Programs for Machine Learning. San Mateo, CA: Morgan Kaufmann.
39. Quinlan, J. R. & Cameron-Jones, R.M. (1995). Oversearching and layered search in empirical learning. Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI-95) (pp. 1019-1024). San Mateo, CA: Morgan Kaufmann.
40. Schaffer, C. (1993). Overfitting avoidance as bias. Machine Learning, 10, 153-178.
41. Shavlik, J. W., Mooney, R. J., & Towell, G. G. (1991). Symbolic and neural learning algorithms: An experimental comparison. Machine Learning, 6, 111-143.
42. Weiss, S. M. & Indurkhya, N. (1991). Reduced complexity rule induction. Proceedings of the 12th International Joint Conference on Artificial Intelligence, IJCAI-91 (pp. 678-684). San Mateo, CA: Morgan Kaufmann.
43. Weiss, S. M. & Indurkhya, N. (1994). Small sample decision tree pruning. Proceedings of the 11th International Conference on Machine Learning (ML-94), (pp. 335-342). San Francisco: Morgan-Kaufman.
44. White, A. P. & Liu, W. Z. (1994). Bias in information-based measures in decision tree induction. Machine Learning, 15, 321-329.