Further experimental evidence against the utility of occam's razor

Journal of Artificial Intelligence Research

Volumn 4, Issue , 1996, Pages 397-417

  Webb, Geoffrey I ^a  

^a DEAKIN UNIVERSITY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

DECISION TREES; OCCAM RAZOR;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTATIONAL COMPLEXITY; DATA STRUCTURES; DECISION THEORY;

LEARNING SYSTEMS;

EID: 0029679031     PISSN: 10769757     EISSN: None     Source Type: Journal    
DOI: 10.1613/jair.228     Document Type: Article

References (36)

1. Aha, D. W., Kibler, D., & Albert, M. K. (1991). Instance-based learning algorithms. Machine Learning, 6, 37-66.
2. Ali, K., Brunk, C., & Pazzani, M. (1994). On learning multiple descriptions of a concept. In Proceedings of Tools with Artificial Intelligence New Orleans, LA.
3. Berkman, N. C., & Sandholm, T. W. (1995). What should be minimized in a decision tree: A re-examination. Technical report 95-20, University of Massachusetts at Amherst, Computer Science Department, Amherst, Mass.
4. Blumer, A., Ehrenfeucht, A., Haussler, D., & Warmuth, M. K. (1987). Occam's Razor. Information Processing Letters, 24, 377-380.
5. Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and Regression Trees, Wadsworth International, Belmont, Ca.
6. Brodley, C. E. (1995). Automatic selection of split criterion during tree growing based on node selection. In Proceedings of the Twelth International Conference on Machine Learning, pp. 73-80 Taho City, Ca. Morgan Kaufmann.
7. Bunge, M. (1963). The Myth of Simplicity. Prentice-Hall, Englewood Cliffs, NJ.
8. Clark, P., & Niblett, T. (1989). The CN2 induction algorithm. Machine Learning, 3, 261-284.
9. Fayyad, U. M., & Irani, K. B. (1990). What should be minimized in a decision tree? In AAAI-90: Proceedings Eighth National Conference on Artificial Intelligence, pp. 749-754 Boston, Ma.
10. Good, I. J. (1977). Explicativity: A mathematical theory of explanation with statistical applications. Proceedings of the Royal Society of London Series A, 354, 303-330.
11. Holte, R. C. (1993). Very simple classification rules perform well on most commonly used datasets. Machine Learning, 11(1), 63-90.
12. Holte, R. C., Acker, L. E., & Porter, B. W. (1989). Concept learning and the problem of small disjuncts. In Proceedings of the Eleventh International Joint Conference on Artificial Intelligence, pp. 813-818 Detroit. Morgan Kaufmann.
13. Lubinsky, D. J. (1995). Increasing the performance and consistency of classification trees by using the accuracy criterion at the leaves. In Proceedings of the Twelth International Conference on Machine Learning, pp. 371-377 Taho City, Ca. Morgan Kaufmann.
14. Michalski, R. S. (1984). A theory and methodology of inductive learning. In Michalski, R. S., Carbonell, J. G., & Mitchell, T. M. (Eds.), Machine Learning: An Artificial Intelligence Approach, pp. 83-129. Springer-Verlag, Berlin.
15. Murphy, P. M. (1995). An empirical analysis of the benefit of decision tree size biases as a function of concept distribution. Tech. rep. 95-29, Department of Information and Computer Science, University of California, Irvine.
16. Murphy, P. M., & Aha, D. W. (1993). UCI repository of machine learning databases. [Machine-readable data repository]. University of California, Department of Information and Computer Science, Irvine, CA.
17. Murphy, P. M., & Pazzani, M. J. (1994). Exploring the decision forest: An empirical investigation of Occam's razor in decision tree induction. Journal of Artificial Intelligence Research, 1, 257-275.
18. Niblett, T., & Bratko, I. (1986). Learning decision rules in noisy domains. In Bramer, M. A. (Ed.), Research and Development in Expert Systems III, pp. 25-34. Cambridge University Press, Cambridge.
19. Nock, R., & Gascuel, O. (1995). On learning decision committees. In Proceedings of the Twelth International Conference on Machine Learning, pp. 413-420 Taho City, Ca. Morgan Kaufmann.
20. Oliver, J. J., & Hand, D. J. (1995). On pruning and averaging decision trees. In Proceedings of the Twelth International Conference on Machine Learning, pp. 430-437 Taho City, Ca. Morgan Kaufmann.
21. Pearl, J. (1978). On the connection between the complexity and credibility of inferred models. International Journal of General Systems, 4, 255-264.
22. Quinlan, J. R. (1986). Induction of decision trees. Machine Learning, 1, 81-106.
23. Quinlan, J. R. (1990). Learning logical definitions from relations. Machine Learning, 5, 239-266.
24. Quinlan, J. R. (1991). Improved estimates for the accuracy of small disjuncts. Machine Learning, 6, 93-98.
25. Quinlan, J. R. (1993). C4.5: Programs For Machine Learning. Morgan Kaufmann, Los Altos.
26. Rao, R. B., Gordon, D., & Spears, W. (1995). For every generalization action is there really an equal and opposite reaction? Analysis of the conservation law for generalization performance. In Proceedings of the Twelth International Conference on Machine Learning, pp. 471-479 Taho City, Ca. Morgan Kaufmann.
27. Rendell, L., & Seshu, R. (1990). Learning hard concepts through constructive induction: Framework and rationale. Computational Intelligence, 6, 247-270.
28. Rissanen, J. (1983). A universal prior for integers and estimation by minimum description length. Annals of Statistics, 11, 416-431.
29. Rissanen, J. (1987). Stochastic complexity. Journal of the Royal Statistical Society Series B, 49(3), 223-239.
30. Schaffer, C. (1992). Sparse data and the effect of overfitting avoidance in decision tree induction. In AAAI-92: Proceedings of the Tenth National Conference on Artificial Intelligence, pp. 147-152 San Jose, CA. AAAI Press.
31. Schaffer, C. (1993). Overfitting avoidance as bias. Machine Learning, 10, 153-178.
32. Schaffer, C. (1994). A conservation law for generalization performance. In Proceedings of the 1994 International Conference on Machine Learning San Mateo, Ca. Morgan Kaufmann.
33. Ting, K. M. (1994). The problem of small disjuncts: Its remedy in decision trees. In Proceedings of the Tenth Canadian Conference on Artificial Intelligence, pp. 63-70. Morgan Kaufmann,.
34. Wallace, C. S., &: Boulton, D. M. (1968). An information measure for classification. Computer Journal, 11, 185-194.
35. Wallace, C. S., & Freeman, P. R. (1987). Estimation and inference by compact coding. Journal of the Royal Statistical Society Series B, 49(3), 240-265.
36. Webb, G. I. (1994). Generality is more significant than complexity: Toward alternatives to Occam's razor. In Zhang, C., Debenham, J., & Lukose, D. (Eds.), AI'94 - Proceedings of the Seventh Australian Joint Conference on Artificial Intelligence, pp. 60-67 Armidale. World Scientific.