Occam's Two Razors: The Sharp and the Blunt

Proceedings of the 4th International Conference on Knowledge Discovery and Data Mining, KDD 1998

Volumn , Issue , 1998, Pages 37-43

  Domingos, Pedro ^a  

^a INSTITUTO SUPERIOR TÉCNICO   (Portugal)

Author keywords

[No Author keywords available]

Indexed keywords

OCCAM'S RAZOR; THEORETICAL ARGUMENTS;

EID: 85166350822     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (55)

1. Abu-Mostafa, Y. S. 1989. Learning from hints in neural networks. Journal of Complexity 6:192-198.
2. Akaike, H. 1978. A Bayesian analysis of the minimum A I C procedure. Annals of the Institute of Statistical Mathematics 30A:9-14.
3. Bernardo, J. M., and Smith, A. F. M. 1994. Bayesian Theory. New York, NY: Wiley.
4. Bishop, C. M. 1995. Neui-al Networks for Pattern Recognition. Oxford, U K: Oxford University Press.
5. Blumer, A.; Ehrenfeucht, A.; Haussler, D.; and Warmutli, M. K. 1987. Occam's razor. Information Processing Letters 24:377-380.
6. Cestnik, B., and Bratko, I. 1988. Learning redundant rules in noisy domains. In Proceedings of the Eighth European Conference on Artificial Intelligence, 348¬ 356. Munich, Germany: Pitman.
7. Cheeseman, P. 1990. On finding the most probable model. In Shrager, J., and Langley, P., eds., Computational Models of Scientific Discovery and Theory Formation. San Mateo, CA: Morgan Kaufmann. 73¬ 95.
8. Chickering, D. M., and Heckerman, D. 1997. Efficent approximations for the marginal likelihood of Bayesian networks with hidden variables. Machine Learning 29:181-212.
9. Clark, P.. and Matwin, S. 1993. Using qualitative models to guide inductive learning. In Proceedings of the Tenth International Conference on Machine Learning, 49-56. Amherst, MA: Morgan Kaufmann.
10. Clearwater, S., and Provost, F. 1990. RL4: A tool for knowledge-based induction. In Proceedings of the Second IEEE International Conference on Tools for Artificial Intelligence, 24-30. San Jose, CA: IEEE Computer Society Press.
11. Cohen, P. R., and Jensen, D. 1997. Overfitting explained. In Preliminary Papers of the Sixth International Workshop on Artificial Intelligence and Statistics, 115-122. Fort. Lauderdale, F L: Society for Artificial Intelligence and Statistics.
12. Cover, T. M., and Thomas, J. A. 1991. Elements of Information Theory. New York, NY: Wiley.
13. Craven, M. W. 1996. Extracting Comprehensible Models from Trained Neural Networks. Ph.D. Dissertation, Department of Computer Sciences, University of Wisconsin - Madison, Madison, W I.
14. Datta, P., and Kibler, D. 1995. Learning prototypical concept descriptions. In Proceedings of the Twelfth International Conference on Machine Learning, 158¬ 166. Tahoe City, CA: Morgan Kaufmann.
15. Domingos, P., and Pazzani, M. 1997. On the optimality of the simple Bayesian classifier under zero-one loss. Machine Learning 29:103-130.
16. Domingos, P. 1996. Unifying instance-based and rule-based induction. Machine Learning 24:141-168.
17. Domingos, P. 1997a. Knowledge acquisition from examples via multiple models. In Proceedings of the Fourteenth International Conference on Machine Learning, 98-106. Nashville, T N: Morgan Kaufmann.
18. Domingos, P. 1997b. Why does bagging work? A Bayesian account and its implications. In Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining, 155-158. Newport Beach, CA: AAAI Press.
19. Domingos, P. 1998. A process-oriented heuristic for model selection. In Proceedings of the Fifteenth International Conference on Machine Learning. Madison, W I: Morgan Kaufmann.
20. Donoho, S., and Rendell, L. 1996. Constructive induction using fragmentary knowledge. In Proceedings of the Thirteenth International Conference on Machine Learning, 113-121. Bari, Italy: Morgan Kaufmann.
21. Elomaa, T. 1994. In defense of C4.5: Notes on learning one-level decision trees. In Proceedings of the Eleventh International Conference on Machine Learning, 62¬ 69. New Brunswick, N J: Morgan Kaufmann.
22. Fisher, D. H., and Schlimmer, J. C. 1988. Concept simplification and prediction accuracy. In Proceedings of the Fifth International Conference on Machine Learning, 22-28. Ann Arbor, MI: Morgan Kaufmann.
23. Friedman, J. H. 1996. On bias, variance, 0/1 - loss, and the curse-of-dimensionality. Technical report, Department of Statistics and Stanford Linear Accelerator Center, Stanford University, Stanford, CA. ftp://playfair.Stanford.edu/pub/friedman/kdd.ps.Z.
24. Gams, M. 1989. New measurements highlight the importance of redundant knowledge. In Proceedings of the Fourth European Working Session on Learning, 71-79. Montpellier, France: Pitman.
25. Haussler, D. 1988. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence 36:177-221.
26. Holte, R. C. 1993. Very simple classification rules perform well on most commonly used dataseis. Machine Learning 11:63-91.
27. Jensen, D., and Schmill, M. 1997. Adjusting for multiple comparisons in decision tree pruning. In Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining, 195-198. Newport Beach, CA: AAAI Press.
28. Langley, P. 1996. Induction of condensed determinations. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, 327-330. Portland, OR: AAAI Press.
29. Lawrence, S.; Giles, C. L.; and Tsoi, A. C. 1997. Lessons in neural network training: Overfitting may be harder than expected. In Proceedings of the Fourteenth National Conference on Artificial Intelligence, 540-545. Providence, R I: AAAI Press.
30. Lee, Y.; Buchanan, B. G.; and Aronis, J. M. 1998. Knowledge-based learning in exploratory sci-enee: Learning rules to predict rodent carcinogenicity. Machine Learning 30:217-240.
31. MacKay, D. 1992. Bayesian interpolation. Neural Computation 4:415-447.
32. Mingers, J. 1989. An empirical comparison of pruning methods for decision tree induction. Machine Learning 4:227-243.
33. Mitchell, T. M. 1980. The need for biases in learning generalizations. Technical report, Rutgers University, Computer Science Department, New Brunswick, N J.
34. Murphy, P., and Pazzani, M. 1994. Exploring the decision forest: An empirical investigation of Occam's razor in decision tree induction. Journal of Artificial Intelligence Research 1:257-275.
35. Murthy, S., and Salzberg, S. 1995. Lookahead and pathology in decision tree induction. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, 1025-1031. Montréal, Canada: Morgan Kaufmann.
36. Ourston, D., and Mooney, R. J. 1994. Theory refinement combining analytical and empirical methods. Artificial Intelligence 66:273-309.
37. Pazzani, M.; Mani, S.; and Shankle, W. R. 1997. Beyond concise and colorful: Learning intelligible rules. In Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining, 235-238. Newport Beach, CA: AAAI Press.
38. Pearl, J. 1978. On the connection between the complexity and credibility of inferred models. International Journal of General Systems 4:255-264.
39. Piatetsky-Shapiro, G. 1996. Editorial comments. KDD Nuggets 96:28.
40. Quinlan, J. R., and Cameron-Jones, R. M. 1995. Oversearching and layered search in empirical learning. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, 1019-1024. Montréal, Canada: Morgan Kaufmann.
41. Quinlan, J. R., and Rivest, R. L. 1989. Inferring decision trees using the minimum description length principle. Information and Computation 80:227-248. Quinlan, J. R. 1996. Bagging, boosting, and C4.5. In Proceedings of the Thirteenth National Conference on Artificial Intelligence, 725-730. Portland, OR: AAAI Press.
42. Rao, J. S., and Potts, W. J. E. 1997. Visualizing bagged decision trees. In Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining. Newport Beach, CA: AAAI Press.
43. Rao, R. B.; Gordon, D.; and Spears, W. 1995. For every action, is there really an equal and opposite reaction? Analysis of the conservation law for generalization performance. In Proceedings of the Twelfth International Conference on Machine Learning, 471-479. Tahoe City, CA: Morgan Kaufmann.
44. Rissanen, J. 1978. Modeling by shortest data description. Automatica 14:465-471.
45. Schaffer, C. 1993. Overfitting avoidance as bias. Machine Learning 10:153-178.
46. Schaffer, C. 1994. A conservation law for generalization performance. In Proceedings of the Eleventh International Conference on Machine Learning, 259¬ 265. New Brunswick, N J: Morgan Kaufmann.
47. Schuurmans, D.; Ungar, L. H.; and Foster, D. P. 1997. Characterizing the generalization performance of model selection strategies. In Proceedings of the Fourteenth International Conference on Machine Learning, 340-348. Nashville, T N: Morgan Kaufmann.
48. Schuurmans, D. 1997. A new metric-based approach to model selection. In Proceedings of the Fourteenth National Conference on Artificial Intelligence, 552-558. Providence, R I: AAAI Press.
49. Schwarz, G. 1978. Estimating the dimension of a model. Annals of Statistics 6:461-464.
50. Tornay, S. C. 1938. Ockham: Studies and Selections. La Salle, I L: Open Court.
51. Vapnik, V. N. 1995. The Nature of Statistical Learning Theory. New York, NY: Springer-Verlag.
52. Wallace, C. S., and Boulton, D. M. 1968. An information measure for classification. Computer Journal 11:185-194.
53. Webb, G. I. 1996. Further experimental evidence against the utility of Occam's razor. Journal of Artificial Intelligence Research 4:397-417.
54. Webb, G. I. 1997. Decision tree grafting. In Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence, 846-851. Nagoya, Japan: Morgan Kaufmann.
55. Wolpert, D. 1996. The lack of a priori distinctions between learning algorithms. Neural Computation 8:1341-1390.