Three new MDL-based pruning techniques for robust rule induction

Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science

Volumn 220, Issue 4, 2006, Pages 553-564

  Pham, D T ^a   Afify, A A ^a  

^a CARDIFF UNIVERSITY   (United Kingdom)

Author keywords

Inductive learning; Machine learning; Minimum description length principle; Noise handling; Pruning; Rule induction

Indexed keywords

CLASSIFICATION (OF INFORMATION); DATA REDUCTION; KNOWLEDGE BASED SYSTEMS; LEARNING ALGORITHMS; ROBUSTNESS (CONTROL SYSTEMS); SET THEORY;

INDUCTIVE LEARNING; MINIMUM DESCRIPTION LENGTH (MDL) PRINCIPLE; NOISE HANDLING; PRUNING; RULE INDUCTION;

LEARNING SYSTEMS;

CLASSIFICATION (OF INFORMATION); DATA REDUCTION; KNOWLEDGE BASED SYSTEMS; LEARNING ALGORITHMS; LEARNING SYSTEMS; ROBUSTNESS (CONTROL SYSTEMS); SET THEORY;

EID: 33845688829     PISSN: 09544062     EISSN: None     Source Type: Journal    
DOI: 10.1243/09544062C18404     Document Type: Article

References (35)

1. Cohen, W. W. Fast effective rule induction. In Proceedings of the 12th International Conference on Machine learning, Tahoe City, CA, 1995, pp. 115-123 (Morgan Kaufmann, San Fransisco, CA).
2. Fürnkranz, J. Pruning algorithms for rule learning. Mach. Learn., 1996, 27, 139-171.
3. Pham, D. T. and Afify, A. A. SRI: a scalable rule induction algorithm. Proc. Instn Mech. Engrs, Part C:J. Mechanical Engineering Science, 2006, 220(C4), 537-552.
4. Clark, P. and Niblett, T. The CN2 induction algorithm. Mach. Learn., 1989, 3(4), 261-284.
5. Fürnkranz, J. Efficient pruning methods for relational learning. PhD Thesis, Technisch-Naturwissens chaftlichen Fakultät, Techniscen Universität Wien, 1994.
6. Frank, E. Pruning decision trees and lists. PhD Thesis, Department of Computer Science, University of Waikato, Hamilton, New Zealand, 2000.
7. Mingers, J. An empirical comparison of pruning methods for decision tree induction. Mach. Learn., 1989, 4, 227-243.
8. Breslow, A. and Aha, D. W. Simplifying decision trees: a survey. Knowl. Eng. Rev., 1996, 12, 1-40.
9. Esposito, F., Malerba, D., and Semeraro, G. A comparative analysis of methods for pruning decision trees. IEEE Trans. Patt. Anal. Mach. Int., 1997, 19(5), 476-491.
10. Quinlan, J. R. Simplifying decision trees. Int. J. Man Mach. Stud., 1987, 27, 221-234.
11. Pagallo, G. and Haussler, D. Boolean feature discovery in empirical learning. Mach. Learn., 1990, 3, 71-99.
12. Brunk, C. A. and Pazzani, M. J. An investigation of noise-tolerant relational concept learning algorithms. In Proceedings of the 8th International Workshop on Machine learning, Evanston, Illinois, 1991, 389-393.
13. Cohen, W. W. Efficient pruning methods for separate-and-conquer rule learning systems. In Proceedings of the 13th International Conference on Artificial intelligence, Chambery, France, 1993, 988-994 (Morgan Kaufmann, San Fransisco, CA).
14. Frank, E. and Witten, I. H. Reduced-error prunning with significant tests. Working paper, Department of Computer Science, University of Waikato, Hamilton, New Zealand, 1999.
15. Elomaa, T. and Kääriäinen, M. An analysis of reduced error pruning. J. Artif. Int. Res., 2001, 15, 163-187.
16. Fürnkranz, J. and Widmer, G. Incremental reduced error pruning. In Proceedings of the 11th International Conference on Machine learning, New Brunswick, NJ, 1994, pp. 70-77 (Morgan Kaufmann, San Fransisco, CA).
17. Fürnkranz, J. Top-down pruning in relational learning. In Proceedings of the 11th European Conference on Artificial intelligence, Amsterdam, The Netherlands, 1994, 453-457.
18. Rissanen, J. Stochastic complexity and modelling. Ann. Stat., 1986, 14(3), 1080-1100.
19. Barron, A., Rissanen, J., and Yu, B. The minimum description length principle in coding and modelling. IEEE Trans. Inform. Theory, 1998, 44(6), 2743-2760.
20. Grunwald, P. Model selection based on minimum description length. J. Math. Psychol, 2000, 44, 133-170.
21. Tirri, H. MDL and classification in machine learning. - In Proceedings of the Neural Information Processing Systems Workshop on Minimum description length (NIPS-2001), Whistler, British Columbia, Canada, 2001 (MIT Press, Menlo Park, CA).
22. Georgeff, M. P. and Wallace, C. S. A general selection criterion for inductive inference. In Proceedings of the 6th European Conference on Artificial intelligence (ECAI-84), Pisa, Italy, 1984, 473-482.
23. Quinlan, J. R. and Rivest, R. L. Inferring decision trees using minimum description length principle. Inform. Comp., 1989, 80, 227-248.
24. Quinlan, J. R. C4.5: Programs for machine learning, 1993 (Morgan Kaufmann, San Mateo, CA).
25. Quinlan, J. R. The minimum description length principle and categorical theories. In Proceedings of the 11th International Conference on Machine learning, New Brunswick, NJ, 1994,233-241 (Morgan Kaufmann, San Fransisco, CA).
26. Quinlan, J. R. MDL and categorical theories (continued). In Proceedings of the 12th International Conference on Machine learning, Lake Tahoe City, CA, 1995, 464-470 (Morgan Kaufmann, San Fransisco, CA).
27. Mehta, M., Rissanen, J., and Agrawal, R. MDL-based decision tree pruning. In Proceedings of the 1st International Conference on Knowledge discovery in databases and data mining (ADD-95), Montreal, Canada, 1995, 216-221 (AMI Press, Menlo Park, CA).
28. Pfahringer, B. Practical uses of the minimum description length principle in inductive learning. PhD Thesis, Institut Für Med.Kybernetik u. AI, Techniscen Universität Wien, 1995.
29. Pfahringer, B. Compression-based pruning of decision lists. In Proceedings of the 9th European Conference on Machine learning, Prague, Czech Republic, 1997, 199-212 (Springer-Verlag, Berlin).
30. Robnik-Sikonja, M. and Kononenko, I. Pruning regression trees with MDLP. In Proceedings of the 13th European Conference on Artificial intelligence (ECAI-98), Brighton, UK, 1998, 455-459 (John Wiley and Sons, Chichester).
31. Pham, D. T. and Afify, A. A. A new MDL-based pruning technique for rule induction algorithms. Technical report, Manufacturing Engineering Centre, Cardiff University, 2004.
32. RuleQuest. Data Mining Tools C5.0. Pty Ltd, 30 Athena Avenue, St Ives NSW 2075, Australia, available from http://www.rulequest.com/see5-info.html, accessed 1 February 2003.
33. Blake, C. L. and Merz, C. J. UCI Repository of Machine Learning Databases. University of California, Department of Information and Computer Science, Irvine, CA, 1998, available from http://www.ics.uci.edu/~mlearn/ MLRepository.html, accessed 1 February 2003.
34. Devijver, P. A. and Kittler, J. Pattern recognition: a statistical approach, 1982 (Prentice Hall, Englewood Cliffs, London).
35. Efron, B. and Tibshirani, R. An introduction to the bootstrap, 1993 (Chapman & Haty USA).