RULES-6: A simple rule induction algorithm for handling large data sets

Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science

Volumn 219, Issue 10, 2005, Pages 1119-1137

  Pham, D T ^a   Afify, A A ^a  

^a CARDIFF UNIVERSITY   (United Kingdom)

Author keywords

Classification; Discretization; Inductive learning; Noise handling; Rule induction

Indexed keywords

CLASSIFICATION (OF INFORMATION); CONSTRAINT THEORY; DATA HANDLING; LEARNING SYSTEMS; MECHANISMS; OPTIMIZATION; PROBABILITY;

CONTINUOUS ATTRIBUTES; DISCRETIZATION METHOD; INDUCTION ALGORITHMS; INDUCTIVE LEARNING; LARGE DATA SETS; NOISE HANDLING; RULE INDUCTION;

LEARNING ALGORITHMS;

MATHEMATICAL ANALYSIS;

EID: 28944445792     PISSN: 09544062     EISSN: None     Source Type: Journal    
DOI: 10.1243/095440605X31931     Document Type: Article

References (69)

1. Braha, D. Data mining for design and manufacturing: methods and applications, 2001 (Klauwer Academic Publishers, Boston).
2. Monostori, L. AI and machine learning techniques for managing complexity, changes and uncertainties in manufacturing. In Proceedings of the 15th Triennial World Congress, Barcelona, Spain, 2002, pp. 119-130.
3. Pham, D. T. and Afify, A. A. Machine learning techniques and their applications in manufacturing. Proc. Instn Mech. Engrs, Part B: J. Engineering Manufacture, 2005, 219(B5), 395-412.
4. Breiman, L., Friedman, J. H., Olshen, R. A., and Stone, C. J. Classification and regression trees, 1984 (Belmont, Wadsworth).
5. Quinlan, J. R. Learning efficient classification procedures and their application to chess endgames. In Machine learning: an artificial intelligence approach (Eds R. S. Michalski, J. G. Carbonell, and T. M. Mitchell), 1983, Vol. I, pp. 463-482 (Tioga Publishing Co., Palo Alto, CA).
6. Quinlan, J. R. Induction of decision trees. Mach. Learn., 1986, 1, 81-106.
7. Quinlan, J. R. C4.5: Programs for machine learning, 1993 (Morgan Kaufmann, San Mateo, CA).
8. ISL. Clementine data mining package, 1998 (SPSS UK Ltd., Surrey, UK).
9. RuleQuest. Data Mining Tools C5.0. Pty Ltd, 30 Athena Avenue, St Ives NSW 2075, Australia, available from: http://www.rulequest.com/ see5-info.html, accessed 1 February 2003.
10. Mehta, M., Agrawal, R., and Rissanen, J. SLIQ: a fast scalable classifier for data mining. In Proceedings of the fifth International Conference on Extending data-base technology, Avignon, France, 1996, pp. 18-32.
11. Shafer, J., Agrawal, R., and Mehta, M. SPRINT: a scalable parallel classifier for data mining. In Proceedings of the 22nd International Conference on Very large data bases (VLDB), Mumbai (Bombay), India, 1996, pp. 544-555.
12. Rastogi, R. and Shim, K. PUBLIC: a decision tree classifier that integrates building and pruning. In Proceedings of the 24th International Conference on Very large data bases (VLDB), New York, USA, 1998, pp.404-415.
13. Michalski, R. S. On the quasi-minimal solution of the general covering problem. In Proceedings of the fifth International Symposium on Information processing (FCIP 69), Bled, Yugoslavia, 1969, A3 (Switching Circuits), pp. 125-128.
14. Michalski, R. S., Mozetic, I., Hong, J., and Lavrac, N. The multi-purpose incremental learning system AQ15 and its testing application to three medical domains. American association of artificial intelligence, 1986, pp. 1041-1045 (Morgan Kaufmann, Los Altos, CA).
15. Michalski, R. S. and Kaufman, K. A. The AQ19 system for machine learning and pattern discovery: a general description and user guide. Reports of the Machine Learning and Inference Laboratory, MLI 01-2, George Mason University, Fairfax, VA, USA, 2001.
16. Clark, P. and Niblett, T. The CN2 induction algorithm. Mach. Learn., 1989, 3(4), 261-284.
17. Clark, P. and Boswell, R. Rule induction with CN2: some recent improvements. In Proceedings of the fifth European Conference on Artificial intelligence, Porto, Portugal, 1991, pp. 151-163.
18. Cohen, W. W. Fast effective rule induction. In Proceedings of the 12th International Conference on Machine learning, Lake Tahoe City, California, USA, 1995, pp. 115-123 (Morgan Kaufmann, San Francisco, CA).
19. Cohen, W. W. and Singer, Y. A simple, fast and effective rule learner. In Proceedings of the 16th National Conference on Artificial intelligence, Menlo Park, CA, 1999, pp. 335-342 (AAAI/MIT Press, Menlo Park, CA).
20. Rivest, R. Learning decision lists. Mach. Learn., 1987, 2, 229-246.
21. Pham, D. T. and Aksoy, M. S. An algorithm for automatic rule induction. Artificial Intelligence in Engineering, 1993, 8(4), 227-282.
22. Pham, D. T. and Aksoy, M. S. RULES: a simple rule extraction system. Expert Syst. Appl., 1995, 8(1), 59-65.
23. Pham, D. T. and Aksoy, M. S. A new algorithm for inductive learning. J. Syst. Eng., 1995, 5, 115-122.
24. Pham, D. T. and Dimov, S. S. An efficient algorithm for automatic knowledge acquisition. Pattern Recog., 1997, 30(7), 1137-1143.
25. Lee, C. Generating classification rules from databases. In Proceedings of the Ninth Conference on Application of artificial intelligence in engineering, PA, USA, 1994, pp. 205-212.
26. Pham, D. T. and Dimov, S. S. An algorithm for incremental inductive learning. Proc. Instn. Mech. Engrs, Part B: J. Engineering Manufacture, 1997, 211, 239-249.
27. Pham, D. T., Bigot, S., and Dimov, S. S. RULES-5: a rule induction algorithm for problems involving continuous attributes. Proc. Instn Mech. Engrs, Part C: J. Mechanical Engineering Science, 2003, 217(C12), pp. 1273-1286.
28. Pham, D. T. and Dimov, S. S. An approach to concurrent engineering. Proc. Instn Mech. Engrs, Part B: J. Engineering Manufacture, 1998, 212(B1), 13-27.
29. Jennings, N. R. Automated visual inspection of engine valve stem seals. Internal Report, University of Wales Cardiff, Cardiff, UK, 1996.
30. Pagallo, G. and Haussler, D. Boolean feature discovery in empirical learning. Mach. Learn., 1990, 3, 71-99.
31. Domingos, P. A unified approach to concept learning. PhD Thesis, University of California, Irvine, 1997.
32. Holte, R. C., Acker, L. E., and Porter, B. W. Concept learning and the problem of small disjuncts. In Proceedings of the 11th International Joint Conference on Artificial intelligence, Detroit, Michigan, USA, 1989, pp. 813-818.
33. Weiss, G. M. Learning with rare cases and small disjuncts. In Proceedings of the 12th International Conference on Machine learning, Lake Tahoe City, California, USA, 1995, pp. 558-565 (Morgan Kaufmann, San Francisco, CA).
34. Weiss, G. M. and Hirsh, H. The problem with noise and small disjuncts. In Proceedings of the 15th International Conference on Machine learning, Madison, Wisconsin, USA, 1998, pp. 574-578 (Morgan Kaufmann, San Francisco, CA).
35. Frayman, Y., Ting, K. M., and Wang, L. A fuzzy neural network for data mining: dealing with the problem of small disjuncts. In IEEE International Joint Conference on Neural networks (IJCNN-99), Washington, DC, 1999, 4, pp. 2490-2493.
36. Weiss, G. M. and Hirsh, H. A quantitative study of small disjuncts. In Proceedings of the 17th National Conference on Artificial intelligence, Austin, Texas, 2000, pp. 665-670.
37. Weiss, S. and Indurkhya, N. Reduced complexity rule induction. In Proceedings of the 12th International Joint Conference on Artificial intelligence, Sydney, Australia, 1991 pp. 678-684 (Morgan Kaufmann, San Francisco, CA).
38. Cendrowska, J. PRISM: an algorithm for inducing modular rules. Int. J. Man-Mach Studies, 1987, 27, 349-370.
39. Fürnkranz, J. and Widmer, G. Incremental reduced error pruning. In Proceedings of the 11th International Conference on Artificial intelligence, Amsterdam, The Netherlands, 1994, pp. 453-457.
40. Muggleton, S. Foundations of inductive logic programming, 1995 (Prentice Hall, Englewood Cliffs, NJ).
41. Webb, G. Systematic search for categorical attribute-value data-driven machine learning. In AI'94-Proceedings of the sixth Australian joint conference on artificial intelligence (Eds C. Rowles, H. Liu, and N. Foo), Melbourne, 1993, pp. 342-347 (World Scientific, Chicago).
42. Webb, G. OPUS: an efficient admissible algorithm for unordered search. J. Artif. Intell. Res., 1995, 3, 431-465.
43. Segal, R. B. Machine learning as massive search. PhD Thesis, Department of Computer Science and Engineering, University of Washington, 1997.
44. Cestnik, B. Estimating probabilities: a crucial task in machine learning. In Proceedings of the Third European Conference on Artificial intelligence (ECAI-90), Stockholm, Sweden, 1990, pp. 147-149 (Pitman, London).
45. Afify, A. A. Design and analysis of scalable rule induction systems. PhD Thesis, University of Wales Cardiff, School of Engineering, Systems Engineering Division, Cardiff, UK, 2004.
46. Good, I. J. The estimation of probabilities: an essay on modern Bayesian methods, 1965 (MIT Press, Cambridge, MA).
47. Mitchell, T. M. Machine learning, 1997 (McGraw Hill, New York).
48. Ho, K. M. and Scott, P. D. Zeta: a global method for discretization of continuous variables. IEEE Trans. Knowl. Data Eng., 1997, 9(5), 718-730.
49. Zighed, D. A., Rakotomalala, R., and Feschet, F. Optimal multiple intervals discretization of continuous attributes for supervised learning. In Proceedings of the Third International Conference on Knowledge discovery and data mining (KDD-97), Newport Beach, California, USA, 1997, pp. 295-298 (AAAI Press, Menlo Park, CA).
50. Frank, E. and Witten, I. H. Making better use of global discretization. In Proceedings of the 15th International Conference on Machine learning, Madison, Wisconsin, USA, 1998, pp. 152-160.
51. Trautzsch, S. and Perner, P. Multi-interval discretization methods for decision tree learning. In Advances in pattern recognition (Eds A. Amin, D. Dori, P. Pudil, and H. Freeman), 1998, 1451, pp. 475-482 (Springer-Verlag, Germany).
52. An, A. and Cercone, N. Discretisation of continuous attributes for learning classification rules. In Proceedings of the Third Pacific-Asia Conference on Knowledge discovery and data mining (PAKDD-99), Kyoto, Japan, 1999, pp. 509-514.
53. Wong, A. K. C. and Chiu, D. K. Y. Synthesizing statistical knowledge from incomplete mixed-mode data. IEEE Trans. Pattern Anal. Mach. Intell., 1987, 9(6), 796-805.
54. Holte, R. C. Very simple classification rules perform well on most commonly used data sets. Mach. Learn., 1993, 11, 63-90.
55. Catlett, J. On changing continuous attributes into ordered discrete attributes. In Proceedings of the European Working Session on Learning, Porto, Portugal, 1991, pp. 164-178 (Springer-Verlag, Germany).
56. Fayyad, U. M. and Irani, K. B. Multi-interval discretization of continuous-valued attributes for classification. In Proceedings of the 13th International Joint Conference on Artificial intelligence, Chambery, France, 1993, pp. 1022-1027.
57. Fayyad, U. M. and Irani, K. B. On the handling of continuous-valued attributes in decision tree generation. Mach. Learn., 1992, 8, 87-102.
58. Maass, W. Efficient agnostic PAC-learning with simple hypotheses. In Proceedings of the Seventh Annual ACM Conference on Computational learning theory, New Brunswick, New Jersey, USA, 1994, pp. 67-75.
59. Auer, P., Holte, R. C., and Maass, W. Theory and application of agnostic PAC-learning with small decision trees. In Proceedings of the 12th International Conference on Machine learning, Lake Tahoe City, California, USA, 1995, pp. 21-29 (Morgan Kaufmann, San Francisco, CA).
60. Fulton, T., Kasif, S., and Salzberg, S. Efficient algorithms for finding multi-way splits for decision trees. In Proceedings of the 12th International Conference on Machine learning, Lake Tahoe City, California, USA, 1995, pp. 244-251 (Morgan Kaufmann, San Francisco CA).
61. Birkendorf, A. On fast and simple algorithms for finding maximal subarrays and applications on computational learning theory. In Lecture notes in artificial intelligence, 1997, 1208, pp. 198-209 (Springer-Verlag, Heidelberg).
62. Rousu, J. Efficient range partitioning in classification learning. PhD Thesis, Department of Computer Science, University of Helsinki, Finland, 2001.
63. Cai, Z. Technical aspects of data mining. PhD Thesis, University of Wales Cardiff, Cardiff, UK, 2001.
64. Dougherty, J., Kohavi, R., and Sahami, M. Supervised and unsupervised discretization of continuous features. In Proceedings of the 12th international Conference on Machine learning, Lake Tahoe City, California, USA, 1995, pp. 194-202 (Morgan Kaufmann, San Francisco, CA).
65. Ventura, D. and Martinez, T. R. An empirical comparison of discretization methods. In Proceedings of the 10th International Symposium on Computer and information sciences, Aydin, Turkey, 1995, pp. 443-450.
66. Kohavi, R. and Sahami, M. Error-based and entropybased discretization of continuous features. In Proceedings of the second International Conference on Knowledge discovery in databases (Eds E. Simondis, U. Fayyad), 1996, pp. 114-119 (AAAI Press, Menlo Park, CA).
67. Blake, C. L. and Merz, C. J. UCI repository of machine learning databases. University of California, Department of Information and Computer Science, Irvine, CA, 1998, available from http://www.ics.uci. edu/~mlearn/MLRepository.html, accessed 1 February 2003.
68. Devijver, P. A. and Kittler, J. Pattern recognition: a statistical approach, 1982 (Prentice Hall, Englewood Cliffs, London).
69. Efron, B. and Tibshirani, R. An introduction to the bootstrap, 1993 (Chapman and Hall, USA).