Highly scalable and robust rule learner: Performance evaluation and comparison

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 36, Issue 1, 2006, Pages 32-53

  Kurgan, Lukasz A ^a   Cios, Krzysztof J ^b,c,d   Dick, Scott ^a  

^a UNIVERSITY OF ALBERTA   (Canada)

^b UNIVERSITY OF COLORADO   (United States)

^c University of Colorado at Boulder   (United States)

^d 4cData   (United States)

Author keywords

Complexity; Data mining; DataSqueezer; Machine learning; Missing data; Rule induction; Rule learner

Indexed keywords

ALGORITHMS; COMPETITIVE INTELLIGENCE; DATA MINING; DATA PROCESSING; DATABASE SYSTEMS;

COMPLEXITY; DATASQUEEZER; EXTRACTION ALGORITHM; MISSING DATA; ROBUST RULE LEARNER; RULE INDUCTION;

LEARNING SYSTEMS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; COMPARATIVE STUDY; DATA BASE; DECISION SUPPORT SYSTEM; EVALUATION; FACTUAL DATABASE; INFORMATION RETRIEVAL; METHODOLOGY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; DATABASE MANAGEMENT SYSTEMS; DATABASES, FACTUAL; DECISION SUPPORT TECHNIQUES; INFORMATION STORAGE AND RETRIEVAL;

EID: 31744432403     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2005.852983     Document Type: Article

References (82)

1. T. Abraham and J. F. Roddick, "Incremental meta-mining from large temporal data sets, advances in database technologies," in Proc. 1st Int. Workshop on Data Warehousing and Data Mining (DWDM'98), 1999, pp. 41-54.
2. S. D. Bay and M. J. Pazzani, "Detecting group differences: Mining contrast sets," Data Mining Knowl. Disc., vol. 5, no. 3, pp. 213-246, 2001.
3. J. A. Blackard, "Comparison of Neural Networks and Discriminant Analysis in Predicting Forest Cover Types," Ph.D. dissertation, Dept. Forest Sciences, Colorado State Univ., Fort Collins, 1998.
4. C. L. Blake and C. J. Merz. (1998) UCI Repository of ML Databases. Univ. of California, Department of Information and Computer Science, Irvine, CA. [Online]. Available: http://www.ics.uci.edu/~mlearn/ ML-Repository.html
5. E. Boros, P. L. Hammer, T. Ibaraki, A. Kogan, E. Mayoraz, and I. Muchnik, "An implementation of logical analysis of data," IEEE Trans. Knowl. Data Eng., vol. 12, no. 2, pp. 292-306, Feb. 2000.
6. R. Brachman and T. Anand, "The process of knowledge discovery in databases: A human-centered approach," in Advances in Knowledge Discovery and Data Mining, U. M. Fayyad, G. Piatesky-Shapiro, P. Smyth, and R. Uthurusamy, Eds. Cambridge, MA: AAAI/MIT Press, 1996.
7. L. Breiman, J. Friedman, R. Olshen, and C. Stone, Classification and Regression Trees. London, U.K.: Chapman & Hall, 1984.
8. J. Catlett, "Megainduction: A Test flight," in Proc. 8th Int. Workshop on Machine Learning, Ithaca, NY, 1991, pp. 596-599.
9. J. Catlett, "On changing continuous attributes into ordered discrete attributes," in Proc. Eur. Working Session on Learning, 1991, pp. 164-178.
10. P. Chan and S. Stolfo, "On the accuracy of meta-learning for scalable data mining," J. Intell. Inform. Syst., vol. 8, pp. 5-28, 1997.
11. l N. V. Chawla, K. W. Bowyer, L. Hall, and W. P. Kegelmeyer, "SMOTE: Synthetic minority over-sampling technique," J. Artif. Intell. Res., vol. 16, pp. 321-357, 2002.
12. N. Chawla, L. Hall, K. Bowyer, and W. Kegelmeyer, "Learning ensembles from bites: A scalable and accurate approach," J. Mach. Learn. Res., vol. 5, pp. 421-451, 2004.
13. J. Y. Ching, A. K. C. Wong, and K. C. C. Chan, "Class-dependent discretization for inductive learning from continuous and mixed mode data," IEEE Trans. Pattern Anal. Mach. Intell., vol. 17, no. 7, pp. 641-651, Jul. 1995.
14. M. Chisholm and P. Tadepalli., "Learning decision rules by randomized iterative local search," in Proc. Int. Conf. Machine Learning, 2002, pp. 75-82.
15. D. Chiu, A. Wong, and B. Cheung, "Information discovery through hierarchical maximum entropy discretization and synthesis," in Knowledge Discovery in Databases, G. Piatesky-Shapiro and W. J. Frowley, Eds. Cambridge, MA: MIT Press, 1991.
16. K. J. Cios and L. A. Kurgan, "Hybrid inductive machine learning: An overview of CLIP algorithms," in New Learning Paradigms in Soft Computing, L. C. Jain and J. Kacprzyk, Eds. Berlin, Germany: Physica-Verlag (Springer), 2002, pp. 276-322.
17. K. J. Cios and L. A. Kurgan, "Trends in data mining and knowledge discovery," in Advanced Techniques in Knowledge Discovery and Data Mining, N. R. Pal, L. C. Jain, and N. Teoderesku, Eds. Berlin, Germany: Physica-Verlag (Springer), 2005, pp. 1-26.
18. K. J. Cios, W. Pedrycz, and R. Swiniarski, Data Mining Methods for Knowledge Discovery. Norwell, MA: Kluwer, 1998.
19. K. J. Cios and L. A. Kurgan, "CLIP4: Hybrid inductive machine learning algorithm that generates inequality rules," in Inform. Sci., S. K. Pal and A. Ghosh, Eds., 2004, vol. 163, Special Issue on Soft Computing Data Mining, pp. 37-83.
20. K. J. Cios and G. Moore, "Uniqueness of medical data mining," Artif. Intell. Med., vol. 26, no. 1-2, pp. 1-24, 2002.
21. P. Clark and R. Boswell, "Rule induction with CN2: Some recent improvements," in Proc. 5th Eur. Machine Learning Conf. (EWSL-91), Berlin, Germany, 1991, pp. 151-163.
22. P. Clark and T. Niblett, "The CN2 algorithm," Mach. Learn., vol. 3, pp. 261-283, 1989.
23. W. Cohen and Y. Singer, "A simple, fast and effective rule learner," in Proc. 16th Nat. Conf. Artificial Intelligence, 1999, pp. 335-342.
24. W. Cohen, "Efficient pruning methods for separate-and-conquer rule learning systems," in Proc. 13th Int. Joint Conf. Artificial Intelligence, Chambery, France, 1993, pp. 988-994.
25. W. Cohen, "Fast effective rule induction," in Proc. 12th Int. Conf. Machine Learning, Lake Tahoe, CA, 1995, pp. 115-123.
26. W. Cohen, "Grammatically biased learning: Learning logic programs using an explicit antecedent description language," Artif. Intell., vol. 68, pp. 303-366, 1994.
27. W. Cohen, "Learning trees and rules with set-valued features," in Proc. 13th Nat. Conf. Artificial Intelligence, Portland, OR, 1996, pp. 709-716.
28. C. Cortes and V. N. Vapnik, "Support vector networks," Mach. Learn., vol. 20, pp. 273-279, 1995.
29. O. Dain, R. Cunningham, and S. Boyer, "IREP++ a faster rule learning algorithm," in Proc. 4th SIAM Int. Conf. Data Mining, Lake Buena Vista, FL, 2004, pp. 138-146.
30. T. G. Dietterich and R. S. Michalski, "Inductive learning of structural descriptions: Evaluation criteria and comparative review of selected methods," Artif. Intell., vol. 16, no. 3, pp. 257-294, 1981.
31. P. Domingos, "The RISE system: Conquering without separating," in Proc. 6th IEEE Int. Conf. Tools with Artificial Intelligence, New Orleans, LA, 1994, pp. 704-707.
32. R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973.
33. U. M. Fayyad and K. B. Irani, "Multi-interval discretization of continuous-valued attributes for classification learning," in Proc. 13th Int. Joint Conf. Artificial Intelligence, San Francisco, CA, 1993, pp. 1022-1027.
34. U. M. Fayyad, G. Piatesky-Shapiro, P. Smyth, and R. Uthurusamy, Advances in Knowledge Discovery and Data Mining: AAAI/MIT Press, 1996.
35. A. Freitas, "A survey of parallel data mining," in Proc. 2nd Int. Conf. Practical Applications of Knowledge Discovery and Data Mining, London, U.K., 1998, pp. 287-300.
36. J. Furnkranz, "Separate-and-conquer rule learning," Artif. Intell. Rev., vol. 13, no. 1, pp. 3-54, 1999.
37. J. Furnkranz and G. Widmer, "Incremental reduced error pruning," in Machine Learning: Proc. 11th Annu. Conf., New Brunswick, NJ, 1994, pp. 70-77.
38. L. Hall, N. Chawla, K. Bowyer, and W. Kegelmeyer, "Learning rules from distributed data," in Proc. Workshop on Large-Scale Parallel KDD Systems, KDD'99, 1999, pp. 77-83.
39. S. Hettich and S. D. Bay. (1999) The UCI KDD Archive. University of California, Dept. Inform. Comput. Sci., Irvine, CA. [Online]. Available: http://kdd.ics.uci.edu
40. M. Holsheimer and A. P. Siebes, "Data Mining: The Search for Knowledge in Databases,", Tech. Rep. CS-R9406, 1994.
41. R. C. Holte, "Very simple classification rules perform well on most commonly used datasets," Mach. Learn., vol. 11, pp. 63-90, 1993.
42. K. A. Kaufman and R. S. Michalski, "Learning from inconsistent and noisy data: The AQ18 approach," in Proc. 11th Int. Symp. Methodologies for Intelligent Systems, Warsaw, Poland, 1999, pp. 411-419.
43. R. Kerber, "ChiMerge: Discretization of numeric attributes," in Proc. 9th Int. Conf. Artificial Intelligence (AAAI-91), 1992, pp. 123-128.
44. Y. Kodratoff, Introduction to Machine Learning. New York: Morgan-Kaufmann, 1988.
45. R. Kufrin, "Generating C4.5 production rules in parallel," in Proc. 14th Nat. Conf. Artificial Intelligence (AAAI-97), 1997, pp. 565-670.
46. L. A. Kurgan and K. J. Cios, "CAIM discretization algorithm," IEEE Trans. Knowl. Data Eng., vol. 16, no. 2, pp. 145-153, Feb. 2004.
47. L. A. Kurgan and K. J. Cios, "Discretization algorithm that uses class-attribute interdependence maximization," in Proc. 2001 Int. Conf. Artificial Intelligence (IC-AI 2001), Las Vegas, NV, 2001, pp. 980-987.
48. L. A. Kurgan and K. J. Cios, "Ensemble of classifiers to improve accuracy of the CLIP4 machine learning algorithm," in Proc. SPIE's Int. Conf. Sensor Fusion: Architectures, Algorithms, and Applications VI, AeroSence 2002, Orlando, FL, 2002, pp. 22-31.
49. L. A. Kurgan and K. J. Cios, "Fast class-attribute interdependence maximization (CAIM) discretization algorithm," in Proc. 2003 Int. Conf. Machine Learning and Applications (ICMLA'03), Los Angeles, CA, 2003, pp. 30-36.
50. L. A. Kurgan and K. J. Cios, "Meta mining architecture for supervised learning," in Proc. 7th Int. Workshop on High Performance and Distributed Mining, in conjunction with the 4th Int. SIAM Conf. Data Mining, Lake Buena Vista, FL, 2004, pp. 18-26.
51. L. A. Kurgan, K. J. Cios, M. Sontag, and F. J. Accurso, "Mining the cystic fibrosis data," in Next Generation of Data-Mining Applications, J. Zurada and M. Kantardzic, Eds. New York: IEEE Press, 2005, pp. 415-444.
52. L. A. Kurgan, "Meta Mining System for Supervised Learning," Ph.D, Dept. Comput. Sci., Univ. Colorado at Boulder, 2003.
53. P. Langley and H. Simon, "Applications of machine learning and rule induction," Commun. ACM, vol. 38, no. 11, pp. 55-64, 1995.
54. P. Langley, Elements of Machine Learning. New York: Morgan-Kaufmann, 1996.
55. T.-S. Lim, W.-Y. Loh, and Y.-S. Shih, "A comparison of prediction accuracy, complexity, and training time of thirty-three old and new classification algorithms," Mach. Learn., vol. 40, pp. 203-228, 2000.
56. R. S. Michalski, I. Mozetic, J. Hong, and N. Lavrac, "The multipurpose incremental learning system AQ15 and its testing application to three medical domains," in Proc. 5th Nat. Conf. Artificial Intelligence, 1986, pp. 1041-1045.
57. T. M. Mitchell, Machine Learning. New York: McGraw-Hill, 1997.
58. G. Pagallo and D. Haussler, "Boolean feature discovery in empirical learning," Mach. Learn., vol. 5, no. 1, pp. 71-99, 1990.
59. Machine Learning and its Applications, vol. 2049, Lecture Notes in Computer Science, G. Paliouras, V. Karkaletsis, and C. Spyropoulos, Eds., Springer, Berlin, Germany, 2001.
60. M. Pazzani and D. Kibler, "The utility of knowledge in inductive learning," Mach. Learn., vol. 9, no. 1, pp. 57-94, 1992.
61. F. Provost and D. Hennessy, "Scaling up: Distributed machine learning with cooperation," in Proc. 13th Nat. Conf. Artificial Intelligence, 1996, pp. 74-79.
62. F. Provost and T. Fawcett, "Analysis and visualization of classifier performance: Comparison under imprecise class and cost distribution," in Proc. 3rd Int. Conf. Knowledge Discovery and Data Mining, 1997, pp. 43-48.
63. F. Provost and V. Kolluri, "A survey of methods for scaling up inductive algorithms," Data Mining Knowl. Disc., vol. 3, no. 2, pp. 131-169, 1999.
64. F. Provost and T. Fawcett, "Robust classification for imprecise environments," Mach. Learn., vol. 42, no. 3, pp. 203-231, 2001.
65. J. R. Quinlan, C4.5 Programs for Machine Learning. New York: Morgan-Kaufmann, 1993.
66. J. R. Quinlan, "Improved use of continuous attributes in C4.5," J. Artif. Intell. Res., vol. 7, pp. 77-90, 1996.
67. J. R. Quinlan, "Induction of decision trees," Mach. Learn., vol. 1, pp. 81-106, 1986.
68. J. R. Quinlan, "Learning logical definitions from relations," Mach. Learn., vol. 5, pp. 239-266, 1990.
69. J. R. Quinlan, "Simplifying decision trees," Int. J. Man-Mach. Stud., vol. 27, no. 3, pp. 221-248, 1987.
70. J. R. Quinlan, "Some elements of machine learning," in Proc. 16th Int. Conf. Machine Learning, 1999, pp. 523-525.
71. S. Ranka, "Real-time data mining on grids," in (Invited Presentation) 7th Int. Workshop on High Performance and Distributed Mining, in Conjunction with the 4th Int. SIAM Conf. Data Mining, Lake Buena Vista, FL, 2004.
72. J. F. Roddick and M. Spiliopoulou, "A survey of temporal knowledge discovery paradigms and methods," IEEE Trans. Knowl. Data Eng., vol. 14, no. 4, pp. 750-767, Apr. 2002.
73. RuleQuest Research, C5.0 (2003). [Online]. Available: http://www.rule-quest.com/see5-info.html
74. R. E. Schapire and Y. Singer, "Improved boosting algorithms using confidence-rated predictions," in Proc. 11th Annu. Conf. Computational Learning Theory, 1998, pp. 80-91.
75. M. Sebag, "Delaying the choice of bias: A disjunctive version space approach," in Proc: 13th Int. Conf. Machine Learning, 1996, pp. 444-452.
76. M. Spiliopoulou and J. F. Roddick, "Higher order mining: Modeling and mining the results of knowledge discovery," in Data Mining I: Proc. 2nd Int. Conf. Data Mining Methods and Databases, 2000, pp. 309-320.
77. H. Theron and I. Cloete, "BEXA: A covering algorithm for learning propositional concept descriptions," Mach. Learn., vol. 24, pp. 5-40, 1996.
78. V. N. Vapnik, The Nature of Statistical Learning Theory. New York: Springer, 1995.
79. P. Vlachos. (2000) StatLib Project Repository. [Online]. Available: http://lib.stat.cmu.edu
80. G. I. Webb. and J. Agar, "Inducing diagnostic rules for glomerular disease with the DLG machine learning algorithm," Artif. Intell. Med., vol. 4, pp. 3-14, 1992.
81. S. Weiss and N. Indurkhya, "Reduced complexity rule induction," in Proc. 12th Int. Joint Conf. Artificial Intelligence, Sydney, Australia, 1991, pp. 678-684.
82. R. Winter and K. Auerbach, "Contents under pressure," Intell. Enterprise, May 2004.