Computational intelligence methods for rule-based data understanding

Proceedings of the IEEE

Volumn 92, Issue 5, 2004, Pages 771-805

  Duch, WŁodzisŁaw ^a,b   Setiono, Rudy ^c   Zurada, Jacek M ^d  

^a NICOLAUS COPERNICUS UNIVERSITY   (Poland)

^b NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

^c NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^d University of Louisville   (United States)

Author keywords

Data mining; Decision support; Decision trees; Feature selection; Fuzzy systems; Inductive learning; Logical rule extraction; Machine learning (ML); Neural networks, neurofuzzy systems

Indexed keywords

DATABASE SYSTEMS; ERROR ANALYSIS; FUZZY SETS; LEARNING SYSTEMS; NEURAL NETWORKS; OPTIMIZATION; STATISTICAL METHODS; TREES (MATHEMATICS); VISUALIZATION;

DECISION TREES; FEATURE SELECTION; FUZZY SYSTEMS; INDUCTIVE LEARNING; LOGICAL RULE EXTRACTION; NEUROFUZZY SYSTEMS;

DATA MINING;

EID: 21244495584     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/JPROC.2004.826605     Document Type: Conference Paper

References (156)

1. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning. New York: Springer-Verlag. 2001.
2. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed. New York: Wiley, 2001.
3. T. Oates and D. Jensen, "Large datasets lead to overly complex models: An explanation and a solution," in Proc. 4th int. Conf. Knowledge Discovery and Data Mining, 1998, pp. 294-298.
4. R. Andrews, J. Diederich, and A. B. Tickle, "A survey and critique of techniques for extracting rules from trained artificial neural networks," Knowl.-Based Syst., vol. 8, pp. 373-389, 1995.
5. R. Michalski, "A theory and methodology of inductive learning," Artif. Intell., vol. 20, pp. 111-161, 1983.
6. I. Roth and V. Bruce, Perception and Representation, 2nd ed. Maidenhead, U.K.: Open Univ. Press, 1995.
7. J. Tukey, Exploratory Data Analysis. Menlo Park. CA: Addison Wesley, 1977.
8. M. Jambu, Exploratory and Multivariate Data Analysis. Boston, MA: Academic, 1991.
9. W. Duch, "Coloring black boxes: Visualization of neural network decisions," in Proc. Int. Joint Conf. Neural Networks, vol. 1, 2003, pp. 1735-1740.
10. M. Jordan and T. J. Sejnowski, Eds., Graphical Models: Foundations of Neural Computation. Cambridge, MA: MIT Press, 2001.
11. W. Duch, R. Adamczak, and K. Grabczewski, "A new methodology of extraction, optimization and application of crisp and fuzzy logical rules," IEEE Trans. Neural Networks, vol. 12, pp. 277-306, Mar. 2001.
12. T. Fawcett, "Using rule sets to maximize ROC performance," in Prof. IEEE Int. Conf. Data Mining, 2001. pp. 131-138.
13. N. Kasabov, Foundations of Neural Networks, Fuzzy Systems and Knowledge Engineering. Cambridge, MA: MIT Press, 1996.
14. V. Kecman, Learning and Soft Computing. Cambridge, MA: MIT Press, 2001.
15. B. Kosko, Neural Networks and Fuzzy Systems. Engelwood Cliffs, NJ: Prentice-Hall, 1992.
16. L. A. Zadeh, "Fuzzy sets," Inf. Control, vol. 8, pp. 338-353, 1965.
17. T. Bilgiç and I. B. Türkşcn. "Measurements of membership functions: Theoretical and empirical work," in Fundamentals of Fuzzy Sets, D. Dubois and H. Prade, Eds. Boston, MA: Kluwer, 2000, vol. 1, pp. 195-232.
18. W. Duch and K. Grudziński, "Prototype based rules - new way to understand the data," in Proc. Int. Joint Conf. Neural Networks, 2001, pp. 1858-1863.
19. Z. Pawlak, Rough Sets - Theoretical Aspects of Reasoning About Data. Boston, MA: Kluver, 1991.
20. S. K. Pal and A. Skowron, Rough Fuzzy Hybridization: A New Trend in Decision-Making. New York: Springer-Verlag, 1999.
21. L. I. Kuncheva, "How good are fuzzy if-then classifiers?," IEEE Trans. Syst., Man. Cybern. B, vol. 30, pp. 501-509, Aug. 2000.
22. O. Sourina and S. H. Boey, "Geometric query model for scientific and engineering databases," Int. J. Inf. Technol., vol. 2, pp. 41-54, 1996.
23. W. Duch and G. H. F. Diercksen, "Feature space mapping as a universal adaptive system," Comput. Phys. Commun., vol. 87, pp. 341-371, 1995.
24. T. Mitchell, Machine Learning. New York: McGraw-Hill, 1997.
25. A. F. Bowers, C. Giraud-Carrier, and J. W. Lloyd, "Classification of individuals with complex structure," in Proc. 17th Int. Conf. Machine Learning (ICML 2000), pp. 81-88.
26. J. R. Quinlan and R. M. Cameron-Jones, "Induction of logic programs: FOIL and related systems," New Gener. Comput., vol. 13, pp. 287-312, 1995.
27. W. E. Combs and J. E. Andrews, "Combinatorial rule explosion eliminated by a fuzzy rule configuration," IEEE Trans. Fuzzy Syst., vol. 6, pp. 1-11, Feb. 1998.
28. M. K. Güven and K. M. Passino, "Avoiding exponential parameter growth in fuzzy systems," Trans. IEEE Fuzzy Syst., vol. 9, pp. 194-199, Feb. 2001.
29. W. Duch, R. Adamczak, and K. Grabczewski, "Neural optimization of linguistic variables and membership functions," in Proc. Int. Conf. Neural Information Processing, vol. 2, 1999, pp. 616-621.
30. D. Hand, H. Mannila, and P. Smyth, Principles of Data Mining. Cambridge, MA: MIT Press, 2001.
31. D. Dougherty, R. Kohavi, and M. Sahami, "Supervised and unsupervised discretization of continuous features," presented at the Machine Learning 12th Int. Conf., San Mateo, CA, 1995.
32. R. C. Holte, "Very simple classification rules perform well on most commonly used datasets," Mach. Learn., vol. 11, pp. 63-91, 1993.
33. U. M. Fayyad and K. B. Irani, "Multi-interval discretization of continuous valued attributes for classification learning," presented at the 13th Int. Joint Conf. Artificial Intelligence, San Mateo, CA, 1993.
34. R. Kohavi and M. Sahami, "Error-based and entropy-based discretization of continuous features," in Proc. 2nd Int. Conf. Knowledge Discovery and Data Mining, 1996, pp. 114-119.
35. S. Monti and G. F. Cooper, "A latent variable model for multivariate discretization," presented at the Uncertainly 99: The 7th Int. Workshop Artificial Intelligence and Statistics, Fort Lauderdale, FL. 1999.
36. H. Liu and R. Setiono, "Dimensionality reduction via discretization," Knowl.-Based Syst., vol. 9, pp. 67-72, 1996.
37. K. Grabczewski and W. Duch, "A general purpose separability criterion for classification systems," in Proc. 4th Conf. Neural Networks and Their Applications, 1999, pp. 203-208.
38. W. Duch, R. Adamczak, and N. Jankowski, "Initialization of adaptive parameters in density networks," in Proc. 3rd Conf. Neural Networks, 1997, pp. 99-104.
39. _, "New developments in the feature space mapping model," in Proc. 3rd Conf. Neural Networks, 1997, pp. 65-70.
40. W. Duch and N. Jankowski, "A survey of neural transfer functions," Neural Comput. Surv., vol. 2, pp. 163-213, 1999.
41. J. R. Quinlan, "Induction of decision trees," Mach. Learn., vol. 1, pp. 81-106, 1986.
42. _, C4.5: Programs for Machine Learning. San Mateo, CA: Morgan Kaufman, 1993.
43. K. Grabczewski and W. Duch, "Heterogenous forests of decision trees," in Lecture Notes in Computer Science, Artificial Neural Networks. London, U.K.: Springer-Verlag, 2002, vol. 2415, pp. 504-509.
44. C. E. Brodley and P. E. Utgoff, "Multivariate decision trees," Mach. Learn., vol. 19, pp. 45-77, 1995.
45. D. Heath, S. Kasif, and S. Salzberg, "Learning oblique decision trees," in Proc. 13th Int. Joint Conf. Artificial Intelligence, 1993, pp. 1002-1007.
46. S. Murthy, S. Kasif, S. Salzberg, and R. Beigel, "OC1: Randomized induction of oblique decision trees," in Proc. AAAI Conf., 1993, pp. 322-327.
47. O. L. Mangasarian, R. Setiono, and W. Wolberg, "Pattern recognition via linear programming: Theory and application to medical diagnosis," in Large-Scale Numerical Optimization, T. F. Coleman and Y. Li, Eds. Philadelphia, PA: SIAM, 1989, pp. 22-30.
48. R. Setiono and H. Liu, "A connectionist approach to generating oblique decision trees," IEEE Trans. Syst., Man, Cybern. B, vol. 29, pp. 440-444, June 1999.
49. L. Breiman, J. H. Friedman, R. A. Oslhen, and C. J. Stone, Classification and Regression Trees. Belmont, CA: Wadsworth, 1984.
50. L. Breiman, "Bias-variance, regularization, instability and stabilization," in Neural Networks and Machine Learning, C. Bishop, Ed. New York: Springer-Verlag, 1998.
51. R. S. Michalski, "On the quasiminimal solution of the general covering problem," in Proc. 5th Int. Symp. Information Processing, 1969, pp. 125-128.
52. R. S. Michalski and K. A. Kaufman, "Data mining and knowledge discovery: A review of issues and a multistrategy approach," in Machine Learning and Data Mining: Methods and Applications, R. S. Michalski, I. Bratko, and M. Kubat, Eds. New York: Wiley, 1997.
53. P. Clark and T. Niblett, "The CN2 induction algorithm," Mach. Learn., vol 3, pp. 261-283, 1988.
54. W. W. Cohen, "Fast effective rule induction," in Proc. 12th Int. Conf. Machine Learning, 1995, pp. 115-123.
55. T. M. Mitchell, "Generalization as search," Artif. Intell., vol. 18, pp. 203-226, 1982.
56. N. Lavrac and S. Dzeroski, Inductive Logic Programming: Techniques and Applications. Chichester, U.K.: Elus Horwood, 1994.
57. H. Lu, R. Setiono, and H. Liu, "Effective data mining using neural networks," IEEE Trans. Knowledge Data Eng., vol. 8, pp. 957-961, Dec. 1996.
58. R. Setiono and H. Liu, "Symbolic representation of neural networks," IEEE Computer, vol. 29, pp. 71-77, Mar. 1996.
59. R. Setiono, "Extracting M-of-N rules from trained neural networks," IEEE Trans. Neural Networks, vol. 11, pp. 306-512, Mar. 2001.
60. R. Setiono and W. K. Leow, "FERNN: An algorithm for fast extraction of rules from neural networks," Appl. Intel., vol. 12, pp. 15-25, 2000.
61. A. B. Tickle, R. Andrews, M. Golea, and J. Diederich, "The truth will come to light: Directions and challenges in extracting the knowledge embedded within trained artificial neural networks," IEEE Trans. Neural Networks, vol. 9, pp. 1057-1068, Nov. 1998.
62. K. Saito and R. Nakano, "Medical diagnostic expert system based on PDP model," in Proc. IEEE Int. Conf. Neural Networks, vol. 1, 1988, pp. 255-262.
63. S. Gallant, Neural Network Learning and Expert Systems. Cambridge, MA: MIT Press, 1993.
64. S. Thrun, "Extracting rules from artilcial neural networks with distributed representations," in Advances in Neural Information Processing Systems 7, G. Tesauro, D. Touretzky, and T. Leen, Eds. Cambridge, MA: MIT Press, 1995.
65. C. A. Jensen, R. D. Reed, R. J. Marks II, M. A. El-Sharkawi, J.-B. Jung, R. T. Miyamoto, G. M. Anderson, and C. J. Eggen, "Inversion of feedforward neural networks: Algorithms and applications," Proc. IEEE, vol. 87, pp. 1536-1549, Sept. 1999.
66. C. Hernández-Espinosa, M. Fernández Redondo, and M. Ortiz-Gómez, "Inversion of a neural network via interval arithmetic for rule extraction," in Lecture Notes in Computer Science, Artificial Neural Networks and Neural Information Processing, 2003, vol. 2714, pp. 670-677.
67. S. H. Huang and H. Xing, "Extract intelligible and concise fuzzy rules from neural networks," Fuzzy Sets Syst., vol. 132, pp. 233-243, 2002.
68. E. M. Fu, "Rule learning by searching on adapted nets," in Proc. 9th Nat. Conf. Artificial Intelligence, 1991, pp. 590-595.
69. _, "Knowledge-based connectionism for revising domain theories," IEEE Trans. Syst., Man. Cybern., vol. 23, pp. 173-182, Jan.-Feb. 1993.
70. _, Neural Networks in Computer Intelligence. New York: Mc-Graw Hill, 1994.
71. _, "Rule generation from neural networks," IEEE Trars. Syst., Man. Cybern., vol. 28, pp. 1114-1124, Aug. 1994.
72. R. Setiono, "Extracting rules from neural networks by pruning and hidden-unit splitting," Neural Comput., vol. 9, no. 1, pp. 205-225, 1997.
73. I. K. Sethi and J. H. Yoo, "Symbolic approximation of feedforward neural networks," in Pattern Recognition in Practice, E. S. Gelsema and L. N. Kanal, Eds. New York: North-Holland, 1994, vol. 4.
74. G. Towell and J. Shavlik, "Extracting refined rules from knowledge-based neural networks," Mach. Learn., vol. 13, pp. 71-101, 1993.
75. Y. Hayashi, "A neural expert system with automated extraction of fuzzy if-then rules," in Advances in Neural Information Processing Systems, R. Lippmann, J. Moody, and D. Touretzky, Eds. San Mateo, CA: Morgan Kaufmann, 1991, vol. 3, pp. 578-584.
76. G. Towell and J. Shavlik, "Knowledge-based artificial neural networks," Artif. Intell., vol. 70, pp. 119-165, 1994.
77. C. McMillan, M. C. Mozer, and P. Smolensky, "Rule induction through integrated symbolic and subsymbolic processing," in Advances in Neural Information Processing Systems, J. Moody, S. Hanson, and R. Lippmann, Eds, San Mateo, CA: Morgan Kaufmann, 1992, vol. 4, pp. 969-976.
78. J. A. Alexander and M. C. Mozer, "Template-based algorithms for connectionist rule extraction," in Advances in Neural Information Processing Systems, G. Tesauro, D. Touretzky, and T. Leen, Eds. Cambridge, MA: MIT Press, 1995, vol. 7, pp. 609-616.
79. R. Setiono and H. Liu, "Understanding neural networks via rule extraction," in Proc. 14th Int. Joint Conf. Artificial Intelligence, 1995, pp. 480-485.
80. M. Ishikawa, "Rule extraction by succesive regularization," in Proc. 1996 IEEE Int. Conf. Neural Networks, 1996, pp. 1139-1143.
81. P. Géczy and S. Usui, "Rule extraction from trained neural networks," Behaviormetrika, vol. 26, pp. 89-106, 1999.
82. W. Duch, R. Adamczak, and K. Grabczewski. (1996) Extraction of logical rules from training data using backpropagation networks. Proc. 1st Online Workshop Soft Computing [Online], pp. 25-30. Available: http://www.bioele.nuee. nagoya-u.ac.jp/wsc1/
83. M. W. Craven and J. W. Shavlik, "Using sampling and queries to extract rules from trained neural networks," in Proc. 11th Int. Conf. Machine Learning, 1994, pp. 37-45.
84. E. Pop, R. Hayward, and J. Diederich, "RULENEG: Extracting rules from a trained ANN by stepwise negation," Neurocomputing Res. Centre. Queensland Univ. Technol., Brisbane, Qld., Aust., OUT NRC Tech. Rep., 1994.
85. S. Sestito and T. Dillon, Automated Knowledge Acquisition. Upper Saddle River, NJ: Prentice-Hall, 1994.
86. M. J. Healy and T. P. Caudell, "Acquiring rule sets as a product of learning in a logical neural architecture," IEEE Trans. Neural Networks, vol. 8, pp. 461-174, May 1997.
87. A.-H. Tan, "Rule learning and extraction with self organizing neural networks," in Proc. 1993 Connectionist Models Summer School, pp. 192-199.
88. A. Ultsch, "Knowledge extraction from self-organizing neural networks." in Information and Classification, O. Opitz, B. Lausen, and R. Klar, Eds. Berlin, Germany: Springer-Verlag. 1993, pp. 301-306.
89. D. Michie, D. J. Spiegelhalter, and C. C. Taylor, Machine Learning, Neural and Statistical Classification. London, U.K.: Ellis Horwood, 1994.
90. A. B. Tickle, M. Orlowski, and J. Diederich, "DEDEC: Decision detection by rule extraction from neural networks," Neurocomputing Res. Centre, Queensland Univ. Technol., Brisbane, Qld., Aust., QUT NRC Tech. Rep., 1994.
91. M. W. Craven and J. W. Shavlik, "Extracting tree-structured representations of trained networks," in Advances in Neural Information Processing Systems, D. Touretzky, M. Mozer, and M. Hasselmo, Eds. Cambridge, MA: MIT Press, 1996, vol. 8, pp. 24-30.
92. J.-S. R. Jang and C. T. Sun, "Functional equivalence between radial basis function neural networks and fuzzy inference systems," IEEE Trans. Neural Networks, vol. 4, pp. 156-158, Jan. 1993.
93. V. Tresp, J. Hollatz, and S. Ahmad, "Network structuring and training using Rule based knowledge," in Advances in Neural Information Processing Systems, J. Moody, S. Hanson, and R. Lippmann, Eds. San Mateo, CA: Morgan Kaufmann, 1993, vol. 4, pp. 871-878.
94. W. Duch, "Floating Gaussian mapping: A new model of adaptive systems," Neural Netw. World, vol. 4, pp. 645-654, 1994.
95. R. Andrews and S. Geva, "Rules and local function networks," presented at the Rule Extraction From Trained Artificial Neural Networks Workshop, R. Andrews and J. Diederich, Eds., Brighton, U.K., 1996.
96. _, "Refining expert knowledge with an artificial neural network," in Proc. Int. Conf. Neural Information Processing, vol. 2, 1997, pp. 847-850.
97. _, "Rule extraction from a constrained error back propagation MLP," in Proc. 5th Aust. Conf. Neural Networks, 1994, pp. 9-12.
98. P. Langley, H. A. Simon, G. L. Bradshaw, and J. M. Zytkow, Scientific Discovery: Computational Explorations of the Creative Processes. Cambridge, MA: MIT Press, 1987.
99. J. J. Mahoney and R. J. Mooney, "Combining neural and symbolic learning to revise probabilistic rule bases," in Advances in Neural Information Processing Systems, S. J. Hanson, J. D. Cowan, and C. L. Giles, Eds. San Mateo, CA: Morgan Kaufmann, 1993, vol. 5, pp. 107-114.
100. D. Nauck, F. Klawonn, and R. Kruse, Foundations of Neuro-Fuzzy Systems. Chichester, U.K.: Wiley, 1997.
101. D. Nauck, U. Nauck, and R. Kruse, "Generating classification rules with the neuro-fuzzy system NEFCLASS," presented at the Biennial Conf. North Amer. Fuzzy Information Processing Soc. (NAFIPS'96), Berkeley, CA.
102. S. K. Halgamuge and M. Glesner, "Neural networks in designing fuzzy systems for real world applications," Fuzzy Sets Syst., vol. 65, pp. 1-12, 1994.
103. J. M. Żurada and A. Łozowski, "Generating linguistic rules from data using neuro-fuzzy framework," in Proc. 4th Int. Conf. Soft Computing, vol. 2, 1996, pp. 618-621.
104. H. Surmann and M. Maniadakis, "Learning feed-forward and recurrent fuzzy systems: A genetic approach," J. Syst. Architect., vol. 47, pp. 649-662, 2001.
105. W. Duch, R. Adamczak, and K. Grabczewski, "Extraction of logical rules from backpropagation networks," Neural Process. Lett., vol. 7, pp. 1-9, 1998.
106. J. M. Żurada, Introduction to Artificial Neural Systems. St. Paul, MN: West, 1992.
107. D. J. MacKay, "A practical Bayesian framework for backpropagation networks," Neural Comput., vol. 4, pp. 448-472, 1992.
108. C. Bishop, Neural Networks for Pattern Recognition. Oxford, U.K.: Clarendon Press, 1995.
109. W. Duch and K. Grabczewski, "Searching for optimal MLP," in Proc. 4th Conf. Neural Networks and Their Applications. 1999, pp. 65-70.
110. M. Kordos and W. Duch, "Multilayer perceptron trained with numerical gradient," in Proc. Int. Conf. Artificial Neural Networks (ICANN) and Int. Conf. Neural Information Processing (ICONIP), 2003, pp. 106-109.
111. _, "Search-based training for logical rule extraction by multi-layer perceptron," in Proc. Int. Conf. Artificial Neural Networks (ICANN) and Int. Conf. Neural Information Processing (ICONIP), 2003, pp. 86-89.
112. L. Kanal and V. Kumar, Eds., Search in Artificial Intelligence. New York: Springer-Verlag, 1988.
113. W. Duch, N. Jankowski, K. Grabczewski, and R. Adamczak, "Optimization and interpretation of rule-based classifiers," in Advances in Soft Computing. Berlin, Germany: Physica-Verlag, 2000, pp. 1-13.
114. W. Duch, R. Adamczak, and K. Grabczewski, "Optimization of logical rules derived by neural procedures," presented at the Int. Joint Conf. Neural Networks, Washington, DC, 1999, paper no. 741.
115. J. A. Hanley and B. J. McNeil, "The meaning and use of the area under a receiver operating characteristic (ROC) curve," Radiology, vol. 143, pp. 29-36, 1982.
116. S. K. Pal and S. Mitra, Neuro-Fuzzy Pattern Recognition. New York: Wiley, 1999.
117. R. J. Marks, II, S. Oh, P. Arabshahi, T. P. Caudell, J. J. Choi, and B. J. Song, "Steepest descent adaptation of min-max fuzzy if-then rules," presented at the Int. Joint Conf. Neural Networks, Beijing, China, 1992.
118. UCI repository of machine learning databases, J. Mertz and P. M. Murphy. [Online]. Available: http://www.ics.uci.edu/pub/machine-learning-data-bases
119. D. Nauck, U. Nauck, and R. Kruse, "Generating classification rules with the neuro-fuzzy system NEFCLASS," presented at the Biennial Conf. North Amer. Fuzzy Information Processing Soc. (NAFIPS'96), Berkeley, CA.
120. S. K. Halgamuge and M. Glesner, "Neural networks in designing fuzzy systems for real world applications," Fuzzy Sets Syst., vol. 65, pp. 1-12, 1994.
121. C. Browne, I. Düntsch, and G. Gediga, "IRIS revisited: A comparison of discriminant and enhanced rough set data analysis," in Rough Sets in Knowledge Discovery, L. Polkowski and A. Skowron, Eds. Heidelberg, Germany: Physica-Verlag, 1998, vol. 2, pp. 345-368.
122. I. Jagielska, C. Matthews, and T. Whitfort, "The application of neural networks, fuzzy logic, genetic algorithms and rough sets to automated knowledge acquisition," in Proc. 4th Int. Conf. Soft Computing, vol. 2, 1996, pp. 565-569.
123. N. Kasabov, R. Kozma, and W. Duch, "Rule extraction from linguistic rule networks and from fuzzy neural networks: Prepositional versus fuzzy rules," in Proc. 4th Int. Conf. Neural Networks and Their Applications, 1998, pp. 403-406.
124. J. Teghem and M. Benjelloun, "Some experiments to compare rough sets theory and ordinal statistical methods," in Intelligent Decision Support: Handbook of Applications and Advances of Rough Set Theory, R. Słowiński, Ed. Dordrecht, The Netherlands: Kluwer, 1992, vol. 11, System Theory, Knowledge Engineering and Problem Solving, pp. 267-284.
125. W. Duch, R. Adamczak, K. Grabczewski, M. Ishikawa, and H. Ueda, "Extraction of crisp logical rules using constrained backpropagalion networks-comparison of two new approaches," in Proc. Eur. Symp. Artificial Neural Networks (ESANN'97), pp. 109-114.
126. S. M. Weiss and C. A. Kulikowski, Computer Systems That Learn. San Mateo, CA: Morgan Kauffman, 1990.
127. R. S. Míchalskí, I. Mozetic, J. Hong, and N. Lavrac, "The multi-purpose incremental learning system AQ15 and its testing application to three medical domains," in Proc. 5th Nat. Conf. Artificial Intelligence, 1986, pp. 1041-1045.
128. M. Tan and L. Eshelman, "Using weighted networks to represent classification knowledge in noisy domains," in Proc. 5th Int. Conf. Machine Learning. 1988, pp. 121-134.
129. G. Cestnik, I. Konenenko, and I. Bratko, "Assistant-86: A knowledge-elicilation tool for sophisticated users," in Progress in Machine Learning, I. Bratko and N. Lavrac, Eds. Wilmslow, U.K.: Sigma, 1987, pp. 31-45.
130. J. W. Grzymała-Busse and T. Soe, "Inducing simpler rules from reduced data," in Proc. Workshop Intelligent Information Systems VII, 1998, pp. 371-378.
131. F. Zarndt, "A comprehensive case study: An examination of machine learning and connectionist algorithms," M.Sc. Thesis, Dept. Comput. Sci., Brigham Young Univ., Provo, UT, 1995.
132. K. P. Bennett and O. L. Mangasarian, "Robust linear programming discrimination of two linearly inseparable sets," Optim. Methods Softw., vol. 1, pp. 23-34, 1992.
133. R. Setiono, "Generating concise and accurate classification rules for breast cancer diagnosis," Artif. Intell. Med., vol. 18, pp. 205-219, 2000.
134. B. Ster and A. Dobnikar et al., "Neural networks in medical diagnosis: Comparison with other methods," in Proc. Int. Conf. EANN'96, A. Bulsari et al., Eds., pp. 427-430.
135. N. Shang and L. Breiman, "Distribution based trees are more accurate," in Proc. Int. Conf. Neural Information Processing, vol. I, 1996, pp. 133-138.
136. B. D. Ripley, Pattern Recognition and Neural Networks. Cambridge, U.K.: Cambridge Univ. Press, 1996.
137. N. Jankowski and V. Kadirkamanathan, "Statistical control of RBF-like networks for classification," in Proc. 7th Int. Conf. Artificial Neural Networks, 1997, pp. 385-390.
138. Ghostminer software, W. Duch, N. Jankowski, K. Grabczewski, A. Naud, and R. Adamczak. [Online]. Available: http://www.fqspl.com.pl/ghostminer/
139. S. M. Weiss and I. Kapouleas, "An empirical comparison of pattern recognition, neural nets and machine learning classification methods." in Readings in Machine Learning, J. W. Shavlik and T. G. Dietterich, Eds. San Mateo, CA: Morgan Kauffman, 1990.
140. W. Schiffman, M. Joost, and R. Werner, "Comparison of optimized backpropagation algorithms," in Proc. Eur. Symp. Artificial Neural Networks, 1993, pp. 97-104.
141. W. Duch, R. Adamczak, K. Grabczewski, and G. Żal, "Hybrid neural-global minimization method of logical rule extraction," Int. J. Adv. Comput. Intell., vol. 3, pp. 348-356, 1999.
142. Y. Hayashi, A. Imura, and K. Yoshida, "Fuzzy neural expert system and its application to medical diagnosis," in Proc. 8th Int. Congr. Cybernetics and Systems, 1990, pp. 54-61.
143. S. Mitra, R. De, and S. Pal, "Knowledge based fuzzy MLP for classification and rule generation," IEEE Trans. Neural Networks, vol. 8, pp. 1338-1350, Nov. 1997.
144. Y. Hayashi, R. Setiono, and K. Yoshida, "A comparison between two neural network rule extraction techniques for the diagnosis of hepatobiliary disorders," Artif. Intell. Med., vol. 20, pp. 205-216, 2000.
145. W. Duch, R. Adamczak, K. Grabczewski, G. Żal, and Y. Hayashi, "Fuzzy and crisp logical rule extraction methods in application to medical data," in Fuzzy Systems in Medicine, P. S. Szczepaniak, P. J. G. Lisboa, and J. Kacprzyk, Eds. Berlin, Germany: Physica-Verlag, 2000, pp. 593-616.
146. W. Duch, R. Adamczak, and Y. Hayashi, "Neural eliminators and classifiers," in Proc. 7th Int. Conf. Neural Information Processing (ICONIP 2000), S.-Y. Lee, Ed., pp. 1029-1034.
147. J. N. Butcher and S. V. Rouse, "Personality: Individual differences and clinical assessment," Annu. Rev. Psychol., vol. 47, pp. 87-111, 1996.
148. W. Duch, R. Adamczak, and K. Grabczewski, "Neural methods for analysis of psychometric data," in Proc. Int. Conf. Engineering Applications of Neural Networks (EANN'99), pp. 45-50.
149. R. R. Yager and D. Filev, "Relational partitioning of fuzzy rules," Fuzzy Sets Syst., vol. 80, pp. 57-69, 1996.
150. A. E. Gaweda and J. M. Zurada, "Fuzzy neural network with relational fuzzy rules," in Proc. Int. Joint Conf. Neural Networks (IJCNN'00), vol. 3, pp. 1-2.
151. _, "Data-driven linguistic modeling using relational fuzzy rules," IEEE Trans. Fuzzy Syst., vol. 11, pp. 121-134, Feb. 2003.
152. J. C. Bezdek, Pattern Recognition With Fuzzy Objective Function Algorithms. New York: Plenum, 1981.
153. J. S. R. Jang, C.-T. Sun, and E. Mizutani, Neuro-Fuzzy and Soft-Computing. Upper Saddle River, NJ: Prentice-Hall, 1997.
154. L. Goldfarb, J. Abela, V. C. Bhavsar, and V. N. Kamat, "Can a vector space based learning model discover inductive class generalization in a symbolic environment?," Pattern Recognit. Lett., vol. 16, pp. 719-726, 1995.
155. P. Frasconi, M. Gori, and A. Sperduti, "A general framework for adaptive processing of data structures," IEEE Trans. Neural Networks, vol. 9, pp. 768-786, Sept. 1998.
156. R. Hayward, C. Ho-Stuart, J. Diederich, and E. Pop, "RULENEG: Extracting rules from a trained ANN by stepwise negation," Neurocomputing Res. Centre, Queensland Univ. Technol., Brisbane, Qld., Aust., QUT NRC Tech. Rep., 1996.