Is it worth generating rules from neural network ensembles?

Journal of Applied Logic

Volumn 2, Issue 3, 2004, Pages 325-348

  Bologna, Guido ^a  

^a SWISS INSTITUTE OF BIOINFORMATICS   (Switzerland)

Author keywords

Decision trees; Ensembles; Neural networks; Proteins; Rule extraction

Indexed keywords

ALGORITHMS; COMPUTATIONAL COMPLEXITY; MATHEMATICAL MODELS; TREES (MATHEMATICS); VECTORS;

DECISION TREES; DISCRETIZED INTERPRETABLE MULTILAYER PERCEPTRON; NEURAL NETWORK ENSEMBLES; RULE EXTRACTION; TRAINING ALGORITHMS;

MULTILAYER NEURAL NETWORKS;

EID: 10944228774     PISSN: 15708683     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jal.2004.03.004     Document Type: Article

References (45)

1. Andrews R. Geva S. Rule extraction from a constrained error back-propagation multi-layer perceptron Proc. of the Australian Conference on Neural Networks 1994 9-12
2. Andrews R. Diederich J. Tickle A.B. Survey and critique of techniques for extracting rules from trained artificial neural networks Knowledge Based Systems 8 6 1995 373-389
3. Andrews R. Geva S. Rule extraction from local cluster neural nets Neurocomput. 47 2002 1-20
4. Bairoch A. Apweiler R. The SWISS-PROT protein sequence database and its supplement TrEMBL in 2000 Nucl. Acids Res. 28 2000 45-48
5. Bauer E. Kohavi R. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants Machine Learning 36 1999 105-139
6. Baxt W. Improving the accuracy of an artificial neural network using multiple differently trained networks Neural Comput. 4 1992 772-780
7. Blake C.L. Merz C.J. UCI repository of machine learning databases http://www.ics.uci.edu/mlearn/MLRepository.html, University of California, Department of Information and Computer Science, Irvine CA
8. Bologna G. Pellegrini C. Rule extraction from modular transparent boxes Proc. of the Conference on Neural Networks and their Applications (NEURAP), Marseille, France 1998 393-398
9. Bologna G. Pellegrini C. Rule extraction from the oblique multi-layer perceptron Proc. of the Australian Conference on Neural Networks 1998 260-264
10. Bologna G. Symbolic rule extraction from the DIMLP neural network Wermter S. Sun R. Neural Hybrid Systems 2000 240-254 Springer Berlin
11. Bologna G. A study on rule extraction from several combined neural networks Internat. J. Neural Syst. 11 3 2001 247-255
12. Bologna G. A model for single and multiple knowledge based networks Artificial Intelligence in Medicine 28 2003 141-163
13. Breiman L. Bagging predictors Machine Learning 26 1996 123-140
14. Condorcet J.A. Sur les elections par scrutiny Histoire de l'Academie Royale des Sciences 1784 31-34
15. Craven M. Shavlik J. Extracting tree-structured representations of trained networks Proc. Neural Inform. Process. Syst. 8 1996 24-30
16. M. Craven, Extracting comprehensible models from trained neural networks, PhD Thesis, University of Wisconsin, 1996
17. Domingos P. Discovery via multiple models Intelligent Data Anal. 2 3 1998 187-202
18. Duch W. Adamczak R. Grabczewski K. A new methodology of extraction, optimization and application of crisp and fuzzy logical rules Trans. Neural Networks 12 2 2001 277-306
19. Fu L. Rule learning by searching on adapted nets Proc. of the 9th National Conference on Artificial Intelligence, Anaheim, CA 1991 590-595
20. Gallant S.I. Connectionist expert systems Comm. ACM 31 1998 152-169
21. Garcez A.S.D. Broda K. Gabbay D.M. Symbolic knowledge extraction from trained neural networks: A sound approach Artificial Intelligence 125 2001 155-207
22. Hertz J.A. Krogh A. Palmer R.G. Introduction to The Theory of Neural Computation 1991 Addison-Wesley Reading, MA
23. Jang J.S.R. ANFIS. Adaptive-network-based fuzzy inference system Trans. System Man Cybernet. 23 1993 665-685
24. Kasabov N. Woodford B.J. Rule insertion and rule extraction from evolving fuzzy networks: Algorithms and applications for building adaptive intelligent expert systems Proc. of the International Fuzzy Systems Conference 1999 1406-1411
25. Krishnan R. Sivakumar G. Bhattacharya P. A search technique for rule extraction from trained neural networks Pattern Recogn. Lett. 20 1999 273-280
26. Maurer-Stroh S. Eisenhaber B. Eisenhaber F. N-terminal N-myristoylation of proteins: Refinement of the sequence motif and its taxon-specific differences J. Mol. Biol. 317 2002 523-540
27. Muggleton S. King R.D. Sternberg M.J.J. Predicting protein secondary structure using logic-based machine Protein Engineering 5 1992 647-657
28. Nauck D. Kruse R. Obtaining interpretable fuzzy classification rules from medical data Artificial Intelligence in Medicine 16 1999 149-169
29. Opitz D. Maclin R. Popular ensemble methods: An empirical study Artificial Intelligence Res. 11 1999 169-198
30. Quinlan J.R. C4.5: Programs for Machine Learning 1993 Morgan Kaufmann
31. Rosenblatt F. Principles of Neurodynamics 1962 Spartan New York
32. Saito K. Nakano R. Rule extraction from facts and neural networks Proc. of the International Neural Network Conference, San Diego, CA 1990 379-382
33. Setiono R. Huan L. Neurolinear: From neural networks to oblique decision rules Neurocomput. 17 1997 1-24
34. Setiono R. Generating concise and accurate classification rules for breast cancer diagnosis Artificial Intelligence in Medicine 18 2000 205-219
35. Sigrist C.J. Cerutti L. Hulo N. Gattiker A. Falquet L. Pagni M. Bairoch A. Bucher P. PROSITE: A documented database using patterns and profiles as motif descriptors Brief Bioinform. 3 2002 265-274
36. Taha I.A. Ghosh J. Symbolic interpretation of artificial neural networks Trans. Neural Networks 11 3 1999 448-463
37. S. Thrun, et al., The MONK's problems-a performance comparison of different learning algorithms, Technical Report CS-CMU-91-197, Carnegie Mellon University, 1991
38. Thrun S. Extracting rules from artificial neural networks with distributed representations Tesauro G. Touretzky D.S. Leen T.K. Neural Information Processing Systems 7 1995 505-512 MIT Press Cambridge, MA
39. G. Towell, Symbolic knowledge and neural networks: Insertion, refinement and extraction, PhD Thesis, University of Wisconsin, 1991
40. Towell G. Shavlik J. The extraction of refined rules from knowledge based neural networks Machine Learning 131 1993 71-101
41. Wall R. Cunningham P. Walsh P. Byrne S. Explaining the output of ensembles in medical decision support on a case by case basis Artificial Intelligence in Medicine 28 2003 141-163
42. P. Werbos, Beyond regression: New tools for prediction and analysis in the behavioral sciences, PhD Thesis, Harvard University, 1974
43. Wolpert D.H. Macready W.G. The no free lunch theorems IEEE Trans. Evolutionary Comput. 1 1 1997 67-82
44. Zadeh L.A. Fuzzy sets Inform. Control 8 1965 338-353
45. Zhou Z.-H. Jiang Y. Chen S.-F. Extracting symbolic rules from trained neural network ensembles AI Commun. 16 1 2003 3-15