The truth will come to light: Directions and challenges in extracting the knowledge embedded within trained artificial neural networks

IEEE Transactions on Neural Networks

Volumn 9, Issue 6, 1998, Pages 1057-1068

  Tickle, Alan B ^a   Andrews, Robert ^a   Golea, Mostefa ^b   Diederich, Joachim ^a  

^a QUEENSLAND UNIVERSITY OF TECHNOLOGY   (Australia)

^b AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

Author keywords

Feedforward neural networks; Finite state automata; Fuzzy neural networks; Knowledge insertion; Recurrent neural networks; Rule extraction; Rule refinement

Indexed keywords

FUZZY NEURAL NETWORKS; KNOWLEDGE INSERTION; RULE EXTRACTION; RULE REFINEMENT;

ALGORITHMS; COMPUTATIONAL COMPLEXITY; FINITE AUTOMATA; KNOWLEDGE ACQUISITION; KNOWLEDGE BASED SYSTEMS; RECURRENT NEURAL NETWORKS;

FEEDFORWARD NEURAL NETWORKS;

EID: 0032208720     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.728352     Document Type: Article

References (44)

1. R. Andrews, J. Diederich, and A. B. Tickle, "A survey and critique of techniques for extracting rules from trained artificial neural networks," Knowledge-Based Syst., vol. 8, no. 6, pp. 373-389, 1995.
2. R. Andrews, R. Cable, J. Diederich, S. Geva, M. Golea, R. Hayward, C. Ho-Stuart, and A. B. Tickle, "An evaluation and comparison of techniques for extracting and refining rules from artificial neural networks," QUT NRC, Feb. 1996.
3. G. A. Carpenter and A. W. Tan, "Rule extraction: From neural architecture to symbolic representation," Connection Sci., vol. 7, no. 1, pp. 3-27, 1995.
4. A. Cleeremans, D. Servan-Schreiber, and J. L. McClelland, "Finite-state automata and simple recurrent networks," Neural Comput., vol. 1, no. 3, pp. 372-381, 1989.
5. M. Craven, "Extracting comprehensible models from trained neural networks," Ph.D. dissertation, Univ. Wisconsin, Madison, WI, 1996.
6. M. W. Craven and J. W. Shavlik, "Learning to predict reading frames in E. coli sequences," in Proc. 26th Hawaii Int. Conf. Syst. Sci., Wailea, HI, 1993, pp. 773-782.
7. _, "Using sampling and queries to extract rules from trained neural networks, machine learning," in Proc. 11th Int. Conf., Amherst, MA, 1994, pp. 73-80.
8. R. Durbin and D. Rumelhart, "Product units: A computationally powerful and biologically plausible extension," Neural Comput., vol. 1, no. 1, pp. 133-166, 1989.
9. J. L. Elman, "Finding structure in time," Cognitive Sci., vol. 14, pp. 179-211, 1990.
10. R. Filer, I. Sethi, and J. Austin, "A comparison between two rule extraction methods for continuous input data," in Proc. NIPS'97 Rule Extraction From Trained Artificial Neural Networks Wkshp., Queensland Univ. Technol., 1996, pp. 38-45.
11. L. M. Fu, "Rule learning by searching on adapted nets," in Proc. 9th Nat. Conf. Artificial Intell., Anaheim, CA, 1991, pp. 590-595.
12. _, "Rule generation from neural networks," IEEE Trans. Syst., Man, Cybern., vol. 28, pp. 1114-1124, 1994.
13. C. L. Giles, C. B. Miller, D. Chen, H. Chen, G. Z. Sun, and Y. C. Lee, "Learning and extracting finite-state automata with second-order recurrent neural networks," Neural Comput., vol. 4, pp. 393-405, 1992.
14. C. L. Giles and C. W. Omlin, "Rule refinement with recurrent neural networks," in Proc. IEEE Int. Conf. Neural Networks, San Francisco, CA, Mar. 1993, pp. 801-806.
15. _, "Extraction, insertion, and refinement of symbolic rules in dynamically driven recurrent networks," Connection Sci., vol. 5, nos. 3/4, pp. 307-328, 1993.
16. _, "Rule revision with recurrent networks," IEEE Trans. Knowledge and Data Eng., vol. 8, p. 183, 1996.
17. M. Golea, "On the complexity of rule extraction from neural networks and network querying," in Proc. Rule Extraction From Trained Artificial Neural Networks Wkshp., Society for the Study of Artificial Intelligence and Simulation of Behavior Workshop Series (AISB'96), Univ. Sussex, Brighton, U.K., Apr. 1996, pp. 51-59.
18. _, "On the complexity of extracting simple rules from trained neural nets," 1997, to appear.
19. J. Hopcroft and J. Ullman, Introduction to Automata Theory, Languages, and Computation. Reading, MA: Addison-Wesley, 1979.
20. M. A. Hume, The Happy Way to Reading. London, U.K.: Blackie & Son, 1950.
21. I. Ivanova and M. Kubat, "Initialization of neural networks by means of decision trees," Knowledge-Based Syst., vol. 8, no. 6, pp. 333-344, 1995.
22. R. Krishnan, "A systematic method for decompositional rule extraction from neural networks," in Proc. NIPS'97 Rule Extraction From Trained Artificial Neural Networks Wkshp., Queensland Univ. Technol., 1996, pp. 38-45.
23. H. Liu and S. T. Tan, "X2R: A fast rule generator," in Proc. IEEE Conf. Syst., Man, Cybern. New York: IEEE Press, 1995, pp. 1631-1635.
24. F. Maire, "A partial order for the M-of-N rule extraction algorithm," IEEE Trans. Neural Networks, vol. 8, pp. 1542-1544, Nov. 1997.
25. P. M. Murphy and M. J. Pazzani, "ID2-of-3: Constructive induction of M-of-N concepts for discriminators in decision trees," in Proc. 8th Int. Machine Learning Wkshp., Evanston, IL, 1991, pp. 183-187.
26. C. W. Omlin and C. L. Giles, "Extraction of rules from discrete-time recurrent neural networks," Neural Networks, vol. 9, no. 1, pp. 41-52, 1996.
27. _, "Constructing deterministic finite-state automata in recurrent neural networks," J. ACM, vol. 43, no. 6, pp. 937-972, 1996.
28. C. W. Omlin, K. Thornber, and C. L. Giles, "Fuzzy finite-state automata can be deterministically encoded into recurrent neural networks," IEEE Trans. Fuzzy Syst., 1998, to be published.
29. D. W. Optiz and J. W. Shavlik, "Dynamically adding symbolically meaningful nodes to knowledge-based neural networks," Knowledge-Based Syst., vol. 8, no. 6, pp. 301-311, 1995.
30. R. Quinlan, C4.5: Programs for Machine Learning. San Mateo, CA: Morgan-Kaufmann, 1993.
31. J. Rissanen, Stochastic Complexity in Statistical Inquiry. Singapore: World, 1989.
32. K. Saito and R. Nakano, "Law discovery using neural networks," in Proc. NIPS'96 Rule Extraction From Trained Artificial Neural Networks Wkshp., Queensland Univ. Technol., 1996, pp. 62-69.
33. I. Schellhammer, J. Diederich, M. Towsey, and C. Brugman, "Knowledge extraction and recurrent neural networks: An analysis of an Elman network trained on a natural language learning task," QUT-NRC Tech. Rep. 97-IS1, 1997.
34. R. Setiono, "Extracting rules from neural networks by pruning and hidden unit splitting," Neural Comput., vol. 9, pp. 205-225, 1997.
35. A. W. Tan, "Cascade ARTMAP: Integrating neural computation and symbolic knowledge processing," in IEEE Trans. Neural Networks, vol. 8, pp. 23-250, 1997.
36. S. B. Thrun, J. Bala, E. Bloedorn, I. Bratko, B. Cestnik, J. Cheng, K. De Jong, S. Dzeroski, S. E. Fahlman, D. Fisher, R. Hamann, K. Kaufman, S. Keller, I. Kononenko, J. Kreuziger, R. S. Michalski, T. Mitchell, P. Pachowicz, Y. Reich, H. Vafaie, K. Van de Welde, W. Wenzel, J. Wnek, and J. Zhang, "The MONK's problems: A performance comparison of different learning algorithms," Carnegie Mellon Univ., Pittsburgh, PA, Tech. Rep. CMU-CS-91-197, Dec. 1991.
37. S. B. Thrun, "Extracting provably correct rules from artificial neural networks," Inst. Informatik III, Univ. Bonn, Germany, Tech. Rep. IAI-TR-93-5, 1994.
38. A. B. Tickle, R. Hayward, and J. Diederich, "Recent developments in techniques for extracting rules from trained artificial neural networks," Herbstschule Konnektionismus (HeKonn '96), Münster, Germany, Oct. 1996.
39. A. B. Tickle, R. Andrews, M. Golea, and J. Diederich, "Rule extraction from trained artificial neural networks," Neural Network Analysis, Architectures, and Applications, A. Browne, Ed. Bristol, U.K.: Inst. Phys. Publishing, 1997, pp. 61-99.
40. A. B. Tickle, M. Golea, R. Hayward, and J. Diederich, "The truth is in there: Current issues in extracting rules from trained feedforward artificial neural networks," in Proc. 1997 IEEE Int. Conf. Neural Networks (ICNN'97), vol. 4, Houston, TX, June 1997, pp. 2530-2534.
41. A. B. Tickle, "Machine learning, neural networks and information security: Techniques for extracting rules from trained feedforward artificial neural networks and their application in an information security problem domain," Ph.D. dissertation, 1997.
42. G. Towell and J. Shavlik, "The extraction of refined rules from knowledge-based neural networks," Machine Learning, vol. 131, pp. 71-101, 1993.
43. L. G. Valiant, "A theory of the learnable," Commun. ACM, vol. 27, no. 11, pp. 1134-1142, 1984.
44. R. J. Williams and D. Zipser, "A learning algorithm for continually running fully recurrent neural networks," Neural Comput., vol. 1, pp. 270-280, 1989.