Reverse engineering the neural networks for rule extraction in classification problems

Neural Processing Letters

Volumn 35, Issue 2, 2012, Pages 131-150

  Gethsiyal Augasta, M ^a   Kathirvalavakumar, T ^b  

^a Sarah Tucker College   (India)

^b VHNSN COLLEGE   (India)

Author keywords

Classification; Neural networks; Pedagogical; Pruning; Reverse engineering; Rule extraction

Indexed keywords

BLACK BOXES; CLASSIFICATION ACCURACY; CONTINUOUS MODE; DATA SETS; ENGINEERING TECHNIQUES; INPUT NEURONS; MIXED MODE; PEDAGOGICAL; PEDAGOGICAL APPROACH; PRUNING; REAL DATA SETS; RULE EXTRACTION; RULE EXTRACTION ALGORITHMS; RULE EXTRACTION METHOD; SET OF RULES; TRAINED NEURAL NETWORKS;

ALGORITHMS; IONOSPHERE; NEURAL NETWORKS; REVERSE ENGINEERING;

CLASSIFICATION (OF INFORMATION);

EID: 84861794233     PISSN: 13704621     EISSN: 1573773X     Source Type: Journal    
DOI: 10.1007/s11063-011-9207-8     Document Type: Article

References (43)

1. Han J, Kambler M (2001) Data Mining: concepts and techniques. Morgan Kaufmann, San Francisco
2. Cohen S, Rokach L, Maimon O (2007) Decision-tree instance-space decomposition with grouped gainratio. Inf Sci 177(17):3592-3612 (Pubitemid 46920524)
3. Yager RR (2006) An extension of the nave Bayesian classifier. Inf Sci 176(5):577-588
4. Kaikhah K, Doddmeti S (2006) Discovering trends in large datasets using neural network. Appl Intel 29:51-60 (Pubitemid 43130241)
5. Dam H, Abbass HA, Lokan C, Xin Yao (2008) Neural based learning classifier systems. IEEE Trans Knowl Data Eng 20(1):26-39
6. Mantas CJ, Puche JM,Mantas JM (2006) Extraction of similarity based fuzzy rules from artificial neural networks. Int J Approx Reason 43:202-221 (Pubitemid 44345154)
7. Setiono R, Liu H (1995) Understanding neural networks via rule extraction. In: Proceedings of 14th international joint conference on artificial intelligence, pp 480-485
8. Tsukimoto H (2000) Extracting rules from trained neural networks. IEEE Trans Neural Netw 11(2):377-389
9. Jacobson H (2005) Rule extraction from recurrent neural networks: a taxonomy and review. Neural Comput 17(6):1223-1263
10. Nayak R (2009) Generating rules with predicates, terms and variables from the pruned neural networks. Neural Netw 22:405-414
11. Taylor BJ, Darrah MA (2005) Rule extraction as a formal method for the verification and validation of neural networks. IEEE Int Joint Conf Neural Netw 5:2915-2920 (Pubitemid 44591664)
12. Tickle AB, Andrews R, Golea M, Diederich J (1998) The truth will come to light: directions and challenges in extracting the knowledge embeddedwithin trained artificial neural networks. IEEE TransNeural Netw 6:1057-1068 (Pubitemid 128742391)
13. Chauvin Y (1990) Generalization performance of overtrained backpropagation networks. In: Hlomeida LB, Wellekens CJ (eds) Proceedings of neural networks euroship workshop, pp 46-55
14. Reed R (1993) Pruning algorithm-a survey. IEEE Trans Neural Netw 4:740-747
15. Zurada JM (2002) Introduction to artificial neural systems. Jaisco Publishing House, Mumbai
16. Setiono R, Baesens B, Mues C (2008) A note on knowledge discovery using neural networks and its application to credit card screening. Eur J Oper Res 192(1):326-332
17. Attik M,Bougrain L, Alexandra F (2005) Neural network topology optimization. In: Proceedings of ICANN'05, vol 3697, pp 53-58
18. Hassibi B, Stork DG,Wolf GJ (1993) Optimal brain surgeon and general network pruning. In: Proceedings of IEEE ICNN 1, pp 293-299
19. Le Cun Y, Denker JS, Solla SA (1990) Optimal brain damage. In: Touretzky DS (ed) Advances neural information processing systems. Morgan Kaufmann, San Mateo, pp 598-605
20. Engelbrecht AP (2001) A new pruning heuristic based on variance analysis of sensitivity information. IEEE Trans Neural Netw 12(6):1386-1399
21. Xing H-J, Hu B-G (2009) Two phase construction of multilayer perceptrons using information theory. IEEE Trans Neural Netw 20(4):715-721
22. Castellano G, Fanelli AM, Pelillo M (1997) An iterative pruning algorithm for feedforward neural networks. IEEE Trans Neural Netw 8(3):519-530
23. Zeng X, Yeung DS (2006) Hidden neuron pruning of multilayer perceptrons using a quantified sensitivity measure. Neurocomputing 69:825-837 (Pubitemid 43230385)
24. Chikofsky EJ, Cross JH (1990) Reverse engineering and design recovery: a taxonomy. IEEE Softw 7(1):13-17
25. Nattapon P, Nittaya K, Kittisak K (2010) Database reverse engineering based on association rule mining. IJCSI Int J Comput Sci Issues 7(2):10-14
26. Setiono R, Liu H (1996) Symbolic representation of neural networks. IEEE Comput 29(3):71-77 (Pubitemid 126515072)
27. Setiono R (1997) Extracting rules from neural networks by pruning and hidden-unit splitting. Neural Comput 9(1):205-225 (Pubitemid 127622533)
28. Gupta A, Park S, Lam SW (1999) Generalized analytic rule extraction for feedforward neural networks. IEEE Trans Knowl Data Eng 11(6):985-991 (Pubitemid 30589454)
29. Etchells TA, Lisboa PJG (2006) Orthogonal search-based rule extraction (OSRE) for trained neural networks: a practical and efficient approach. IEEE Trans Knowl Data Eng 17(2):374-384 (Pubitemid 43380061)
30. Setiono R, Baesens B, Mues C (2008) Recursive neural network rule extraction for data with mixed attributes. IEEE Trans Knowl Data Eng 19(2):299-307 (Pubitemid 351359294)
31. Emad EW, Wunsch II DC (2007) Neural network explanation using inversion. Neural Netw 20:78-93 (Pubitemid 44960621)
32. Andrews R, Diederich J, Tickle AB (1996) Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowl Based Syst 8(6):373-389
33. Setiono R, Liu H (1997) Neurolinear: from neural networks to oblique decision rules. Neural Comput 17(1):1-24 (Pubitemid 27433442)
34. Kim D, Lee J (2000) Handling continuous-valued attributes in decision tree with neural network modeling. In: Proceedings of the 11th European conference on machine learning. Lecture notes in computer science, vol 1810, pp 211-219
35. Koichi O, Yoichi H, Gong T, Rudy S (2008) Greedy rule generation from discrete data and its use in neural network rule extraction. Neural Netw 21:1020-1028
36. Saito K, Nakano R (1988) Medical diagnostic expert system based on pdp model. In: Proceedings of IEEE international conference on neural networks, vol 1, pp 255-262
37. Sestito S, Dillon TS (1992) Automated knowledge acquisition of rules with continuously valued attributes. In: Proceedings of 12th international conference on expert systems and their applications, pp 645-656
38. Thrun SB (1993) Extracting provably correct rules from artificial neural networks. Technical Report IAI-TR-93-5, Institut für informatik III, University of Bonn
39. Pop E, Hayward R, Diederich J (1994) RULENEG: extracting rules from a trained ANN by stepwise negation. Tech Rep, QUT NRC
40. CravenMW, Shavlik JW (1994) Using sampling and queries to extract rules from trained neural networks. In: Proceedings of the 11th international conference on machine learning, San Francisco, CA
41. Filer R, Sethi I, Austin J (1997) A comparison between two rule extraction methods for continuous input data. In: Proceedings of neural information processing systems, rule extraction from trained artificial neural networks workshop, pp 38-45
42. Hruschka ER, Ebecken NFF (2006) Extracting rules from multilayer perceptrons in classification problems: a clustering-based approach. Neurocomputing 70:384-397 (Pubitemid 44739711)
43. Kahramanli H, Allahverdi N (2009) Rule extraction from trained adaptive neural networks using artificial immune systems. Expert Syst Appl 36:1513-1522