Guiding hidden layer representations for improved rule extraction from neural networks

IEEE Transactions on Neural Networks

Volumn 22, Issue 2, 2011, Pages 264-275

  Huynh, Thuan Q ^a   Reggia, James A ^a  

^a Institute of Advanced Computer Science   (United States)

Author keywords

Hidden layer representation; neural networks; penalty function; rule extraction

Indexed keywords

CLASSIFICATION ACCURACY; COMPUTATIONAL EXPERIMENT; DATA SETS; ENCODINGS; ERROR BACK-PROPAGATION; HIDDEN LAYERS; HUMAN-READABLE; INPUT PATTERNS; LEARNED PATTERNS; LEARNING METHODS; PENALTY FUNCTION; RULE COMPLEXITY; RULE EXTRACTION; RULE EXTRACTION FROM NEURAL NETWORKS; WEIGHT MATRICES;

BACKPROPAGATION; CLASSIFICATION (OF INFORMATION);

NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; CLASSIFICATION; COMPUTER PROGRAM; COMPUTER SIMULATION; HUMAN; METHODOLOGY; STANDARD;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; CLASSIFICATION; COMPUTER SIMULATION; HUMANS; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; SOFTWARE DESIGN; SOFTWARE VALIDATION;

EID: 79951671371     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2010.2094205     Document Type: Article

References (29)

1. R. Andrews, A. Tickle, and J. Diederich, "A review of techniques for extracting rules from trained artificial neural networks," in Clinical Applications of Artificial Neural Networks. Cambridge, U.K.: Cambridge Univ. Press, 2001, pp. 256-297.
2. S. Bader, S. Holldobler, and V. Mayer-Eichberger, "Extracting propositional rules from feed-forward neural networks a new decompositional approach," in Proc. 3rd Int. Workshop Neural-Symbolic Learn. Reason., 2007, pp. 1-6.
3. W. Duch, R. Adamczak, and K. Grabczewski, "A new methodology of extraction, optimization and application of crisp and fuzzy logical rules," IEEE Trans. Neural Netw., vol. 12, no. 2, pp. 277-306, Mar. 2001. (Pubitemid 32371484)
4. H. Jacobsson, "Rule extraction from recurrent neural networks: A taxonomy and review," Neural Comput., vol. 17, no. 6, pp. 1223-1263, Jun. 2005. (Pubitemid 40653138)
5. R. Nayak, "Generating rules with predicates, terms and variables from the pruned neural networks," Neural Netw., vol. 22, no. 4, pp. 405-414, May 2009.
6. T. Etchells and P. Lisboa, "Orthogonal search-based rule extraction (OSRE) for trained neural networks: A practical and efficient approach," IEEE Trans. Neural Netw., vol. 17, no. 2, pp. 374-384, Mar. 2006. (Pubitemid 43380061)
7. R. Setiono, B. Baesens, and C. Mues, "Recursive neural network rule extraction for data with mixed attributes," IEEE Trans. Neural Netw., vol. 19, no. 2, pp. 299-307, Feb. 2008. (Pubitemid 351359294)
8. J. Huysmans, R. Setiono, B. Baesens, and J. Vanthienen, "Minerva: Sequential covering for rule extraction," IEEE Trans. Syst., Man, Cybern., Part B: Cybern., vol. 38, no. 2, pp. 299-309, Apr. 2008. (Pubitemid 351479553)
9. H. Tsukimoto, "Extracting rules from trained neural networks," IEEE Trans. Neural Netw., vol. 11, no. 2, pp. 377-389, Mar. 2000.
10. R. Setiono, W. Leow, and J. Zurada, "Extraction of rules from artificial neural networks for nonlinear regression," IEEE Trans. Neural Netw., vol. 13, no. 3, pp. 564-577, May 2002. (Pubitemid 34669648)
11. S. Haykin, Neural Networks: A Comprehensive Foundation. Upper Saddle River, NJ: Prentice-Hall, 1994.
12. J. Quinlan, C4.5: Programs for Machine Learning. San Mateo, CA: Morgan Kaufmann, 1993.
13. H. Lu, R. Setiono, and H. Liu, "Neurorule: A connectionist approach to data mining," in Proc. 21st Int. Conf. Very Large Data Bases, 1995, pp. 478-489.
14. R. Setiono and H. Liu, "Neurolinear: From neural networks to oblique decision rules," Neurocomputing, vol. 17, no. 1, pp. 1-24, Sep. 1997. (Pubitemid 27433442)
15. G. G. Towell and J. W. Shavlik, "Extracting refined rules from knowledge-based neural networks," Mach. Learn., vol. 13, no. 1, pp. 71-101, Oct. 1993.
16. T. Huynh and R. Setiono, "Effective neural network pruning using crossvalidation," in Proc. Int. Joint Conf. Neural Netw., vol. 2. Jul.-Aug. 2005, pp. 972-977. (Pubitemid 44055583)
17. M. Riedmiller and H. Braun, "RPROP - A fast adaptive learning algorithm," in Proc. Int. Symp. Comput. Inform. Sci., Antalya, Turkey, 1992, pp. 279-286.
18. A. Krogh and J. Hertz, "A simple weight decay can improve generalization," in Advances in Neural Information Processing Systems. San Mateo, CA: Morgan Kaufmann, 1992, pp. 950-957.
19. H. Liu and R. Setiono, "Chi2: Feature selection and discretization of numeric attributes," in Proc. 7th IEEE Int. Conf. Tools Artif. Intell., Herndon, VA, Nov. 1995, pp. 388-391.
20. A. Asuncion and D. Newman. (2007). UCI Machine Learning Repository [Online]. Available: http://www.ics.uci.edu/?excl;?mlearn/ MLRepository.html
21. R. Duda, P. Hart, and D. Stock, Pattern Classification and Scene Analysis. New York: Wiley, 2001.
22. P. Horton and K. Nakai, "A probabilistic classification system for predicting the cellular localization sites of proteins," in Proc. Int. Conf. Intell. Syst. Mol. Biol., vol. 4. 1996, pp. 109-115.
23. J. Foster, E. Barkus, and C. Yavorsky, Understanding and Using Advanced Statistics. Newbury Park, CA: Sage, 2006.
24. S. Cho and J. Reggia, "Learning competition and cooperation," Neural Comput., vol. 5, no. 2, pp. 242-259, Mar. 1993.
25. Z. Xu, R. Zhang, and W. Jing, "When does online BP training converge?" IEEE Trans. Neural Netw., vol. 20, no. 10, pp. 1529-1539, Oct. 2009.
26. O. Ludwig and U. Nunes, "Novel maximum-margin training algorithms for supervised neural networks," IEEE Trans. Neural Netw., vol. 21, no. 6, pp. 972-984, Jun. 2010.
27. M. Hagan and M. Menhaj, "Training feedforward networks with the Marquardt algorithm," IEEE Trans. Neural Netw., vol. 5, no. 6, pp. 989-993, Nov. 1994.
28. B. Wilamowski and H. Yu, "Improved computation for Levenberg- Marquardt training," IEEE Trans. Neural Netw., vol. 21, no. 6, pp. 930-937, Jun. 2010.
29. T. Huynh and J. Reggia, "Improving rule extraction from neural networks by modifying hidden layer representations," in Proc. Int. Joint Conf. Neural Netw., Atlanta, GA, Jun. 2009, pp. 1316-1321.