A dynamic node decaying method for pruning artificial neural networks

IEICE Transactions on Information and Systems

Volumn E86-D, Issue 4, 2003, Pages 736-751

  Shahjahan, Md ^a   Murase, Kazuyuki ^a  

^a UNIVERSITY OF FUKUI   (Japan)

Author keywords

ANNs; Classification; Competition; Decaying; Pruning

Indexed keywords

BACKPROPAGATION; COMPUTATIONAL COMPLEXITY; ERROR ANALYSIS; LEARNING ALGORITHMS; PROBLEM SOLVING; SENSITIVITY ANALYSIS;

CLASSIFICATION ERROR; DYNAMIC NODE DECAYING METHOD; NETWORK PRUNING;

NEURAL NETWORKS;

EID: 0142186186     PISSN: 09168532     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (27)

1. T.-Y. Kwok and D.-Y. Yeung, "Constructive algorithms for structure learning in feedforward neural networks for regression problems," IEEE Trans. Neural Netw., vol.8, no.3, pp.630-645, 1997.
2. R. Reed, "Pruning algorithms - A survey," IEEE Trans. Neural Netw., vol.4, no.5, pp.740-747, 1993.
3. J.K. Kruschke, "Improving generalization in back-propagation networks with distributed bottleneck," Proc. Int. Joint Conf. on Neural Networks, vol.I, pp.443-447, Washington DC, 1989.
4. G. Castellano, A.M. Fanelli, and M. Pelillo, "An iterative pruning Algorithm for feedforward neural networks," IEEE Trans. Neural Netw., vol.8, no.3, pp.519-531, 1997.
5. M. Ishikawa, "Structural learning with forgetting," Neural Networks, vol.9, no.3, pp.509-521, 1996.
6. M.C. Mozer and P. Smolensky, "Skeletonization: A technique for trimming the fat from a network via relevance assessment," Advances in Neural Information Processing Systems, vol.1, pp.107-115, Morgan Kaufman, San Mateo, CA, 1989.
7. Y. Le Cun, J.S. Denker, and S.A. Solla, "Optimal brain damage," Advances in Neural Information Processing Systems, vol.2, pp.598-605, Morgan Kaufman, San Mateo, CA, 1990.
8. B. Hassibi and D.G. Stork, "Second order derivatives for network pruning: Optimal brain surgeon," Advances in Neural Information Processing Systems, vol.5, pp.162-171, Morgan Kaufman, San Mateo, CA, 1993.
9. J. Seitsta and R.J. F Dow, "Creating artificial neural networks that generalize," Neural Networks, vol.4, pp.67-79, 1991.
10. D. Opitz, "The effective size of neural networks: A principle component approach," 14th Int. Conference on Machine Learning, pp. 263-271, Nashville, TN, 1997.
11. A. Krogh and J.A. Hertz, "A simple weight decay can improve generalization," Advances in Neural Information Processing Systems 4, pp.950-957, Morgan Kaufman, 1995.
12. K. Murase, Y. Matsunaga, and Y. Nakaade, "A back propagation algorithm which automatically determines the number of association units," Proc. IJCNN, pp. 783-788, Singapore, 1991.
13. Y. Matsunaga, K. Murase, O. Yamakawa, and M. Tanifuji, "A modified back propagation algorithm that automatically removes the redundant hidden units by competition," IEICE Trans. Inf. & Syst. (Japanese Edition), vol.J79-D-II, no.3, pp.403-412, March 1996.
14. Y. Matsunaga, Y. Nakada, O. Yamakawa, and K. Murase, "A back-propagation algorithm with automatic reduction of association units in multilayer neural network," IEICE Trans. Inf. & Syst. (Japanese Edition), vol.J74-D-II, no.8, pp.1118-1121, Aug. 1991.
15. M. Hagiwara, "Removal of hidden units and weights for Back propagation network," Proc. IJCNN, vol.1, pp.351-354, 1993.
16. L. Prechelt, "Proben1-A set of neural network benchmark problems and benchmarking rules," Tech. Report, Karlsruhe, Germany, Sept. 1994.
17. L. Prechelt, "Connection pruning with static and adaptive pruning schedules," Neurocomputing, vol.16, no.1, pp.49-61, July 1997.
18. D.E. Rumelhart, G.E. Hinton, and R.J Williams, "Learning internal representation by error propagation," in Parallel Distributed Processing, vol.1, ed. D.E. Rumelhart and J.L. McClelland, pp.318-362, MIT Press, Cambridge, MA, 1986.
19. Md. Monirul Islam and K. Murase, "A new algorithm to design compact two hidden layer artificial neural networks," Neural Networks, vol.14, pp. 1265-1278, 2001.
20. M.F. Augusteijn and K.A. Shaw, "Radical pruning: A method to construct skeleton radial basis function networks." Int. Journal of Neural Systems., vol.2, pp.143-154, 2000.
21. X. Yao and Y. Liu, "A new evolutionary system for evolving artificial neural networks," IEEE Trans. Neural Netw., vol.8, no.3, pp.694-713, May 1997.
22. D.W. Opitz and R. Maclin, "Popular ensemble methods: An empirical study," J. Artificial Intelligence Research, vol.11, pp. 169-198, 1999.
23. C. Giles and C.W. Omlin, "Pruning recurrent neural networks for improving generalization performance," IEEE Trans. Neural Netw., vol.5, no.5, pp.848-851, 1994.
24. A.P. Engelbrecht, "A new pruning heuristic based on variance analysis of sensitivity information," IEEE Trans. Neural Netw., vol.12, no.6, pp.1386-1399, Nov. 2001.
25. P.L. Bartlett, "The sample complexity of pattern recognition with neural networks: The size of the weights is more important than the size of the network," IEEE Trans. Inf. Theory, vol.44, no.2, pp.525-536, March 1998.
26. G.E. Hinton and D. van Camp, "Keeping neural network simple by minimizing the description length of the weights," Sixth ACM Conference on Computational Learning Theory, Santa Cruz, July 1993.
27. Md. Monirul Islam and K. Murase, "A new crossover operator and its application to artificial neural networks evolution," IEICE Trans. Inf. & Syst., vol.E84-D, no.9, pp.1144-1154, Sept. 2001.