Hidden neuron pruning of multilayer perceptrons using a quantified sensitivity measure

Neurocomputing

Volumn 69, Issue 7-9 SPEC. ISS., 2006, Pages 825-837

  Zeng, Xiaoqin ^a   Yeung, Daniel S ^b  

^a HOHAI UNIVERSITY   (China)

^b HONG KONG POLYTECHNIC UNIVERSITY   (Hong Kong)

Author keywords

Multilayer perceptron; Neural network; Neuron pruning; Relevance measure; Sensitivity measure

Indexed keywords

COMPUTATIONAL COMPLEXITY; ITERATIVE METHODS; NEUROPHYSIOLOGY; SENSITIVITY ANALYSIS;

NEURON PRUNING; RELEVANCE MEASURE; SENSITIVITY MEASURE;

MULTILAYER NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; BRAIN COMPUTER INTERFACE; COMPUTER ANALYSIS; COMPUTER PROGRAM; MATHEMATICAL COMPUTING; NERVE CELL NETWORK; PRIORITY JOURNAL; QUANTITATIVE ANALYSIS; SENSITIVITY ANALYSIS;

EID: 32544452874     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2005.04.010     Document Type: Article

References (23)

1. P. Burrascano, A pruning technique maximizing generalization, in: Proceedings of the International Joint Conference on Neural Networks, vol. 1, 1993, pp. 347-350.
2. G. Castellano A. Fanelli M. Pelillo An iterative pruning algorithm for feedforward neural networks IEEE Trans. Neural Networks 8 1997 519-531
3. J.Y. Choi C.H. Choi Sensitivity analysis of multilayer perceptron with differentiable activation functions IEEE Trans. Neural Networks 3 1 1992 101-107
4. Y.L. Cun, J.S. Denker, S.A. Solla, Optimal brain damage, in: D.S. Touretzky (Ed.), Proceedings of the Neural Information Processing Systems, vol. 2, 1990, pp. 598-605.
5. A.P. Engelbrecht A new pruning heuristic based on variance analysis of sensitivity information IEEE Trans. Neural Networks 12 6 2001 1386-1398
6. A.P. Engelbrecht, I. Cloete, A sensitivity analysis algorithm for pruning feedforward neural networks, in: Proceedings of IEEE International Conference on Neural Networks, vol. 2, 1996, pp. 1274-1277.
7. J. Gorodkin L.K. Hansen A. Krogh C. Svarer O. Winther A quantitative study of pruning by optimal brain damage Int. J. Neural Systems 4 2 1993 159-169
8. M. Hagiwara A simple and effective method for removal of hidden units and weights Neurocomputing 6 1994 207-218
9. B. Hassibi, D.G. Stork, Second-order derivatives for network pruning: Optimal brain surgeon, in: C.L. Giles, S.J. Hanson, J.D. Cowan (Eds.), Proceedings of the Neural Information Processing Systems, vol. 5, 1992, pp. 164-171.
10. B. Hassibi, D.G. Stork, G. Wolff, Optimal brain surgeon and general network pruning, in: Proceedings of IEEE International Conference on Neural Networks, 1993, pp. 293-299.
11. K.M. Hornik M. Stinchcombea H. White Multilayer feedforward networks are universal approximators Neural Networks 2 1989 359-366
12. E.D. Karnin A simple procedure for pruning back-propagation trained neural networks IEEE Trans. Neural Networks 1 2 1990 239-242
13. T.Y. Kwok D.Y. Yeung Constructive algorithms for structure learning in feedforward neural networks for regression problems IEEE Trans. Neural Networks 8 3 1997 630-645
14. M.C. Mozer, P. Smolensky, Skeletonization: A technique for trimming the fat from a network via relevance assessment, in: D.S. Touretzky (Ed.), Proceedings of the Neural Information Processing Systems, vol. 1, 1989, pp. 107-115.
15. R.H. Nielsen, Theory of the back propagation neural network, in: Proceedings of International Joint Conference on Neural Networks, vol. 1, 1989, 593-603.
16. M.W. Pedersen, L.K. Hansen, J. Larsen, Pruning with generalization based weight saliencies: γ OBD, γ OBS, in: D.S. Touretzky, M.C. Mozer, M.E. Hasselmo (Eds.), Proceedings of the Neural Information Processing Systems, vol. 8, 1996, pp. 521-528.
17. R. Reed Pruning algorithms - a survey IEEE Trans. Neural Networks 4 5 1993 740-747
18. J. Sietsma, R.J.F. Dow, Neural net pruning: Why and how, in: Proceedings of IEEE International Conference on Neural Networks, vol. 1, 1988, pp. 325-333.
19. J. Sietsma R.J.F. Dow Creating artificial neural networks that generalize Neural Networks 4 1991 67-79
20. K. Suzuki I. Horiba N. Sugie A simple neural network pruning algorithm with application to filter synthesis Neural Process. Lett. 13 1 2001 43-53
21. M. Wynne-Jones, Node splitting: A constructive algorithm for feedforward neural networks, in: J. Moody, S. Hanson, R. Lippmann (Eds.), Proceedings of the Neural Information Processing Systems, vol. 4, 1992, pp. 1072-1079.
22. X. Zeng D.S. Yeung A quantified sensitivity measure for multilayer perceptron to input perturbation Neural Comput. 15 1 2003 183-212
23. J.M. Zurada A. Malinowski S. Usui Perturbation method for deleting redundant inputs of perceptron networks Neurocomputing 14 1997 177-193