Hidden neuron pruning for multilayer perceptrons using sensitivity measure

Proceedings of 2002 International Conference on Machine Learning and Cybernetics

Volumn 4, Issue , 2002, Pages 1751-1757

  Yeung, Daniel S ^a   Zeng, Xiao Qin ^a  

^a HONG KONG POLYTECHNIC UNIVERSITY   (Hong Kong)

Author keywords

Multilayer perceptron; Neural networks; Pruning; Sensitivity

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; COMPUTER SIMULATION; INTEGRAL EQUATIONS; PROBABILITY DENSITY FUNCTION; SET THEORY;

HIDDEN NEURON PRUNING; NEURON; SENSITIVITY MEASURE;

MULTILAYER NEURAL NETWORKS;

EID: 0036930799     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (14)

1. T. Y. Kwok and D. Y. Yeung, "Constructive Algorithms for Structure Learning in Feedforward Neural Networks for Regression Problems," IEEE Trans. Neural Networks, vol. 8, no. 3, pp. 630-645, 1997.
2. R. Reed, "Pruning algorithms - A survey," IEEE Trans. Neural Networks, vol. 4, no. 5, pp. 740-747, 1993.
3. A. P. Engelbrecht, "A New Pruning Heuristic Based on Variance Analysis of Sensitivity Information," IEEE Trans. Neural Networks, vol. 12, no. 6, pp. 1386-1398, 2001.
4. M. Hagiwara, "A Simple and effective method for removal of hidden units and weights," Neurocomputing, vol. 6, pp207-218, 1994.
5. M. C. Mozer and P. Smolensky, "Skeletonization: A technique for trimming the fat from a network via relevance assessment," in Advances in Neural Information Processing Systems, D. S. Touretzky, Ed., Denver, 1989, vol. 1, pp. 107-115.
6. E. D. Karnin, "A simple procedure for pruning back-propagation trained neural networks," IEEE Trans. Neural Networks, vol. 1, no. 2, pp. 239-242, 1990.
7. Y. Le Cun, J. S. Denker, and S. A. Solla, "Optimal brain damage", in Advances in Neural Information Processing Systems, D. S. Touretzky, Ed., San Mateo, CA Morgan Kaufmann, 1990, vol. 2, pp. 598-605.
8. B. Hassibi and D. G. Stork, "Second-order derivatives for network pruning: Optimal brain surgeon," in Advances in Neural Information Processing Systems, C. Lee Giles, S. J. Hanson, and J. D. Cowan, Eds., 1993, vol. 5, pp. 164-171.
9. M. W. Pedersen, L. K. Hansen, and J. Larsen, "Pruning with generalization based weight saliencies: γ OBD, γ OBS," in Advances in Neural Information Processing Systems, D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo, Eds., Cambridge, MA: MIT Press, 1996, vol. 8, pp. 521-528.
10. P. Burrascano, "A pruning technique maximizing generalization," in Proc. Int. Joint Conf. Neural Networks, vol. 1, pp. 347-350, 1993.
11. A. P. Engelbrecht and I. Cloete, "A Sensitivity analysis algorithm for pruning feedforward neural networks," in Proc. IEEE Int. Conf. Neural Networks, vol. 2, pp. 1274-1277, 1996.
12. J. M. Zurada, A. Malinowski, and S. Usui, "Perturbation method for deleting redundant inputs of perceptron networks," Neurocomputing, vol. 14, pp. 177-193, 1997.
13. X. Zeng and D. S. Yeung, "A Quantified Sensitivity measure for Multilayer Perceptron to Input Perturbation," to appear in Neural Computation.
14. J. Sietsma and R. J. F. Dow, "Creating Artificial Neural Networks that Generalize," Neural Networks, vol. 4, pp. 67-79, 1991