A penalty-function approach for pruning feedforward neural networks

Neural Computation

Volumn 9, Issue 1, 1997, Pages 185-204

  Setiono, Rudy ^a  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; CLUSTER ANALYSIS; LEARNING; NERVE CELL; PHYSIOLOGY; REPRODUCIBILITY; ROBOTICS; STATISTICAL MODEL;

ALGORITHMS; CLUSTER ANALYSIS; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER); NEURONS; PROBABILITY LEARNING; REPRODUCIBILITY OF RESULTS; ROBOTICS;

EID: 0030633575     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/neco.1997.9.1.185     Document Type: Article

References (25)

1. Ash, T. 1989. Dynamic node creation in backpropagation networks. Connection Science 1(4), 365-375.
2. Chauvin, Y. 1989. A back-propagation algorithm with optimal use of hidden units. In Advances in Neural Information Processing Systems, Vol. 1, pp. 519-526. Morgan Kaufmann, San Mateo, CA.
3. Chung, F. L., and Lee, T. 1992. A node pruning algorithm for backpropagation networks. Int. J. Neural Systems 3(3), 301-314.
4. Dennis, J. E., and Schnabel, R. B. 1983. Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice Hall, Englewood Cliffs, NJ.
5. Fahlman, S. E., and Lebiere, C. 1990. The cascade-correlation learning architecture. In Advances in Neural Information Processing Systems, Vol. 2, pp. 524-532. Morgan Kaufmann, San Mateo, CA.
6. Frean, M. 1990. The upstart algorithm: A method for constructing and training feedforward neural networks. Neural Computation 2(2), 198-209.
7. Gorodkin, J., Hansen, L. K., Krogh, A., and Winther, O. 1993. A quantitative study of pruning by optimal brain damage. Int. J. Neural Systems 4(2), 159-169.
8. Grossman, S. I. 1995. Multivariable Calculus, Linear Algebra, and Differential Equations. Saunders College Publishing, Orlando, FL.
9. Hanson, S. J., and Pratt, L. Y. 1989. Comparing biases for minimal network construction with back-propagation. In Advances in Neural Information Processing Systems, Vol. 1, pp. 177-185. Morgan Kaufmann, San Mateo, CA.
10. Hassibi, B., and Stork, D. G. 1993. Second order derivatives for network pruning: Optimal brain surgeon. In Advances in Neural Information Processing Systems, Vol. 5, pp. 164-171. Morgan Kaufmann, San Mateo, CA.
11. Hertz, J., Krogh, A., and Palmer, R. G. 1991. In Introduction to the Theory of Neural Computation. Addision-Wesley, Redwood City, CA.
12. Hinton, G. E. 1989. Connectionist learning procedures. Artificial Intelligence 40, 185-234.
13. Karnin, E. D. 1990. A simple procedure for pruning back-propagation trained neural networks. IEEE Trans. Neural Networks 1(2), 239-242.
14. Lang, K. J., and Witbrock, M. J. 1988. Learning to tell two spirals apart. In Proc. of the 1988 Connectionist Summer School, pp. 52-59. Morgan Kaufmann, San Mateo, CA.
15. Le Cun, Y., Denker, J. S., and Solla, S. A. 1990. Optimal brain damage. In Advances in Neural Information Processing Systems, Vol. 2, pp. 598-605. Morgan Kaufmann, San Mateo, CA.
16. Mezard, M., and Nadal, J. P. 1989. Learning in feedforward layered networks: The tiling algorithm. Journal of Physics A 22(12), 2191-2203.
17. Mozer, M. C., and Smolensky, P. 1989. Skeletonization: A technique for trimming the fat from a network via relevance assestment. In Advances in Neural Information Processing Systems, Vol. 1, pp. 107-115. Morgan Kaufmann, San Mateo, CA.
18. Setiono, R. 1997. Extracting rules from neural networks by pruning and hidden unit splitting. Neural Computation 9, 205-225.
19. Tenorio, M. F., and Lee, W. 1990. Self-organizing network for optimum supervised learning. IEEE Trans. Neural Networks 1(1), 100-110.
20. Thodberg, H. H. 1991. Improving generalization of neural networks through pruning. Int. J. Neural Systems 1(4), 317-326.
21. Thrun, S. B., Bala, J., Bloedorn, E., Bratko, I., Cestnik, B., Cheng, J., De Jong, K., Dzeroski, S., Fahlman, S. E., Fisher, D., Hamann, R., Kaufman, K., Keller, S., Kononenko, I., Kreuziger, J., Michalski, R. S., Mitchell, T., Pachowicz, P., Reich, Y., Vafaie, H., Van de Weide, W., Wenzel, W., Wnek, J., and Zhang, J. 1991. The MONK's problems - a performance comparison of different learning algorithms. Preprint CMU-CS-91-197. Carnegie Mellon University, Pittsburgh, PA.
22. van Ooyen, A., and Nienhuis, B. 1992. Improving the convergence of the back-propagation algorithm. Neural Networks 5, 465-471.
23. Watrous, R. 1987. Learning algorithms for connectionist networks: Applied gradient methods of nonlinear optimization. Proc. IEEE First International Conference on Neural Networks, pp. 619-627. IEEE Press, New York.
24. Weigend, A. S. 1993. On overfitting and the effective number of hidden units. In Proc. of the 1993 Connectionist Models Summer School, pp. 335-342. Lawrence Erlbaum, Hillsdale, NJ.
25. Weigend, A. S., Rumelhart, D. E., and Huberman, B. A. 1990. Back-propagation, weight-elimination and time series prediction. In Proc. of the 1990 Connectionist Models Summer School, pp. 105-116. Morgan Kaufmann, San Mateo, CA.