Pruning Algorithms—A Survey

IEEE Transactions on Neural Networks

Volumn 4, Issue 5, 1993, Pages 740-747

  Reed, Russell ^a  

^a UNIVERSITY OF WASHINGTON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ASSOCIATIVE PROCESSING; COMPUTATIONAL COMPLEXITY; CONVERGENCE OF NUMERICAL METHODS; CORRELATION METHODS; DATA STRUCTURES; ERROR ANALYSIS; LEARNING SYSTEMS; NEURAL NETWORKS; SYSTEMS ANALYSIS; TREES (MATHEMATICS);

PRUNING ALGORITHMS; TRAINING ERRORS; TRAINING PATTERNS;

ALGORITHMS;

EID: 0027662338     PISSN: 10459227     EISSN: 19410093     Source Type: Journal    
DOI: 10.1109/72.248452     Document Type: Article

References (37)

1. E. B. Baum, “When are k-nearest neighbor and back propagation accurate for feasible sized sets of examples?,” in Neural Networks, Proc. EURASIP Workshop, L. B. Almeida and C. J. Wellekens, Eds., Feb. 1990, pp. 2–25.
2. E. B. Baum and D. Haussler, “What size net gives valid generalization?,” Neural Computation, vol. 1, pp. 151–160, 1989.
3. A. Blumer, A. Ehrenfeucht, D. Haussler, and M. Warmuth, “Learnability and the Vapnik-Chervonenkis dimension,” J. Ass. Comput. Mach., vol. 36, no. 4, pp. 929–965, 1989.
4. Y. Chauvin, “A back-propagation algorithm with optimal use of hidden units,” in Advances in Neural Information Processing (1), D. S. Touretzky, Ed. (Denver 1988), 1989, pp. 519–526.
5. Y. Chauvin, “Dynamic behavior of constrained back-propagation networks,” in Advances in Neural Information Processing (2), D.S. Touretzky, Ed. (Denver 1989), 1990, pp. 642–649.
6. Y. Chauvin, “Generalization performance of overtrained backpropagation networks,” in Neural Networks, Proc. EUROSIP Workshop, L. B. Almeida and C. J. Wellekens, Eds., Feb. 1990, pp. 46–55.
7. Y. Le Cun, J. S. Denker, and S. A. Solla, “Optimal brain damage,” in Advances in Neural Information Processing (2), D.S. Touretzky, Ed. (Denver 1989), 1990, pp. 598–605.
8. A. Ehrenfeucht, D. Haussler, M. Kearns, and L. Valiant, “A general lower bound on the number of examples needed for learning,” in Proc. 1988 Workshop Computational Learning Theory, 1988.
9. S. J. Hanson and L. Y. Pratt, “Comparing biases for minimal network construction with back-propagation,” in Advances in Neural Information Processing (1), D.S. Touretzky, Ed. (Denver 1988), 1989, pp. 177–185.
10. F. Hergert, W. Finnoff, and H. G. Zimmermann, “A comparison of weight elimination methods for reducing complexity in neural networks,” in Proc. Int. Joint Conf Neural Networks, San Diego, CA, vol. III, 1992, pp. 980–987.
11. M. Ishikawa, “A structural learning algorithm with forgetting of link weights,” Tech. Rep. TR-90-7, Electrotechnical Lab., Tsukuba-City, Japan, 1990.
12. C. Ji, R. R. Snapp, and D. Psaltis, “Generalizing smoothness constraints from discrete samples,” Neural Computation, vol. 2, no. 2, pp. 188–197, 1990.
13. E. D. Karnin, “A simple procedure for pruning back-propagation trained neural networks,” IEEE Trans. Neural Networks, vol. 1, no. 2, pp. 239–242, 1990.
14. A. Krogh and J. A. Hertz, “A simple weight decay can improve generalization,” in Advances in Neural Information Processing (4), J. E. Moody, S. J. Hanson, and R. P. Lippmann, Eds., 1992, pp. 951–957.
15. J. K. Kruschke, “Creating local and distributed bottlenecks in hidden layers of back-propagation networks,” in Proc. 1988 Connectionist Models Summer School, D. Touretzky, G. Hinton, and T. Sejnowski, Eds., 1988, pp. 120–126.
16. J. K. Kruschke, “Improving generalization in back-propagation networks with distributed bottlenecks,” in Proc. Int. Joint Conf Neural Networks, Washington DC, vol. I, 1989, pp. 443—447 0.
17. S. Y. Kung and Y. H. Hu, “A Frobenius approximation reduction method (FARM) for determining optimal number of hidden units,” in Proc. Int. Joint Conf. Neural Networks, vol. II, Seattle, 1991, pp. 163–168.
18. E. Levin, N. Tishby, and S. A. Solla, “A statistical approach to learning and generalization in layered neural networks,” Proc IEEE, vol. 78, no. 10, pp. 1568–1574, Oct. 1990.
19. M. C. Mozer and P. Smolensky, “Skeletonization: A technique for trimming the fat from a network via relevance assessment,” in Advances in Neural Information Processing (1), D.S. Touretzky, Ed. (Denver 1988), 1989, pp. 107–115.
20. S. J. Nowlan and G. E. Hinton, “Simplifying neural networks by soft weight-sharing,” Neural Computation, vol. 4, no. 4, pp. 473–493, 1992.
21. O. M. Omidvara and C. L. Wilson, “Optimizaton of neural network topology and information content using Boltzmann methods,” in Proc. Int. Joint Conf. Neural Networks (Baltimore), vol. IV, 1992, pp. 594–599.
22. D. C. Plaut, S. J. Nowlan, and G. E. Hinton, “Experiments on learning by back propagation,” Tech. Rep. CMU-CS-86-126, Carnegie-Mellon Univ., 1986.
23. S. Ramachandran and L. Y. Pratt, “Information measure based skeletonisation,” in Advances in Neural Information Processing (4), J. E. Moody, S. J. Hanson, and R. P. Lippmann, Eds., 1992, pp. 1080–1087.
24. A. Sankar and R. J. Mammone, “Optimal pruning of neural tree networks for improved generalization,” in Proc. Int. Joint Conf. Neural Networks, vol. II (Seattle), 1991, pp. 219–224.
25. D. B. Schwartz, V. K. Samalan, S. A. Solla, and J. S. Denker, “Exhaustive learning,” Neural Computation, vol. 2, no. 3, pp. 374–385, 1990.
26. B. E. Segee and M. J. Carter, “Fault tolerance of pruned multilayer networks,” in Proc. Int. Joint Conf. Neural Networks, vol. II (Seattle), pp. 447–452, 1991.
27. J. Sietsma and R. J. F. Dow, “Creating artificial neural networks that generalize,” Neural Networks, vol. 4, no. 1, pp. 67–69, 1991.
28. J. Sietsma and R. J. F. Dow, “Neural net pruning—why and how,” in Proc. IEEE Int. Conf. Neural Networks, vol. I (San Diego), 1988, pp. 325–333.
29. W. E. Simon and J. R. Carter, “Learning to identify letters with REM equations,” in Proc. Int. Joint Conf. Neural Networks, vol. I (Washington, DC), 1990, pp. 727–730.
30. W. E. Simon and J. R. Carter, “Removing and adding network connections with recursive error minimization (REM) equations,” in Applications of Artificial Neural Networks, S.K. Rogers, Ed. 1990, pp. 600–606.
31. N. Tishby, E. Levin, and S. A. Solla, “Consistent inference of probabilities in layered networks: Predictions and generalization,” in Proc. Int. Joint Conf. Neural Networks, 1989, p. 403.
32. L. G. Valiant, “A theory of the learnable,” Commun. Ass. Comput. Mach., vol. 27, no. 11, pp. 1134–1142, 1984.
33. J.H. Wang, T. F. Krile, and J. F. Walkup, “Reduction of interconnection weights in higher order associative memory networks,” in Proc. Int. Joint Conf. Neural Networks, vol. II (Seattle), 1991, pp. 177–182.
34. A. S. Weigend, D. E. Rumelhart, and B. A. Huberman, “Back-propagation, weight-elimination and time series prediction,” in Proc. 1990 Connectionist Models Summer School, D. Touretzky, J. Elman, T. Sejnowski, and G. Hinton, Eds., 1990, pp. 105–116.
35. A. S. Weigend, D. E. Rumelhart, and B. A. Huberman, “Generalization by weight-elimination applied to currency exchange rate prediction,” in Proc. Int. Joint Conf. Neural Networks, vol. I (Seattle), 1991, pp. 837–841.
36. A. S. Weigend, D. E. Rumelhart, and B. A. Huberman, “Generalization by weight-elimination with application to forecasting,” in Advances in Neural Information Processing (3), R. Lippmann, J. Moody, and D. Touretzky, Eds., 1991, pp. 875–882.
37. D. Whitley and C. Bogart, “The evolution of connectivity: Pruning neural networks using genetic algorithms,” in Proc, Int. Joint Conf. Neural Networks, vol. I (Washington, DC), 1990, p. 134.