Connection pruning with static and adaptive pruning schedules

Neurocomputing

Volumn 16, Issue 1, 1997, Pages 49-61

  Prechelt, Lutz ^a  

^a UNIVERSITY OF KARLSRUHE   (Germany)

Author keywords

Early stopping; Empirical study; Generalization; Pruning

Indexed keywords

LEARNING SYSTEMS; PROBLEM SOLVING; STATISTICAL TESTS;

NEURAL NETWORK PRUNING METHODS;

NEURAL NETWORKS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; METHODOLOGY; PRIORITY JOURNAL; STATISTICAL ANALYSIS; TRAINING;

EID: 0031194103     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0925-2312(96)00054-9     Document Type: Article

References (18)

1. J.D. Cowan, G. Tesauro and J. Alspector, ed., Advances in Neural Information Processing Systems 6 (Morgan Kaufman, San Mateo, CA, 1994).
2. Y.L. Cun, J.S. Denker and S.A. Solla, Optimal brain damage, in: [16, pp. 598-605].
3. S. E. Fahlman, An empirical study of learning speed in back-propagation networks, Technical Report CMU-CS-88-162, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, 1988.
4. S.E. Fahlman and C. Lebiere, The Cascade-Correlation learning architecture, In: [16, pp. 524-532].
5. W. Finnoff, F. Hergert and H.G. Zimmermann, Improving model selection by nonconvergent methods, Neural Networks 6 (1993) 771-783.
6. S. Geman, E. Bienenstock and R. Doursat, Neural networks and the bias/variance dilemma, Neural Comput. 4 (1992) 1-58.
7. S.J. Hanson, J.D. Cowan and C.L. Giles, ed., Advances in Neural Information Processing Systems (Morgan Kaufman, San Mateo, CA, 1993).
8. B. Hassibi and D.G. Stork, Second order derivatives for network pruning: optimal brain surgeon, in: [7, pp. 164-171].
9. B. Hassibi, D.G. Stork, G. Wolff and T. Watanabe, Optimal brain surgeon: extensions and performance comparisons, in: [1].
10. R.P. Lippmann, J.E. Moody and D.S. Touretzky, eds., Advances in Neural Information Processing Systems 3 (Morgan Kaufman, San Mateo, CA, 1991).
11. N. Morgan and H. Bourlard, Generalization and parameter estimation in feedforward nets: some experiments, in: [16, pp. 630-637].
12. S.J. Nowlan and G.E. Hinton, Simplifying neural networks by soft weight-sharing, Neural Comput. 4(4) (1992) 473-493.
13. L. Prechelt, PROBEN 1 - A set of benchmarks and benchmarking rules for neural network training algorithms, Technical Report 21/94, Fakultät für Informatik, Universität Karlsruhe, Germany, 1994. Anonymous FTP: /pub/papers/techreports/1994/1994-21.ps.gz on ftp.ira.uka.de.
14. L. Prechelt, A quantitative study of experimental evaluations of neural network learning algorithms: current research practice, Neural networks 9(3) (1996) 451-462.
15. M. Riedmiller and H. Braun, A direct adaptive method for faster backpropagation learning: The RPROP algorithm, in: Proc. IEEE Internat. Conf. on Neural Networks, San Francisco, CA (1993) 586-591.
16. D.S. Touretzky, ed., Advances in Neural Information Processing Systems 2 (Morgan Kaufman, San Mateo, CA, 1990).
17. A.S. Weigend, D.E. Rumelhart and B.A. Huberman, Generalization by weight-elimination with application to forecasting, in: [10, pp. 875-882].
18. P.M. Williams, Bayesian regularization and pruning using a Laplace prior, Technical Report CSRP-312, School of Cognitive and Computing Sciences, University of Sussex, Brighton, England, 1994. ftp://ftp.cogs.susx.ac.uk/pub/reports/csrp/csrp312.ps.Z.