Subset-based training and pruning of sigmoid neural networks

Neural Networks

Volumn 12, Issue 1, 1999, Pages 79-89

  Zhou, Guian ^a   Si, J ^a  

^a Arizona State University   (United States)

Author keywords

Jacobian matrix; Neural networks; Nonlinear least squares; Pruning; Rank deficiency; Subset based updating; Training; Trust regions

Indexed keywords

ALGORITHMS; CONVERGENCE OF NUMERICAL METHODS; LEARNING SYSTEMS; MATRIX ALGEBRA; VECTORS;

JACOBIAN MATRIX; LEVENBERG-MARQUARDT (LM) METHOD; SIGMOID NEURAL NETWORKS; SUBSET-BASED TRAINING (SBT); SUBSET-BASED TRAINING AND PRUNING (SBTP);

NEURAL NETWORKS;

ACCURACY; ALGORITHM; ARTIFICIAL NEURAL NETWORK; EVALUATION; INTERMETHOD COMPARISON; MATHEMATICAL MODEL; NONLINEAR SYSTEM; PRIORITY JOURNAL; REVIEW; SIGNAL PROCESSING;

EID: 0032980978     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(98)00105-1     Document Type: Article

References (18)

1. Dennis, J.E., & Schnabel, R.B. (1983). Numerical methods for unconstrained optimization and nonlinear equations. Englewood Cliffs, NJ: Prentice Hall.
2. Dongarra, J.J., Bunch, J.R., Moler, C.B., & Stewart, G.W. (1979). UNPACK: users' guide. Philadelphia, PA: SIAM.
3. Hagen, M.T., & Menhaj, M.B. (1994). Training feedforward networks with the Levenberg-Marquardt algorithm. IEEE Transactions on Neural Networks, 6, 989-993.
4. Hassibi, B., & Stork, D.O. (1993). Second order derivatives for network pruning: optimal brain surgeon. In S.J. Hanson (Ed.), Advances in neural information processing systems (Vol. 5, pp. 164-171). San Mateo, CA: Morgan Kaufmann.
5. Haykin, S. (1994). Neural networks: a comprehensive foundation. New Jersey: Macmillan, 1994.
6. Hertz, J., Krogh, A., & Palmer, R.G. (1991). Introduction to the theory of neural computation. Redwood City, CA: Addison-Wesley, 1991.
7. Hinton, G.E. (1986). Learning distributed representations of concepts. In Proceedings of the 8th Annual Conference of the Cognitive Science Society (pp. 1-12). Hillsdale, NJ: Erlbaum.
8. Le Cun, Y., Denker, J.S., & Solla, S.A. (1990). Optimal brain damage.
9. In D.S. Touretzky (Ed.), Advances in neural information processing systems (Vol. 2, pp. 598-605). San Mateo, CA: Morgan Kaufmann.
10. Levin, A.U., Leen, T.K., & Moody, J.E. (1994). Fast pruning using principal components. In G. Tesauro (Ed.), Advances in neural information processing systems (Vol. 6, pp. 35-42). San Mateo, CA: Morgan Kaufmann.
11. Luyben, W.L. (1990). Process modeling, simulation and control for chemical engineering. New York: McGraw-Hill.
12. More, J.J. (1983). Recent development in algorithms and software for trust region methods. In A. Bachern, M. Grötschel, & B. Korte (Eds.), Mathematical programming-the state and the art (pp. 258-287). Berlin: Springer.
13. More,-J.J., & Sorensen, D.C. (1983). Computing a trust region step. SIAM Journal on Scientific and Statistical Computing, 3, 553-572.
14. Saarinen, S., Bramley, R.B., & Cybenko, G. (1991). The numerical solution of neural network training problems. CRSD Report No. 1089, Center for Supercomputing Research and Development, University of Illinois, Urbana.
15. Schwetlick, H., & Tiller, V. (1989). Nonstandard scaling matrices for trust region Gauss-Newton methods. SIAM Journal on Scientific and Statistical Computing, 4, 654-670.
16. Weigend, A.S., Rumelhart, D.E., & Huberman, B.A., Generalization by weight-elimination with application to forecasting. In R.P. Lippmann, J.E. Moody, & D.S. Touretzky (Eds.), Advances in neural information processing systems (Vol. 3, pp. 857-882). San Mateo, CA: Morgan Kaufmann.
17. Weigend, A.S., Huberman, B.A., & Rumelhart, D.E. (1992). Predicting sunspots and exchange rates with connectionist networks. In M. Casdagli, & S. Eubank (Eds.), Nonlinear modeling and forecasting (pp. 395-432). Redwood City, CA: Addison-Wesley.
18. Zhou, G., & Si, J. (1998). A systematic and effective supervised learning mechanism based on Jacobian rank deficiency. Neural Computation, 10, 1031-1045.