Premature saturation in backpropagation networks: Mechanism and necessary conditions

Neural Networks

Volumn 10, Issue 4, 1997, Pages 721-735

  Vitela, Javier E ^a,b   Reifman, Jaques ^a  

^a ARGONNE NATIONAL LABORATORY   (United States)

^b INSTITUTO DE CIENCIAS NUCLEARES   (Mexico)

Author keywords

Backpropagation training algorithm; Feedforward neural networks; Flat spot; Necessary conditions; Premature saturation

Indexed keywords

PREMATURE SATURATION;

ALGORITHMS; BACKPROPAGATION; COMPUTER SIMULATION;

FEEDFORWARD NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; LEARNING; MATHEMATICAL MODEL; PRIORITY JOURNAL;

EID: 0031171576     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(96)00117-7     Document Type: Article

References (12)

1. Balakrishnan, K., & Honavar, V. (1992). Improving convergence of back-propagation by handling flat-spots in the output layer. In I. Aleksander & J. Taylor (Eds.), Proceedings of the International Conference on Artificial Neural Networks: Vol. 2 (pp. 1003-1009). Elsevier Science: North-Holland.
2. Chen, J.R., & Mars, P. (1990). Stepsize variation methods for accelerating the back-propagation algorithm. In M. Caudill (Ed.), Proceedings of the International Joint Conference on Neural Networks: Vol. I (pp. 601-604). Piscataway, NJ: IEEE Neural Networks Council.
3. Dahl, E.D. (1987). Accelerated learning using the generalized delta rule. In M. Caudill & C.B. Butler (Eds.), Proceedings of the International Joint Conference on Neural Networks: Vol. II (pp. 523-530). Piscataway. NJ: SOS Printing.
4. Fahlman, S.E. (1989). Faster-Learning Variations on back-propagation: an empirical study. In D. Touretzky, G. Hinten, & T. Sejnowski (Eds.), Proceedings of the Connectionist Models Summer School (pp. 38-51). San Mateo, CA: Morgan Kaufmann.
5. Franzini, M.A. (1987). Speech recognition with back propagation. In Proceedings of the Ninth Annual Conference of the IEEE Engineering in Medicine and Biology Society: Vol. 33 (pp. 1702-1703). New York, NY: IEEE.
6. Lee, Y., Oh, S.H., & Kim, M.W. ( 1991 ). The effect of initial weights on premature saturation in back-propagation training. In Proceedings of the International Joint Conference on Neural Networks: Vol. II (pp. 765-770). Piscataway, NJ: IEEE Neural Networks Council.
7. Parekh, R., Balakrishnan, K., & Honavar, V. (1993). An empirical comparison of flat-spot elimination techniques in back-propagation networks. In M.L. Padgett (Ed.), Proceedings of the Third Workshop on Neural Networks: Academic/Industrial/NASA/Defense (pp. 55-60). San Diego, CA: Society for Computer Simulation.
8. Reifman, J., & Vitela, J.E. (1994). Accelerating learning of neural networks with conjugate gradients for nuclear power plant applications. Nuclear Technology, 106, 225-241.
9. Reifman, J., Gibbs, K.S., Vitela, J.E., & Benedict, R.W. (1995). Automatic inspection for remotely manufactured fuel elements. In Proceedings of the American Nuclear Society 6th Topical Meeting on Robotics and Remote Systems: Vol. 2 (pp. 696-703). La Grange Park, IL: American Nuclear Society.
10. Rumelhart, D.E., Hinton, G.E., & Williams, R.J. ( 1986). Learning internal representations by error propagation. In D.E. Rumelhart & J.C. McClelland (Eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Vol. I (pp. 319-362). Cambridge, MA: MIT Press.
11. Spartz, R., & Honavar, V. (1993). An empirical analysis of the expected source values rule. In M.L. Padgett (Ed.), Proceedings of the Third Workshop on Neural Networks: Academic/Industrial/NASA/Defense (pp. 95-100). San Diego, CA: Society for Computer Simulation.
12. Vitela, J.E., & Reifman, J. (1993). Enhanced backpropagation training algorithm for transient event identification. Transactions of the American Nuclear Society, 69, 148-149.