Efficient backpropagation learning using optimal learning rate and momentum

Neural Networks

Volumn 10, Issue 3, 1997, Pages 517-527

  Yu, Xiao Hu ^a   Chen, Guo An ^a  

^a SOUTHEAST UNIVERSITY   (China)

Author keywords

backpropagation learning; multilayer feedforward neural networks

Indexed keywords

ALGORITHMS; BACKPROPAGATION; COMPUTER SIMULATION; FEEDFORWARD NEURAL NETWORKS; OPTIMAL SYSTEMS;

BACKPROPAGATION LEARNING; LEARNING RATE; MOMENTUM FACTOR; MULTILAYER FEEDFORWARD NEURAL NETWORKS;

LEARNING SYSTEMS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; LEARNING; PRIORITY JOURNAL;

EID: 0031127257     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(96)00102-5     Document Type: Article

References (21)

1. Allred, L.G., and Kelly, G.E. (1990). Supervised learning techniques for backpropagation networks. In Proceedings of the International Joint Conference on Neural Networks, I, San Diego (pp. 702-709).
2. Battiti R. First- and Second-order methods for learning: Between steepest descent and Newton's method. Neural Computation. 4:1992;141-166.
3. Becker, S., and leCun, Y. (1988). Improving the convergence of back-propagation learning with second order methods. In Proceedings of the 1988 Connectionist Models Summer School (pp. 29-37).
4. Chen, G.-A. (1994). Training Theory for Feedforward Networks with Applications to Channel Equalization. Unpublished MS thesis, Southheast University, Nanjing, P.R. China.
5. Haykin, S. (1994). Neural networks - a comprehensive foundation (pp. 138-219). New York: Macmillan.
6. Hush D.R., Horne B.G. Progress in supervised neural networks. IEEE Signal Processing Magazine. 10(1):1993;8-39.
7. Lippmann R.P. An introduction to computing with neural networks. IEEE ASSP Magazine. 4(2):1987;4-22.
8. Jacobs R.A. Increased rates of convergence through learning rate adaptation. Neural Networks. 1(4):1988;295-308.
9. Kollias S., Anastrassiou D. An adaptive least square algorithm for the efficient training of artificial neural networks. IEEE Transactions on Circuits and Systems. 36:1989;1092-1101.
10. Møller M.S. A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks. 6(4):1993;525-534.
11. Monhandes, M., Codrington, C.W., & Gelfand, S.B. (1994). Two adaptive stepsize rules for gradient descent and their application to the training of feedforward artificial neural networks. In Proceedings of the IEEE International Conference on Neural networks, I, Orlando (pp. 555-560).
12. Peterson G.F., St.Clair D.C., Aylward S.R., Bond W.E. Using Taguchi's method of experimental design to control errors in layered perceptions. IEEE Transactions on Neural Networks. 6(4):1995;949-961.
13. Pushkorius, G.V., & Feldkamp, L.A. (1991). Decoupled extended Kalman training of feedforward layered networks. In Proceedings of the International Joint Conference on Neural Networks, I, Seattle (pp. 771-777).
14. Rumelhart, D.E., & McClelland, J.L. (1986) Parallel distributed processing: explorations in the microstructure of cognition, Vol. I. MIT Press.
15. Singhal, S., & Wu, L. (1989). Training feedforward networks with the extended Kalman algorithm. In Proceedings of the International Conference on Acoustics, Speech and Signal Processing, Scotland (pp. 1187-1190).
16. Vogel T.P., Mangis J.K., Rigler A.K., Zink W.T., Alkon D.L. Accelerating the convergence of the backpropagation method. Biological Cybernetics. 59:1988;257-263.
17. Webb, A.R., Lowe, D., & Bedworth, M.D. (1988). A comparison of nonlinear optimization strategies for feed-forward adaptive layered networks. Royal Signals & Radar Establishment, Memorandum no. 4157.
18. Wolfe, M.A. (1978). Numerical methods for unconstrained optimization. Holland: Van Nostrand Reinhold.
19. Yu X.-H., Cheng S.-X. Training algorithms for backpropagation neural networks with optimal descent factor. Electronics Letters. 26(20):1990;1698-1700.
20. Yu X.-H., Chen G.-A., Cheng S.-X. Acceleration of back-propagation learning using optimized learning rate and momentum. Electronics Letters. 59(14):1993;1288-1290.
21. Yu X.-H., Chen G.-A., Cheng S.-X. Dynamic learning rate optimization of the backpropagation algorithm. IEEE Transactions on Neural Networks. 6(3):1995;669-677.