Fast second order learning algorithm for feedforward multilayer neural networks and its applications

Neural Networks

Volumn 9, Issue 9, 1996, Pages 1583-1596

  Osowski, Stanislaw ^a   Bojarczak, Piotr ^b   Stodolski, Maciej ^a  

^a WARSAW UNIVERSITY OF TECHNOLOGY   (Poland)

^b Higher Engineering School   (Poland)

Author keywords

feedforward neural networks; second order learning algorithms

Indexed keywords

CONTROL NONLINEARITIES; IDENTIFICATION (CONTROL SYSTEMS); LEARNING ALGORITHMS; MATHEMATICAL MODELS; OPTIMIZATION; SIGNAL PROCESSING;

FAST SECOND ORDER LEARNING ALGORITHMS; FEEDFORWARD NEURAL NETWORKS;

NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; LEARNING; PRIORITY JOURNAL;

EID: 0030477946     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(96)00029-9     Document Type: Article

References (18)

1. Achenie, L.E. (1994). Computational experience with a quasi Newton method based training of the feedforward neural networks. Proc. World Congress on Neural Networks, San Diego, III607-III612.
2. Battiti R. First and second order methods for learning: between steepest descent and Newton methods. Neural Computation. 4:1992;141-166.
3. Battiti, R., & Masulli, F. (1990). BFGS optimization for faster and automated supervised learning. Proc. IEEE Int. NN Conf., Paris (pp. 757-760).
4. Charalambous C. Conjugate gradient algorithm for efficient training of artificial neural networks. IEEE Proc. G. 139:1992;301-309.
5. Denoeux T., Lengelle R. Initializing back propagation with prototypes. Neural Networks. 6:1993;351-363.
6. Fahlman, S. (1988). Fast learning variations on backpropagation: an empirical study. Proc. 1988 Connectionist Models, Summer School (pp. 38-51).
7. Gill, P., Murray, W., & Wright, M. (1981). Practical optimization. New York: Academic Press.
8. Jutten, C., Nguyen Thi, L., Dijkstra, E., Vittoz, E., & Caelen, J. (1991a). Blind separation of sources: algorithm for separation of convolutive mixtures. Proc. Int. Workshop on Higher Order Statistics, Chamrousse (pp. 273-276).
9. Jutten, C., Guerin, A., & Nguyen, Thi H. (1991b). Adaptive optimization of neural algorithms. In A. Prieto (Ed.). Artificial neural networks (pp. 54-61). Berlin: Springer Verlag.
10. Karayiannis, N., & Venetsanopoulos, A. (1992). Fast learning algorithms for neural networks. IEEE Trans. CAS, 39, 453-473.
11. Levin, A. (1993). Predicting with feedforward networks. Proc. IEEE World Congress on Neural Networks. Portland (pp. III385-III388).
12. Li, X., Bodruzzaman, M., Kuah, K., Szu, H., & Telfer, B. (1993). Neural network based nonlinear deconvolution. Proc. IEEE World Congress on NN, Portland (pp. IV817-IV824).
13. Moeller, M.F. (1993). A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks, 6, 525-533; also internal memo of Computer Science Dept, University of Aarhus, Denmark, preprint PB-339.
14. Narendra K., Parthasarathy K. Identification and control of dynamical systems by using neural networks. IEEE Trans. on Neural Networks. 1:1990;4-27.
15. Osowski S. Signal flow graphs and neural networs. Biological Cybernetics. 70:1993;387-395.
16. Osowski, S. (1994). Fast learning algorithms for feedforward multilayer neural networks. Proc. IEEE 1994 IMACS Int. Symp. Signal Processing and Neural Networks, Lille (pp. 27-30).
17. Sperduti A., Starita A. Speed up learning and network optimization with extended backpropagation. Neural Networks. 6:1993;365-383.
18. Widrow, B., & Bilello, M. (1994). Nonlinear adaptive signal processing for inverse control. Proc. World Congress on Neural Networks, San Diego (pp. III3-III13).