A neural network training algorithm utilizing multiple sets of linear equations

Neurocomputing

Volumn 25, Issue 1-3, 1999, Pages 55-72

  Chen, Hung Han ^a   Manry, Michael T ^b   Chandrasekaran, Hema ^b  

^a Cytel Incorporated   (United States)

^b The University of Texas at Arlington   (United States)

Author keywords

Backpropagation; Conjugate gradient method; Fast training; Hidden weight optimization; Learning factor calculation; Levenberg Marquardt algorithm; Multilayer perception; Output weight optimization; Second order methods

Indexed keywords

BACKPROPAGATION; CONVERGENCE OF NUMERICAL METHODS; ERROR ANALYSIS; ITERATIVE METHODS; LEARNING ALGORITHMS; LINEAR EQUATIONS; OPTIMIZATION;

HIDDEN WEIGHT OPTIMIZATION (HWO); LEVENBERG-MARQUARDT ALGORITHM; OUTPUT WEIGHT OPTIMIZATION (OWO);

MULTILAYER NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; CALCULATION; COMPUTER SIMULATION; LINEAR SYSTEM; MATHEMATICAL ANALYSIS; MATHEMATICAL COMPUTING; PRIORITY JOURNAL;

EID: 0032938952     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0925-2312(98)00109-X     Document Type: Article

References (31)

1. Anderson J.A. An Introduction to Neural Networks. 1986;267-271 The MIT Press, Cambridge, MA.
2. Baldi P., Hornik K. Neural networks and principal component analysis: learning from examples without local minima. Neural Networks. 2:1989;53-58.
3. Barton S.A. A matrix method for optimizing a neural network. Neural Comput. 3(3):1991;450-459.
4. Battiti R. First- and second - order methods for learning: between steepest descent and Newton's method. Neural Comput. 4(2):1992;141-166.
5. M.S. Chen, M.T. Manry, Back-propagation representation theorem using power series, Proc. Int. Joint Conf. on Neural Networks, San Diego, vol. 1, 1990, pp. 643-648.
6. M.S. Chen, M.T. Manry, Nonlinear modeling of back-propagation neural networks, Proc. Int. Joint Conf.on Neural Networks, Seattle, 1991, p. A-899.
7. Chen M.S., Manry M.T. Conventional modeling of the multi-layer perceptron using polynomial basis function. IEEE Trans. Neural Networks. 4(1):1993;164-166.
8. M.S. Dawson et al., Inversion of surface parameters using fast learning neural networks, Proc. Int. Geoscience and Remote Sensing Symp., Houston, TX, vol. 2 1992, pp. 910-912.
9. Dawson M.S., Fung A.K., Manry M.T. Surface parameter retrieval using fast learning neural networks. Remote Sensing Rev. 7(1):1993;1-18.
10. Fletcher R. Conjugate direction methods. Murray W. Numerical Methods for Unconstrained Optimization. 1972;Academic Press, New York.
11. M.H. Fun, M.T. Hagan, Levenberg-Marquardt training for modular networks, The 1996 IEEE Int. Conf. Neural Networks, vol. 1, 1996, pp. 468-473.
12. Fung A.K., Li Z., Chen K.S. Backscattering from a randomly rough dielectric surface. IEEE Trans. Geosci. Remote Sensing. 30(2):1992;356-369.
13. A.K. Fung, Microwave Scattering and Emission Models and Their Applications, Artech House, 1994.
14. Gill P.E., Murray W., Wright M.H. Practical Optimization. 1981;Academic Press, New York.
15. A. Gopalakrishnan et al., Constructive Proof of Efficient Pattern storage in the Multilayer Perceptron, Conf. Record of the 27th Annual Asilomar Conf. on Signals, Systems, and Computers, vol. 1, 1993, pp. 386-390.
16. Hagan M.T., Menhaj M.B. Training Feedforward Networks with the Marquardt Algorithm. IEEE Trans. Neural Networks. 5(6):1994;989-993.
17. Kollias S., Anastassiou D. An adaptive least squares algorithm for the efficient training of artificial neural networks. IEEE Trans. Circuits Systems. 36(8):1989;1092-1101.
18. Manry M.T.et al. Fast training of neural networks for remote sensing. Remote Sensing Rev. 9:1994;77-96.
19. Masters T. Neural, Novel & Hybrid Algorithms for Time Series Prediction. 1995;Wiley, New York.
20. McLoone S., Brown M.D., Irwin G., Lightbody G. A hybrid linear/nonlinear training algorithm for feedforward neural networks. IEEE Trans. Neural Networks. 9(9):1998;669-683.
21. Pao Y.H. Adaptive Pattern Recognition and Neural Networks. 1989;Addison-Wesley, New York.
22. D.B. Parker, Learning logic, Invention Report S81-64, File 1, Office of Technology Licensing, Stanford University, 1982.
23. Press W.H.et al. Numerical Recipes. 1986;Cambridge University Press, New York.
24. Rohani K., Chen M.S., Manry M.T. Neural subnet design by direct polynomial mapping. IEEE Trans. Neural Networks. 3(6):1992;1024-1026.
25. Rumelhart D.E., Hinton G.E., Williams R.J. Learning internal representations by error propagation. Rumelhart D.E., McClelland J.L. Parallel Distributed Processing. 1:1986;The MIT Press, Cambridge, MA.
26. Sartori M.A., Antsaklis P.J. A simple method to derive bounds on the size and to train multilayer neural networks. IEEE Trans. Neural Networks. 2(4):1991;467-471.
27. Scalero R.S., Tepedelenlioglu N. A fast new algorithm for training feedforward neural networks. IEEE Trans. Signal Process. 40(1):1992;202-210.
28. P. Werbos, Beyond regression: new tools for prediction and analysis in the behavioral sciences, Ph.D. Dissertation, Committee on Applied Mathematics, Harvard University, Cambridge, MA, Nov. 1974.
29. P. Werbos, Backpropagation: past and future, Proc. IEEE Int. Conf. on Neural Networks, 1988, pp. 343-353.
30. Widrow B., Stearns S.D. Adaptive Signal Processing. 1985;Prentice-Hall, Englewood Cliffs, NJ.
31. Yam J.Y.F., Chow T.W.S. Extended least squares based algorithm for training feedforward networks. IEEE Trans. Neural Networks. 8(3):1997;803-810.