An efficient hidden layer training method for the multilayer perceptron

Neurocomputing

Volumn 70, Issue 1-3, 2006, Pages 525-535

  Yu, Changhua ^a   Manry, Michael T ^b   Li, Jiang ^c   Lakshmi Narasimha, Pramod ^b  

^a FastVDO Inc   (United States)

^b The University of Texas at Arlington   (United States)

^c NATIONAL INSTITUTES OF HEALTH   (United States)

Author keywords

Adaptive learning factor; Convergence; Hidden layer error function; Hidden weight optimization (HWO); Saturation

Indexed keywords

ALGORITHMS; LEARNING SYSTEMS; LINEAR EQUATIONS; OPTIMIZATION;

ADAPTIVE LEARNING FACTOR; HIDDEN LAYER ERROR FUNCTION; HIDDEN WEIGHT OPTIMIZATION (HWO); MEAN SQUARE ERROR;

MULTILAYER NEURAL NETWORKS;

ALGORITHM; ANALYTICAL ERROR; ANALYTICAL PARAMETERS; ARTICLE; CONTROLLED STUDY; CORRELATION ANALYSIS; HELICOPTER; LEARNING; LINEAR SYSTEM; MATHEMATICAL COMPUTING; MEASUREMENT; MECHANICAL STRESS; PRIORITY JOURNAL; PROCESS OPTIMIZATION; REMOTE SENSING; SIMULATION; SURFACE PROPERTY;

EID: 33750376626     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2005.11.008     Document Type: Article

References (30)

1. Barton S.A. A matrix method for optimizing a neural network. Neural Computation 3 3 (1991) 450-459
2. Battiti R. First- and Second-order methods for learning: between steepest descent and Newton's method. Neural computation 4 2 (1992) 141-166
3. Chen H.-H., Manry M.T., and Chandrasekaran H. A neural network training algorithm utilizing multiple set of linear equations. Neurocomputing 25 1-3 (1999) 55-72
4. Y. L.eCun, Generalization and network design strategies, in: Proceedings of Connectionism in perspective, 1989.
5. Dawson M.S., Fung A.K., and Manry M.T. Surface parameter retrieval using fast learning neural networks. Remote Sensing Reviews 7 1 (1993) 1-18
6. Fletcher R. Practical Methods of Optimization (1987), Wiley, New York
7. M.H. Fun, M.T. Hagan, Levenberg-Marquardt training for modular networks, The 1996 IEEE International Conference on Neural Networks, vol. 1, 1996, pp. 468-473.
8. C.M. Hadzer, R. Hasan, et al., Improved singular value decomposition by using neural networks, IEEE International Conference on Neural Networks, vol. 1, 1995 438-442.
9. Hagan M.T., and Menhaj M.B. Training feedforward networks with the Marquardt algorithm. IEEE Transaction on Neural Networks 5 6 (1994) 989-993
10. Hagan M.T., Demuth H.B., and Beale M.H. Neural Network Design (1996), PWS publishing Company
11. Haykin S. Neural networks: a comprehensive foundation (1999), Prentice-Hall, Englewood Cliffs, NJ
12. T.H. Kim, M.T. Manry, F.J. Maldonado, New learning factor and testing methods for conjugate gradient training algorithm, IJCNN'03. International Joint Conference on Neural Networks, 2003, pp. 2011-2016.
13. Lee H.-M., Chen C.-M., and Huang T.-C. Learning efficiency improvement of back-propagation algorithm by error saturation prevention method. Neurocomputing 41 (2001) 125-143
14. Lee Y., Oh S.-H., and Kim M.W. An analysis of premature saturation in back-propagation learning. Neural Networks 6 (1993) 719-728
15. Magoulas G.D., Vrahatis M.N., and Androulakis G.S. Improving the convergence of the backpropagation algorithm using learning adaptation Methods. Neural Computation 11 (1999) 1769-1796
16. Manry M.T., Chandrasekaran H., and Hsieh C.-H. Signal Processing Using the Multiplayer Perceptron (2001), CRC Press, Boca Raton
17. Manry M.T., et al. Fast training of neural networks for remote sensing. Remote Sensing Reviews 9 (1994) 77-96
18. Masters T. Neural, Novel & Hybrid Algorithms for Time Series Prediction (1995), Wiley, New York
19. Oh S.-H. Improving the error back-propagation algorithm with a modified error function. IEEE Transactions on Neural Networks 8 3 (1997) 799-803
20. Oh S.-H., and Lee S.-Y. A new error function at hidden layers for fast training of multilayer perceptrons. IEEE Transactions on Neural Networks 10 (1999) 960-964
21. S.-H. Oh, S.-Y. Lee, Optimal learning rates for each pattern and neuron in gradient descent training of multilayer perceptrons, IJCNN'99. International Joint Conference on Neural Networks, vol 3, 1999, pp. 1635-1638.
22. Oh Y., Sarabandi K., and Ulaby F.T. An empirical model and an inversion technique for radar scattering from bare soil surfaces. IEEE Transactions on Geoscience and Remote Sensing 30 2 (1992) 370-381
23. Ooyen V., and Nienhuis B. Improving the convergence of the back-propagation algorithm. Neural Networks 5 (1992) 465-471
24. Press W.H., et al. Numerical Recipes (1986), Cambridge University Press, New York
25. Rohani K., Chen M.S., and Manry M.T. Neural subset design by direct polynomial mapping. IEEE Transactions on Neural Networks 3 6 (1992) 1024-1026
26. Rumelhart D.E., Hiton G.E., and Williams R.J. Learning internal representation by error propagation. Parallel Distributed Processing 1 (1986) 318-362
27. R.S. Scalero, N. Tepedelenlioglu, A fast new algorithm for training feedforward neural networks, IEEE Transactions on signal processing, vol. 40 (1) 1992 pp. 202-210.
28. Wang G.-J., and Chen C.-C. A fast multilayer neural-network training algorithm based on the layer-by-layer optimizing procedures. IEEE Transactions on Neural Networks 7 3 (1996) 768-775
29. P.J. Werbos, Beyond regression: new tools for prediction and analysis in the behavioral science, Ph.D. Thesis, Harvard University, Cambridge, Mass, 1974.
30. Yam J.Y.F., and Chow T.W.S. A weight initialization method for improving training speed in feedforward neural network. Neurocomputing 30 (2000) 219-232