The layer-wise method and the backpropagation hybrid approach to learning a feedforward neural network

IEEE Transactions on Neural Networks

Volumn 11, Issue 2, 2000, Pages 295-305

  Rubanov, Nickolai S ^a  

^a BELARUSIAN STATE UNIVERSITY   (Belarus)

Author keywords

Feedforward neural network; Generalization error; Layer wise learning method; Learning time; Second order methods

Indexed keywords

APPROXIMATION THEORY; BACKPROPAGATION; CONVERGENCE OF NUMERICAL METHODS; LEARNING ALGORITHMS; OPTIMIZATION; PATTERN RECOGNITION;

LAYER-WISE LEARNING METHODS;

FEEDFORWARD NEURAL NETWORKS;

EID: 0033749078     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.839001     Document Type: Article

References (26)

1. S. Chen, S. Billings, and P. Grant, "Non-linear system identification using neural networks," Int. J. Contr., vol. 51, no. 6, pp. 1191-1214, 1990.
2. N. Morgan and H. A. Bourlard, "Neural networks for statistical recognition of continuous speech," Proc. IEEE, vol. 83, no. 5, pp. 741-770.
3. S. Haykin, "Neural networks expand SP's horizons," IEEE Signal Processing Mag., vol. 3, no. 3, pp. 24-49, 1996.
4. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning internal representations by error propagation," in Parallel Distributed Processing: Explorations in the Micro structure of Cognition, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge, MA: MIT Press, 1986, vol. 1: Foundations.
5. R. Lippmann, "An introduction to computing with neural nets," IEEE ASSP Mag., no. 4, pp. 4-22, 1987.
6. S. W. Ellacot, "Aspects of the numerical analysis of neural networks," in Acta Numerica: Cambridge University Press, 1994.
7. P. Patric van der Smagt, "Minimization methods for training feedforward neural networks," Neural Networks, vol. 7, no. 1, pp. 1-11, 1994.
8. S. T. Hunt and J. R. Deller, "Selective training of feedforward artificial neural networks using matrix perturbation theory," Neural Networks, vol. 8, no. 6, 1995.
9. F. Baermann and F. Biegler-Koenig, "On a class of efficient learning algorithm for multi-layered neural networks," Neural Networks, vol. 5, no. 1, 1992.
10. Y. Iiguni and H. Sakai, "A real-time learning algorithm for multilayered neural network based on the extended Kalman filter," IEEE Trans. Signal Processing, vol. 40, 1992.
11. S. Ergenziger and E. Thompsen, "An accelerated learning algorithm for multilayer perceptron: Optimization layer by layer," IEEE Trans. Neural Networks, vol. 6, pp. 31-42, 1995.
12. M. R. Azimi-Sadjadi and R. J. Liou, "Fast learning process of multilayer neural network using recursive least squares method," IEEE Trans. Signal Processing, vol. 40, 1992.
13. M. Pfister and R. Rojas, "Hybrid learning algorithms for neural networks - The adaptive inclusion of second order information," Zeitschrift fuer Angewandte Mathematik und Mechanik, vol. 76, no. 1, pp. 215-218, 1996.
14. U. Mueller, A. Gunzinger, and W. Guggenbuehl, "Fast neural-net simulation with a DSP processor array," IEEE Trans. Neural Networks, vol. 6, pp. 203-213, 1995.
15. S. E. Fahlman, "An empirical study of learning speed in backpropagation networks,", Tech. Rep. CMU-CS-88-162, 1988.
16. G. Goodwin and K. Sin, Adaptive Filtering, Prediction and Control. Englewood Cliffs, NJ: Prentice-Hall, 1984.
17. F. Natterer and F. Wuebelling, "A propagation-backpropagation method for ultrasound tomography," Inverse Problems, vol. 11, no. 6, 1995.
18. F. Natterer, The Mathematics of Computerized Tomography. New York: Wiley, 1986.
19. D. Young, Iterative Solution of Large Linear Systems. New York: Academic, 1971.
20. S. Luenberger, Linear and Nonlinear Programming, 2nd ed. Reading, MA: Addison-Wesley, 1984.
21. V. Visniauskas, F. Groen, and B. Kroese, "Function approximation with a feedforward network: The optimal number of learning samples and hidden units," Dept. Comput. Syst., Univ. Amsterdam, 1992.
22. A. J. Robinson, "Dynamic error propogation networks," Ph.D. dissertation, Eng. Dept., Cambridge Univ., Cambridge, U.K., 1989.
23. R. P. Gorman and T. J. Sejnowski, "Analysis of hidden units in a layered network trained to classify sonar target," Neural Networks, no. 1, pp. 75-89, 1988.
24. H. Bauschke and J. Borwein, "On projection algorithms for solving convex feasibility problems," SIAM Rev., vol. 38, no. 3, pp. 367-426.
25. G. Strang, Linear Algebra and Its Applications. Philadelphia, PA: Saunders, 1988.
26. A. Lapedes and R. Farber, "Nonlinear signal processing using neural networks, prediction and system modeling," Los Alamos National Laboratory, Tech. Rep. LA-UR-87-2662, 1987.