Variable projection method and Levenberg-Marquardt algorithm for neural network training

IECON Proceedings (Industrial Electronics Conference)

Volumn , Issue , 2006, Pages 4492-4497

  Kim, Cheol Taek ^a   Lee, Ju Jang ^a   Kim, Hyejin ^b  

^a Korea Advanced Institute of Science and Technology (KAIST)   (South Korea)

^b Electronics and Telecommunications Research Institute (ETRI)   (South Korea)

Author keywords

[No Author keywords available]

Indexed keywords

CURVE FITTING; ELECTRONICS INDUSTRY; FEEDFORWARD NEURAL NETWORKS; GOLD; IMAGE CLASSIFICATION; INDUSTRIAL ELECTRONICS; LAGRANGE MULTIPLIERS; LEARNING SYSTEMS; LEAST SQUARES APPROXIMATIONS;

ANNUAL CONFERENCE; EXTREME LEARNING MACHINE; FASTER CONVERGENCE; FEED FORWARD; JACOBIAN; L-M ALGORITHMS; LEVENBERG-MARQUARDT ALGORITHMS; NEURAL NETWORK TRAINING; NUMERICAL EXAMPLES; OPTIMAL SOLUTIONS; SEPARABLE NONLINEAR LEAST SQUARES; VARIABLE PROJECTION;

NEURAL NETWORKS;

EID: 50249137037     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IECON.2006.347644     Document Type: Conference Paper

References (10)

1. G. B. Huang, Q. Y. Zhu, and C. K. Siew, "Extreme Learning Machine: A New Learning Scheme of Feedforward Neural Networks," 2004 International Joint Conference on Neural Networks, Budapest, Hungary, 2004.
2. G. H. Golub and V. Pereyra, "The Differentiation of Pseudo-Inverses and Nonlinear Least Squares Problems Whose Variable Separate," SIAM Journal on Numerical Analysis, vol. 10, no. 2, pp. 413-432, 1973.
3. G. H. Golub and V. Pereyra, "Separable nonlinear least squares: the variable projection method and its applications," Inverse Problems, vol. 19, pp. R1-R26, 2003.
4. S. McLoone, M. D. Brown, G. Irwin and G. Lightbody, "A Hybrid LinearNonlinear Training Algorithm for Feedforward Neural Networks," IEEE Trans. Neural Networks, vol. 9, no. 4, pp. 669-684, 1998.
5. M. T. Hagan and M. B. Menhaj, "Training Feedforward Networks with the Marquardt Algorithm," IEEE Trans. Neural Networks, vol. 5, no. 6, pp. 989-993, 1994.
6. S. Haykin, KALMAN FILTERING AND NEURAL NETWORKS, JOHN WILEY & SONS, INC., New York., Unite States of America, 2001.
7. S. Haykin, ADAPTIVE FILTER THEORY, Prentice-Hall, Inc., New Jersey, United States of America, 2002.
8. S. Haykin, NEURAL NETWORKS.Prentice-Hall, Inc., New Jersey, United States of America, 1999.
9. D. M. Gay, L. Kaufman, "Tradeoffs in Algorithms for Separable Nonlinear Least Squares" Proceedings of the 13th World Congress on Computational and Applied Mathematics, Dublin, 1991
10. J. Sjöberg and M. Viberg, "Separable non-linear least squares minimization - possible improvements for neural net fitting" IEEE Workshop in Neural Networks for Signal Processing, Amelia Island Plantation, FL, 1997.