Variable projections neural network training

Mathematics and Computers in Simulation

Volumn 73, Issue 1-4 SPEC. ISS., 2006, Pages 231-243

  Pereyra, V ^a   Scherer, G ^b   Wong, F ^a  

^a WEIDLINGER ASSOCIATES INC   (United States)

^b UNIVERSITY OF READING   (United Kingdom)

Author keywords

Neural networks; Variable projection algorithm

Indexed keywords

ALGORITHMS; INVERSE PROBLEMS; NONLINEAR EQUATIONS; ROBUSTNESS (CONTROL SYSTEMS);

VARIABLE PROJECTION ALGORITHM;

FEEDFORWARD NEURAL NETWORKS;

EID: 33749266690     PISSN: 03784754     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.matcom.2006.06.017     Document Type: Article

References (33)

1. Araoz J. Personal communication (1983)
2. Bishop Ch.M. Neural Networks for Pattern Recognition (1995), Oxford University Press
3. Björck A. Numerical Methods for Least Squares (1996), SIAM Publications, Philadelphia
4. Buhmann M.D. Radial Basis Functions (2003), Cambridge University Press
5. Coleman T., Garbow B.S., and More J. Software for estimating sparse Jacobian matrices. ACM Trans. Math. Software 10 (1984) 329-345
6. Cybenko G. Approximation by superpositions of a sigmoidal function. Math. Control Signals Syst. 2 (1989) 303-314
7. Eriksson J.E., Gulliksson M., Linström P., and Wedin P.-A. Regularization tools for training feed-forward neural networks, Part I and II. J. Optim. Software 10 (1998) 49-69
8. Gay D.M. Usage summary for selected optimization routines. AT&T Bell Labs. Comp. Sci. Technol. Rep. (1990)
9. D.M. Gay, NSF and NSG; PORT Library, AT & T Bell Labs. (available from na-net), 1997.
10. Gay D.M., and Kaufman L. Tradeoffs in algorithms for separable nonlinear least squares, AT&T Bell Labs. Numer. Anal. Manuscript (1990) 90-111
11. Golub G.H., and LeVeque R. Extensions and uses of the variable projection algorithm for solving nonlinear least squares problems. Proceedings of American Numerical Analysis and Computer Conference (1979)
12. Golub G.H., and van Loan C. Matrix Computations. second ed. (1989), John Hopkins
13. Golub G.H., and Pereyra V. The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate. SIAM J. Numer. Anal. 10 (1973) 413-432
14. Golub G.H., and Pereyra V. Separable nonlinear least squares: the variable projection method and its applications. Inverse Problems 19 (2003) R1-R26
15. Hansen P.Ch. Rank-Deficient and Discrete Ill-Posed Problems (1998), SIAM Publications, Philadelphia
16. http://www.netlib.org.
17. Kaufman L. A variable projection method for solving separable nonlinear least squares problems. BIT 15 (1975) 49-57
18. Kaufman L. Separable nonlinear least squares with multiple right hand sides. AT&T Bell Labs. Numer. Anal. Manuscript (1990) 90-103
19. Kaufman L., and Pereyra V. A method for separable nonlinear least squares with separable equality constraints. SIAM J. Numer. Anal. 15 (1978) 12-20
20. Levenberg K. A method for the solution of certain non-linear problems in least squares. Q. J. Appl. Math. 2 (1948) 164-168
21. L. Ngia, System Modeling Using Basis Functions and Applications to Echo Cancellation. Ph.D. Thesis, Chalmers Inst. of Techn., Sweden, NNSP2000, 2000.
22. Ngia L. Separable Nonlinear Least-Squares Methods for On-Line Estimation of Neural Nets Hammerstein Models. Manuscript (2001), Dept. Signals and Systems, Chalmers University of Technology, Sweden
23. Parisi R., Di Claudio D., Orlandi G., and Rao B.D. A generalized learning paradigm exploiting the structure of feedforward neural networks. IEEE Trans. Neural Networks 7 (1996)
24. Pati Y.C., and Krishnaprasad P.S. Analysis and synthesis of feedforward neural networks using discrete affine wavelet transformations. IEEE Trans. Neural Networks 4 (1993)
25. Pereyra V., and Scherer G. Large least squares scattered data fitting by truncated SVD. Appl. Numer. Math. 44 (2002) 225-239
26. Prechelt L. PROBEN 1-a set of benchmark neural network problems and benchmarking rules, Tech. Rep. 21 (1994), Fakultaet fuer Informatik, Universitaet Karlsruhe
27. Ruhe A., and Wedin P.-A. Algorithms for separable nonlinear least squares problems. SIAM Rev. 22 (1980) 318-337
28. Rust B.W. Fitting nature's basic functions part III: exponentials, sinusoids, and nonlinear least squares. Comput. Sci. Eng. (2002)
29. Saarinen S., Bramley R., and Cybenko G. Ill-Conditioning in neural network training problems. SIAM J. Sci. Stat. Comput. 14 (1993) 693-714
30. Sjöberg J., and Viberg M. Separable non-linear least squares minimization-possible improvements for neural net fitting. IEEE Workshop in Neural Networks for Signal Processing. Amelia Island Plantation, Fl (1997)
31. K. Weigl, M. Berthod, Neural Networks as Dynamical Bases in Function Space. Report #2124, INRIA, Prog. Robotique, Image et Vision. Sophia-Antipolis, France, 1993.
32. Weigl K., and Berthod M. Projection learning: alternative approach to the computation of the projection. Proceeding of European Symposium on Artificial Neural Networks. 19-24. Brussels,1; Belgium (1994)
33. K. Weigl, G. Giraudon, M. Berthod, Application of Projection Learning to the Detection of Urban Areas in SPOT Satellite Images. Report #2143, INRIA, Prog. Robotique, Image et Vision. Sophia-Antipolis, France, 1993.