Nonlinear conjugate gradient methods with structured secant condition for nonlinear least squares problems

Journal of Computational and Applied Mathematics

Volumn 234, Issue 2, 2010, Pages 375-397

  Kobayashi, Michiya ^a   Narushima, Yasushi ^b   Yabe, Hiroshi ^b  

^a ACCENTURE   (United States)

^b TOKYO UNIVERSITY OF SCIENCE   (Japan)

Author keywords

Conjugate gradient method; Global convergence; Least squares problems; Line search; Structured secant condition

Indexed keywords

GAUSS-NEWTON METHODS; GLOBAL CONVERGENCE; GLOBALLY CONVERGENT; LEAST SQUARES PROBLEMS; LEVENBERG-MARQUARDT METHOD; LINE SEARCH; LINE SEARCHES; NEWTON LIKE METHODS; NON-LINEAR OPTIMIZATION PROBLEMS; NONLINEAR CONJUGATE GRADIENT METHOD; NONLINEAR LEAST SQUARES PROBLEMS; NUMERICAL RESULTS; OBJECTIVE FUNCTIONS; QUASI-NEWTON METHODS;

LEAST SQUARES APPROXIMATIONS; NEWTON-RAPHSON METHOD;

CONJUGATE GRADIENT METHOD;

EID: 77649271327     PISSN: 03770427     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.cam.2009.12.031     Document Type: Article

References (32)

1. Dennis Jr. J.E., Gay D.M., and Welsch R.E. An adaptive nonlinear least-squares algorithm. ACM Transactions on Mathematical Software 7 (1981) 348-368
2. Dennis Jr. J.E., Martinez H.J., and Tapia R.A. Convergence theory for the structured BFGS secant method with an application to nonlinear least squares. Journal of Optimization Theory and Applications 61 (1989) 161-178
3. Al-Baali M., and Fletcher R. Variational methods for non-linear least-squares. Journal of the Operational Research Society 36 (1985) 405-421
4. Engels J.R., and Martinez H.J. Local and superlinear convergence for partially known quasi-Newton methods. SIAM Journal on Optimization 1 (1991) 42-56
5. Yabe H., and Yamaki N. Local and superlinear convergence of structured quasi-Newton methods for nonlinear optimization. Journal of the Operations Research Society of Japan 39 (1996) 541-557
6. Huschens J. On the use of product structure in secant methods for nonlinear least squares problems. SIAM Journal on Optimization 4 (1994) 108-129
7. Yabe H., and Ogasawara H. Quadratic and superlinear convergence of the Huschens method for nonlinear least squares problems. Computational Optimization and Applications 10 (1998) 79-103
8. Hestenes M.R., and Stiefel E. Methods of conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards 49 (1952) 409-436
9. Fletcher R., and Reeves C.M. Function minimization by conjugate gradients. Computer Journal 7 (1964) 149-154
10. Zoutendijk G. Nonlinear programming, computational methods. In: Abadie J. (Ed). Integer and Nonlinear Programming (1970), North-Holland, Amsterdam 37-86
11. Al-Baali M. Descent property and global convergence of the Fletcher-Reeves method with inexact line search. IMA Journal of Numerical Analysis 5 (1985) 121-124
12. Sorenson H.W. Comparison of some conjugate direction procedures for function minimization. Journal of The Franklin Institute 288 (1969) 421-441
13. Nocedal J., and Wright S.J. Numerical Optimization. Second Edition. Springer Series in Operations Research (2006), Springer-Verlag, New York
14. Powell M.J.D. Nonconvex minimization calculations and the conjugate gradient method. Lecture Notes in Mathematics vol. 1066 (1984), Springer-Verlag, Berlin 122-141
15. Powell M.J.D. Convergence properties of algorithms for nonlinear optimization. SIAM Review 28 (1986) 487-500
16. Gilbert J.C., and Nocedal J. Global convergence properties of conjugate gradient methods for optimization. SIAM Journal on Optimization 2 (1992) 21-42
17. Dai Y.H., and Yuan Y. A nonlinear conjugate gradient method with a strong global convergence property. SIAM Journal on Optimization 10 (1999) 177-182
18. Hager W.W., and Zhang H. A survey of nonlinear conjugate gradient methods. Pacific Journal of Optimization 2 (2006) 35-58
19. Perry A. A modified conjugate gradient algorithm. Operations Research 26 (1978) 1073-1078
20. Dai Y.H., and Liao L.Z. New conjugacy conditions and related nonlinear conjugate gradient methods. Applied Mathematics and Optimization 43 (2001) 87-101
21. Yabe H., and Takano M. Global convergence properties of nonlinear conjugate gradient methods with modified secant condition. Computational Optimization and Applications 28 (2004) 203-225
22. Zhou W., and Zhang L. A nonlinear conjugate gradient method based on the MBFGS secant condition. Optimization Methods and Software 21 (2006) 707-714
23. Ford J.A., Narushima Y., and Yabe H. Multi-step nonlinear conjugate gradient methods for unconstrained minimization. Computational Optimization and Applications 40 (2008) 191-216
24. Zhang J.Z., Deng N.Y., and Chen L.H. New quasi-Newton equation and related methods for unconstrained optimization. Journal of Optimization Theory and Applications 102 (1999) 147-167
25. Zhang J.Z., and Xu C.X. Properties and numerical performance of quasi-Newton methods with modified quasi-Newton equations. Journal of Computational and Applied Mathematics 137 (2001) 269-278
26. Li D.H., and Fukushima M. A modified BFGS method and its global convergence in nonconvex minimization. Journal of Computational and Applied Mathematics 129 (2001) 15-35
27. Ford J.A., and Moghrabi I.A. Alternative parameter choices for multi-step quasi-Newton methods. Optimization Methods and Software 2 (1993) 357-370
28. Ford J.A., and Moghrabi I.A. Multi-step quasi-Newton methods for optimization. Journal of Computational and Applied Mathematics 50 (1994) 305-323
29. Dai Y.H., Han J.Y., Liu G.H., Sun D.F., Yin H.X., and Yuan Y. Convergence properties of nonlinear conjugate gradient methods. SIAM Journal on Optimization 10 (1999) 345-358
30. Moré J.J., Garbow B.S., and Hillstrom K.E. Testing unconstrained optimization software. ACM Transactions on Mathematical Software 7 (1981) 17-41
31. Moré J.J., and Thuente D.J. Line search algorithms with guaranteed sufficient decrease. ACM Transactions on Mathematical Software 20 (1994) 286-307
32. J. Nocedal, http://www.ece.northwestern.edu/~nocedal/software.html.