Combining trust-region techniques and rosenbrock methods to compute stationary points

Journal of Optimization Theory and Applications

Volumn 140, Issue 2, 2009, Pages 265-286

  Luo, X L ^a,b   Kelley, C T ^c   Liao, L Z ^d   Tam, H W ^d  

^a BEIJING UNIVERSITY OF POSTS AND TELECOMMUNICATIONS   (China)

^b Ministry of Information Industry   (China)

^c NORTH CAROLINA STATE UNIVERSITY   (United States)

^d HONG KONG BAPTIST UNIVERSITY   (Hong Kong)

Author keywords

Gradient system; Ordinary differential equations; Rosenbrock method; Trust region methods; Unconstrained optimization

Indexed keywords

ITERATIVE METHODS; NONLINEAR EQUATIONS; OPTIMIZATION; ORDINARY DIFFERENTIAL EQUATIONS; RUNGE KUTTA METHODS;

BACKWARD DIFFERENCE FORMULA; GRADIENT SYSTEMS; IMPLICIT METHODS; INITIAL VALUE PROBLEMS OF ORDINARY DIFFERENTIAL EQUATIONS; ROSENBROCK METHODS; STATIONARY POINTS; STIFF INITIAL VALUE PROBLEMS; TRUST-REGION METHODS; TRUST-REGION TECHNIQUES; UNCONSTRAINED OPTIMIZATION;

INITIAL VALUE PROBLEMS;

EID: 59349091447     PISSN: 00223239     EISSN: 15732878     Source Type: Journal    
DOI: 10.1007/s10957-008-9469-0     Document Type: Article

References (38)

1. C.A. Botsaris 1978 A class of methods for unconstrained minimization based on stable numerical integration techniques J. Math. Anal. Appl. 63 729 749
2. A.A. Brown M.C. Bartholomew-Biggs 1989 Some effective methods for unconstrained optimization based on the solution of systems of ordinary differential equations J. Optim. Theory Appl. 62 211 224
3. Evtushenko, Yu.G.: Numerical Optimization Techniques. Optimization Software Publications, New York (1985)
4. X.L. Luo L.-Z. Liao H.W. Tam 2007 Convergence analysis of the Levenberg-Marquardt method Optim. Methods Softw. 22 659 678
5. J. Schropp 1995 Using dynamical systems methods to solving minimization problems Appl. Numer. Math. 18 321 335
6. Q.M. Han L.-Z. Liao H.D. Qi L.Q. Qi 2001 Stability analysis of gradient-based neural networks for optimization problems J. Glob. Optim. 19 363 381
7. L.-Z. Liao H.D. Qi L.Q. Qi 2004 Neurodynamical optimization J. Glob. Optim. 28 175 195
8. Helmke, U., Moore, J.B.: Optimization and Dynamical Systems. Springer, Berlin (1994)
9. Łojasiewicz, S.: Sur les trajectoires de gradient d'une fonction analytique. Seminari di Geometria 1982-1983 (Lecture notes), pp. 115-117. Dipartemento di Matematica, Universita di Bologna (1984)
10. P.-A. Absil R. Mahony B. Andrews 2005 Convergence of the iterates of descent methods for analytic cost functions SIAM J. Optim. 16 2 531 547
11. Lageman, C.: Konvergenz reell-analytischer gradientenähnlicher Systeme. Diplomarbeit im Fach Mathematik, Mathematisches Institut, Universität Würzburg, Germany (2002)
12. Grantham, W.J.: Trajectory following optimization by gradient transformation differential equations. In: Proceedings of the 42nd IEEE Conference on Decision and Control, Maui, HI, 9-12 December 2003, pp. 5496-5501
13. Hairer, E., Nørsett, S.P., Wanner, G.: Solving Ordinary Differential Equations I: Nonstiff Problems, 2nd edn. Springer, Berlin (1993)
14. Hairer, E., Wanner, G.: Solving Ordinary Differential Equations II: Stiff and Differential-Algebraic Problems, 2nd edn. Springer, Berlin (1996)
15. A.R. Humphries A.M. Stuart 1994 Runge-Kutta methods for dissipative and gradient dynamical systems SIAM J. Numer. Anal. 31 1452 1485
16. A.M. Stuart A.R. Humphries 1995 The essential stability of local error control for dynamical systems SIAM J. Numer. Anal. 32 1940 1971
17. Conn, A.R., Gould, N., Toint, Ph.L.: Trust-Region Methods. SIAM, Philadelphia (2000)
18. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, Berlin (1999)
19. T.S. Coffey C.T. Kelley D.E. Keyes 2003 Pseudotransient continuation and differential-algebraic equations SIAM J. Sci. Comput. 25 553 569
20. K.R. Fowler C.T. Kelley 2005 Pseudo-transient continuation for nonsmooth nonlinear equations SIAM J. Numer. Anal. 43 1385 1406
21. H.H. Rosenbrock 1963 Some general implicit processes for the numerical solution of differential equations Comput. J. 5 329 331
22. L.F. Shampine 1982 Implementation of Rosenbrock methods ACM Trans. Math. Softw. 8 93 113
23. L.F. Shampine M.W. Reichelt 1997 The Matlab ODE suite SIAM J. Sci. Comput. 18 1 22
24. C.T. Kelley L.-Z. Liao L.Q. Qi M.T. Chu J.P. Reese C. Winton 2008 Projected pseudotransient continuation SIAM J. Numer. Anal. 46 3071 3083
25. D.J. Higham 1999 Trust region algorithms and timestep selection SIAM J. Numer. Anal. 37 194 210
26. X.L. Luo 2005 Singly diagonally implicit Runge-Kutta methods combining line search techniques for unconstrained optimization J. Comput. Math. 23 153 164
27. C.T. Kelley D.E. Keyes 1998 Convergence analysis of pseudo-transient continuation SIAM J. Numer. Anal. 35 508 523
28. Deuflhard, P.: Newton Methods for Nonlinear Problems: Affine Invariance and Adaptive Algorithms. Springer, Berlin (2004)
29. Fletcher, R.: Practical Methods of Optimization. Wiley, New York (1987)
30. K. Levenberg 1944 A method for the solution of certain problems in least squares Q. Appl. Math. 2 164 168
31. D. Marquardt 1963 An algorithm for least-squares estimation of nonlinear parameters SIAM J. Appl. Math. 11 431 441
32. Yuan, Z.D., Fei, J.G., Liu, D.G.: The Numerical Solutions for Initial Value Problems of Stiff Ordinary Differential Equations. Science Press, Beijing (1987) (in Chinese)
33. Luo, X.L.: Unconstrained optimization methods based on dynamic systems (in Chinese). Ph. D. thesis, Institute of Computational Mathematics and Scientific Engineering/Computing, Chinese Academy of Sciences (2001) (in Chinese)
34. X.L. Luo 2003 Convergence analysis of methods based on BDF for unconstrained optimization Math. Numer. Sinica 25 177 184 (in Chinese)
35. Powell, M.J.D.: Convergence properties of a class of minimization algorithms. In: Mangasarian, O.L., Meyer, R.R., Robinson, S.M. (eds.) Nonlinear Programming, vol. 2, pp. 1-27. Academic Press, New York (1975)
36. J.E. Dennis J.J. Moré 1974 A characterization of superlinear convergence and its application to quasi-Newton methods Math. Comput. 28 549 560
37. J.J. Moré B.S. Garbow K.E. Hillstrom 1981 Testing unconstrained optimization software ACM Trans. Math. Softw. 7 17 41
38. MATLAB 6.5: The MathWorks Inc., http://www.mathworks.com (2003)