Accelerated line-search and trust-region methods

SIAM Journal on Numerical Analysis

Volumn 47, Issue 2, 2009, Pages 997-1018

  Absil, P A ^a   Gallivan, K A ^b  

^a UNIVERSITÉ CATHOLIQUE DE LOUVAIN   (Belgium)

^b FLORIDA STATE UNIVERSITY   (United States)

Author keywords

Arnoldi; Jacobi Davidson; Line search; Locally optimal block preconditioned conjugate gradient (LOBPCG); Optimization on manifolds; Riemannian manifold; Riemannian optimization; Sequential subspace method; Subspace acceleration; Trust region

Indexed keywords

ARNOLDI; JACOBI-DAVIDSON; LINE SEARCHES; OPTIMIZATION ON MANIFOLDS; PRECONDITIONED CONJUGATE GRADIENT; RIEMANNIAN MANIFOLD; SUB-SPACE METHODS; TRUST REGION;

OPTIMIZATION;

CONVERGENCE OF NUMERICAL METHODS;

EID: 84856231226     PISSN: 00361429     EISSN: None     Source Type: Journal    
DOI: 10.1137/08072019X     Document Type: Article

References (41)

1. P.-A. ABSIL, C. G. BAKER, AND K. A. GALLIVAN, A truncated-CG style method for symmetric generalized eigenvalue problems, J. Comput. Appl. Math., 189(2006), pp. 274-285.
2. P.-A. ABSIL, C. G. BAKER, AND K. A. GALLIVAN, Trust-region methods on Riemannian manifolds, Found. Comput. Math., 7(2007), pp. 303-330.
3. R. L. ADLER, J.-P. DEDIEU, J. Y. MARGULIES, M. MARTENS, AND M. SHUB, Newton's method on Riemannian manifolds and a geometric model for the human spine, IMA J. Numer. Anal., 22(2002), pp. 359-390.
4. P.-A. ABSIL AND K. A. GALLIVAN, Accelerated Line-search and Trust-region Methods, Technical report FSU-SCS-2005-095, School of Computational Science, Florida State University, Tallahassee, FL, 2005.
5. P. ARBENZ, U. L. HETMANIUK, R. B. LEHOUCQ, AND R. S. TUMINARO, A comparison of eigensolvers for large-scale 3D modal analysis using AMG-preconditioned iterative methods, Internat. J. Numer. Methods Engrg., 64(2005), pp. 204-236.
6. P.-A. ABSIL, R. MAHONY, AND R. SEPULCHRE, Optimization Algorithms on Matrix Manifolds, Princeton University Press, Princeton, NJ, 2008.
7. L. ARMIJO, Minimization of functions having Lipschitz continuous first partial derivatives, Pacific J. Math., 16(1966), pp. 1-3.
8. C. G. BAKER, P.-A. ABSIL, AND K. A. GALLIVAN, An implicit trust-region method on Riemannian manifolds, IMA J. Numer. Anal., to appear.
9. C. G. BAKER, Riemannian Manifold Trust-region Methods with Applications to Eigenproblems, Ph. D. thesis, School of Computational Science, Florida State University, Tallahassee, FL, 2008.
10. D. P. BERTSEKAS, Nonlinear Programming, Athena Scientific, Belmont, MA, 1995.
11. A. R. CONN, N. I. M. GOULD, AND P. L. TOINT, Trust-Region Methods, MPS/SIAM Ser. Optim. 1, SIAM, Philadelphia, 2000.
12. E. R. DAVIDSON, The iterative calculation of a few of the lowest eigenvalues and corresponding eigenvectors of large real-symmetric matrices, J. Comput. Phys., 17(1975), pp. 87-94.
13. M. P. DO CARMO, Riemannian geometry, Math. Theory Appl., Birkhäuser Boston, Boston, MA, 1992. Translated from the second Portuguese edition by Francis Flaherty.
14. A. EDELMAN, T. A. ARIAS, AND S. T. SMITH, The geometry of algorithms with orthogonality constraints, SIAM J. Matrix Anal. Appl., 20(1998), pp. 303-353.
15. D. R. FOKKEMA, G. L. G. SLEIJPEN, AND H. A. VAN der VORST, Accelerated inexact Newton schemes for large systems of nonlinear equations, SIAM J. Sci. Comput., 19(1998), pp. 657-674.
16. D. R. FOKKEMA, G. L. G. SLEIJPEN, AND H. A. VAN der VORST, Jacobi-Davidson style QR and QZ algorithms for the reduction of matrix pencils, SIAM J. Sci. Comput., 20(1998), pp. 94-125.
17. D. GABAY, Minimizing a differentiable function over a differential manifold, J. Optim. Theory Appl., 37(1982), pp. 177-219.
18. G. H. GOLUB AND C. F. VAN LOAN, Matrix Computations, 3rd ed., Johns Hopkins S. Math. Sci., Johns Hopkins University Press, Baltimore, MD, 1996.
19. G. H. GOLUB AND Q. YE, An inverse free preconditioned Krylov subspace method for symmetric generalized eigenvalue problems, SIAM J. Sci. Comput., 24(2002), pp. 312-334.
20. W. W. HAGER, Minimizing a quadratic over a sphere, SIAM J. Optim., 12(2001), pp. 188-208.
21. U. HETMANIUK AND R. LEHOUCQ, Basis selection in LOBPCG, J. Comput. Phys., 218(2006), pp. 324-332.
22. U. HELMKE AND J. B. MOORE, Optimization and Dynamical Systems, Comm. Control Engrg. Ser., Springer-Verlag, London, 1994.
23. M. E. HOCHSTENBACH AND Y. NOTAY, Controlling Inner Iterations in the Jacobi-Davidson Method, SIAM J. Matrix Anal. Appl., to appear.
24. M. E. HOCHSTENBACH AND Y. NOTAY, The Jacobi-Davidson method, GAMM Mitt. Ges. Angew. Math. Mech., 29(2006), pp. 368-382.
25. W. W. HAGER AND S. PARK, Global convergence of SSM for minimizing a quadratic over a sphere, Math. Comp., 74(2005), pp. 1413-1423.
26. A. V. KNYAZEV, Toward the optimal preconditioned eigensolver: Locally optimal block preconditioned conjugate gradient method, SIAM J. Sci. Comput., 23(2001), pp. 517-541.
27. I. LASHUK, M. ARGENTI, E. OVTCHINNIKOV, AND A. KNYAZEV, Preconditioned eigensolver LOBPCG in Hypre and PETSc, in Domain Decomposition Methods in Science and Engineering XVI, Lect. Notes Comput. Sci. Eng. 55, Springer-Verlag, Berlin, 2007.
28. Y. NOTAY, Combination of Jacobi-Davidson and conjugate gradients for the partial symmetric eigenproblem, Numer. Linear Algebra Appl., 9(2004), pp. 21-44.
29. Y. NOTAY, Is Jacobi-Davidson faster than Davidson?, SIAM J. Matrix Anal. Appl., 26(2004), pp. 522-543.
30. J. NOCEDAL AND S. J. WRIGHT, Numerical Optimization, Springer Ser. Oper. Res., Springer-Verlag, New York, 1999.
31. G. NARKISS AND M. ZIBULEVSKY, Sequential Subspace Optimization Method for Large-Scale Unconstrained Problems, Technical report CCIT 559, EE Dept., Technion, Haifa, Israel, 2005.
32. M. SHUB, Some remarks on dynamical systems and numerical analysis, in Dynamical Systems and Partial Differential Equations, Proceedings of the VII ELAM, L. Lara-Carrero and J. Lewowicz, eds., Equinoccio, Universidad Simón Bolívar, Caracas, 1986, pp. 69-91.
33. S. T. SMITH, Geometric Optimization Methods for Adaptive Filtering, Ph. D. thesis, Division of Applied Sciences, Harvard University, Cambridge, MA, 1993.
34. S. T. SMITH, Optimization techniques on Riemannian manifolds, in Hamiltonian and Gradient Flows, Algorithms and Control, Fields Inst. Commun. 3, American Mathematical Society, Providence, RI, 1994, pp. 113-136.
35. A. STATHOPOULOS AND Y. SAAD, Restarting techniques for the (Jacobi-) Davidson symmetric eigenvalue methods, Electron. Trans. Numer. Anal., 7(1998), pp. 163-181.
36. A. STATHOPOULOS, Nearly optimal preconditioned methods for Hermitian eigenproblems under limited memory. Part I: Seeking one eigenvalue, SIAM J. Sci. Comput., 29(2007), pp. 481-514.
37. T. STEIHAUG, The conjugate gradient method and trust regions in large scale optimization, SIAM J. Numer. Anal., 20(1983), pp. 626-637.
38. G. L. G. SLEIJPEN AND H. A. VAN der VORST, A Jacobi-Davidson iteration method for linear eigenvalue problems, SIAM J. Matrix Anal. Appl., 17(1996), pp. 401-425.
39. I. TAKAHASHI, A note on the conjugate gradient method, Inform. Process. Japan, 5(1965), pp. 45-49.
40. C. UDRIŞTE, Convex Functions and Optimization Methods on Riemannian Manifolds, Math. Appl. 297, Kluwer Academic, Dordrecht, the Netherlands, 1994.
41. Y. YANG, Globally convergent optimization algorithms on Riemannian manifolds: Uniform framework for unconstrained and constrained optimization, J. Optim. Theory Appl., 132(2007), pp. 245-265.