Anderson acceleration for fixed-point iterations

SIAM Journal on Numerical Analysis

Volumn 49, Issue 4, 2011, Pages 1715-1735

  Walker, Homer F ^a   Ni, Peng ^a  

^a Life Science and Bioengineering Center   (United States)

Author keywords

Acceleration methods; Alternating least squares; Arnoldi (full orthogonalization) method; Expectation maximization algorithm; Fixed point iterations; Generalized minimal residual method; Iterative methods; Mixture densities; Nonnegative matrix factorization

Indexed keywords

ACCELERATION METHOD; ARNOLDI; EXPECTATION-MAXIMIZATION ALGORITHMS; FIXED-POINT ITERATIONS; GENERALIZED MINIMAL RESIDUAL METHODS; LEAST SQUARE; MIXTURE DENSITIES; NONNEGATIVE MATRIX FACTORIZATION;

ACCELERATION CONTROL; ALGORITHMS; ELECTRONIC STRUCTURE; ITERATIVE METHODS; LEAST SQUARES APPROXIMATIONS; MATRIX ALGEBRA; NUMERICAL METHODS;

ACCELERATION;

EID: 80052747401     PISSN: 00361429     EISSN: None     Source Type: Journal    
DOI: 10.1137/10078356X     Document Type: Article

References (47)

1. D. G. ANDERSON, Iterative procedures for nonlinear integral equations, J. Assoc. Comput. Mach., 12(1965), pp. 547-560.
2. C. BOUTSIDIS AND E. GALLOPOULOS, SVD based initialization: A head start for nonnegative matrix factorization, Pattern Recognition, 41(2008), pp. 1350-1362. (Pubitemid 350212975)
3. C. BREZINSKI, Convergence acceleration during the 20th century, J. Comput. Appl. Math., 122(2000), pp. 1-21. (Pubitemid 32270243)
4. C. BREZINSKI AND M. REDIVO-ZAGLIA, Extrapolation Methods Theory and Practice, North-Holland, Amsterdam, 1991.
5. C. LE BRIS, Computational chemistry from the perspective of numerical analysis, Acta Numer., 14(2005), pp. 363-444. (Pubitemid 40763886)
6. P. N. BROWN, A theoretical comparison of the Arnoldi and GMRES algorithms, SIAM J. Sci. Statist. Comput., 12(1991), pp. 58-78.
7. C. G. BROYDEN, A class of methods for solving nonlinear simultaneous equations, Math. Comp., 19(1965), pp. 577-593.
8. X.-C. CAI AND D. E. KEYES, Nonlinearly preconditioned inexact Newton algorithms, SIAM J. Sci. Comput., 24(2002), pp. 183-200.
9. X.-C. CAI, D. E. KEYES, AND D. P. YOUNG, A nonlinear additive Schwarz preconditioned inexact Newton method for shocked duct flows, in Domain Decomposition Methods in Science and Engineering (Lyon, France, 2000), N. Débit, M. Garbey, R. Hoppe, J. Périaux, D. Keyes, and Y. Kuznetsov, eds., CIMNE, Barcelona, 2000, pp. 345-352.
10. X.-C. CAI AND M. SARKIS, A restricted additive Schwarz preconditioner for general sparse linear systems, SIAM J. Sci. Comput., 21(1999), pp. 792-797.
11. N. N. CARLSON AND K. MILLER, Design and application of a gradient-weighted moving finite element code I: In one dimension, SIAM J. Sci. Comput., 19(1998), pp. 728-765. (Pubitemid 128622531)
12. A. P. DEMPSTER, N. M. LAIRD, AND D. B. RUBIN, Maximum-likelihood from incomplete data via the EM algorithm, J. Royal Statist. Soc. Ser. B, 39(1977), pp. 1-38.
13. J. E. DENNIS, JR., AND R. B. SCHNABEL, Least change secant updates for quasi-Newton methods, SIAM Rev., 21(1979), pp. 443-459.
14. J. E. DENNIS, JR., AND R. B. SCHNABEL, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Classics Appl. Math. 16, SIAM, Philadelphia, 1996.
15. J. E. DENNIS, JR., AND H. F. WALKER, Convergence theorems for least-change secant update methods, SIAM J. Numer. Anal., 18(1981), pp. 949-987.
16. T. EIROLA AND O. NEVANLINNA, Accelerating with rank-one updates, Linear Algebra Appl., 121(1989), pp. 511-520.
17. L. ELDÉN, Matrix Methods in Data Mining and Pattern Recognition, Fundam. Algorithms 4, SIAM, Philadelphia, 2007.
18. V. EYERT, A comparative study on methods for convergence acceleration of iterative vector sequences, J. Comput. Phys., 124(1996), pp. 271-285. (Pubitemid 126170316)
19. H. FANG AND Y. SAAD, Two classes of multisecant methods for nonlinear acceleration, Numer. Linear Algebra Appl., 16(2009), pp. 197-221.
20. M. GARBEY, Acceleration of the Schwarz method for elliptic problems, SIAM J. Sci. Comput., 26(2005), pp. 1871-1893. (Pubitemid 41641354)
21. R. GLOWINSKI, H. B. KELLER, AND L. REINHART, Continuation-conjugate gradient methods for the least squares solution of nonlinear boundary value problems, SIAM J. Sci. Statist. Comput., 6(1985), pp. 793-832.
22. G. H. GOLUB AND C. VAN LOAN, Matrix Computations, 3rd ed., The Johns Hopkins University Press, Baltimore, MD, 1996.
23. K. JBILOU AND H. SADOK, Some results about vector extrapolation methods and related fixedpoint iterations, J. Comput. Appl. Math., 36(1991), pp. 385-398.
24. K. JBILOU AND H. SADOK, Analysis of some vector extrapolation methods for solving systems of linear equations, Numer. Math., 70(1995), pp. 73-89.
25. K. JBILOU AND H. SADOK, Vector extrapolation methods. Applications and numerical comparison, J. Comput. Appl. Math., 122(2000), pp. 149-165. (Pubitemid 32270248)
26. C. T. KELLEY, Iterative Methods for Linear and Nonlinear Equations, Frontiers Appl. Math. 16, SIAM, Philadelphia, 1995.
27. D. A. KNOLL AND D. E. KEYES, Jacobian-free Newton-Krylov methods: A survey of approaches and applications, J. Comput. Phys., 193(2004), pp. 357-397. (Pubitemid 38065123)
28. G. KRESSE AND J. FURTHMÜLLER, Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set, Comput. Mater. Sci., 6(1996), pp. 15-50. (Pubitemid 126412269)
29. G. KRESSE AND J. FURTHMÜLLER, Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set, Phys. Rev. B, 54(1996), pp. 169-186.
30. K. MILLER, Nonlinear Krylov and moving nodes in the method of lines, J. Comput. Appl. Math., 183(2005), pp. 275-287. (Pubitemid 41064383)
31. P. NI, Anderson Acceleration of Fixed-Point Iteration with Applications to Electronic Structure Computations, Ph. D. thesis, Worcester Polytechnic Institute, Worcester, MA, 2009.
32. P. NI AND H. F. WALKER, A Linearly Constrained Least-Squares Problem in Electronic Structure Computations, Tech. Report MS-1-13-46, Mathematical Sciences Department, Worcester Polytechnic Institute, Worcester, MA, 2010.
33. C. W. OOSTERLEE AND T. WASHIO, Krylov subspace acceleration of nonlinear multigrid with application to recirculating flows, SIAM J. Sci. Comput., 21(2000), pp. 1670-1690.
34. M. PERNICE AND H. F. WALKER, NITSOL: A Newton iterative solver for nonlinear systems, SIAM J. Sci. Comput., 19(1998), pp. 302-318. (Pubitemid 128617514)
35. P. PULAY, Convergence acceleration of iterative sequences. The case of SCF iteration, Chem. Phys. Lett., 73(1980), pp. 393-398.
36. P. PULAY, Improved SCF convergence, J. Comput. Chem., 3(1982), pp. 556-560.
37. R. A. REDNER AND H. F. WALKER, Mixture densities, maximum likelihood and the EM algorithm, SIAM Rev., 26(1984), pp. 195-239.
38. Y. SAAD, Krylov subspace methods for solving large unsymmetric linear systems, Math. Comp., 37(1981), pp. 105-126.
39. Y. SAAD, Iterative Methods for Sparse Linear Systems, 2nd ed., SIAM, Philadelphia, 2003.
40. Y. SAAD AND M. H. SCHULTZ, GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems, SIAM J. Sci. Statist. Comput., 7(1986), pp. 856-869.
41. J. SHERMAN AND W. J. MORRISON, Adjustment of an inverse matrix corresponding to changes in the elements of a given column or a given row of the original matrix, Ann. Math. Statistics, 20(1949), p. 621.
42. D. A. SMITH, W. F. FORD, AND A. SIDI, Extrapolation methods for vector sequences, SIAM Rev., 29(1987), pp. 199-233.
43. T. WASHIO AND C. W. OOSTERLEE, Krylov subspace acceleration for nonlinear multigrid schemes, Electron. Trans. Numer. Anal., 6(1997), pp. 271-290. (Pubitemid 38870918)
44. M. A. WOODBURY, Inverting Modified Matrices, Tech. Report Memorandum Report 42, Statistical Research Group, Princeton University, Princeton, NJ, 1950.
45. C. YANG, J. C. MEZA, B. LEE, AND L.-W. WANG, KSSOLV-a MATLAB toolbox for solving the Kohn-Sham equations, ACM Trans. Math. Software, 36(2009), pp. 1-35.
46. U. M. YANG, A Family of Preconditioned Iterative Solvers for Sparse Linear Systems, Ph. D. thesis, Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, IL, 1995.
47. D. P. YOUNG, W. P. HUFFMAN, R. G. MELVIN, C. L. HILMES, AND F. T. JOHNSON, Nonlinear elimination in aerodynamic analysis and design optimization, in Large-Scale PDE-Constrained Optimization, L. T. Biegler, O. Ghattas, M. Heinkenschloss, and B. van Bloemen Waanders, eds., Springer-Verlag, Berlin, Heidelberg, New York, 2003, pp. 17-43.