A review on subspace methods for nonlinear optimization

Proceeding of the International Congress of Mathematicans, ICM 2014

Volumn 4, Issue , 2014, Pages 807-827

  Yuan, Ya Xiang ^a  

^a CHINESE ACADEMY OF SCIENCES   (China)

Author keywords

Nonlinear optimization; Numerical methods; Subproblems; Subspace techniques

Indexed keywords

CONSTRAINED OPTIMIZATION; ITERATIVE METHODS; LEAST SQUARES APPROXIMATIONS; NONLINEAR PROGRAMMING; NUMERICAL METHODS;

DIMENSIONAL SUBSPACE; LARGE-SCALE OPTIMIZATION; MATRIX OPTIMIZATION; NON-LINEAR LEAST SQUARES; NON-LINEAR OPTIMIZATION; NON-LINEAR OPTIMIZATION PROBLEMS; SUBSPACE TECHNIQUES; UNCONSTRAINED OPTIMIZATION;

NONLINEAR EQUATIONS;

EID: 84973258090     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (42)

1. J. Bennett and S. Lanning, The Netflix prize, Proceedings of KDD Cup and Workshop, 2007.
2. D.P. Bertsekas. Nonlinear Programming, Athena Scientific, Belmont, Massachusetts, 1999.
3. M.A. Branch, T.F. Coleman, and Y.Y. Li, A subspace, interior, and conjugate gradient method for large-scale bound-constrained minimization problems, SIAM J. Sci. Comput. 21 (2007), 1-23.
4. P. Brown and Y. Saad, Hybrid Krylov methods for nonlinear systems of equations, SIAM Journal on Scientific and Statistical Computing 11 (1990), 450-481.
5. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers, Foundations and Trends in Machine Learning 3 (2011), 1-122
6. O. Burdakov, L.J. Gong, Y.X. Yuan, and S. Zikrin, On effciently combining limited memory and trust-region techniques, Report, AMSS, CAS, 2013.
7. Celis, M.R., Dennis, J.E., and Tapia, R.A., A trust region algorithm for nonlinear equality constrained optimization, in: P.T. Boggs, R.H. Byrd, and R.B. Schnabel, eds., Numerical Optimization (SIAM, Philadelphia, 1985), 71-82.
8. T. Coleman and Y.Y. Li, An interior point trust region approach for nonlinear minimization subject to bounds, SIAM J. Optimization 6 (1996), 418-445.
9. A. Conn, N. Gould, A. Sartenaer, and Ph. Toint, On iterated-subspace methods for nonlinear optimization, in: J. Adams and J.L. Nazareth, eds., Linear and Nonlinear Conjugate Gradient-Related Methods (1996), 50-79.
10. Y. H. Dai and Y.X. Yuan, Nonlinear Conjugate Gradient Methods (in Chinese), Shanghai Science and Technology Publisher, Shanghai, 2000.
11. M. Elad, B. Matalon, and M. Zibulevsky, Coordinate and subspace optimization methods for linear least squares with non-quadratic regularization, Appl. Comput. Harmon. Anal. 23 (2007) 346-367
12. R. Fletcher, Second order correction for nondifferentiable optimization, in: G.A. Watson, ed., Numerical Analysis, Springer-Verlag, Berlin, (1982), 85-115.
13. P.E. Gill and M.W. Leonard, Reduced-Hessian quasi-Newton methods for unconstrained optimization, SIAM J. Optim. 1 (2001), 209-237.
14. R. Glowinski and A. Marrocco, Sur l'approximation, par éléements finis d'ordre un, et la résolution, par pénalisation-dualité, d'une classe de problèmes de Dirichlet non linéaires, Laboria, 1975.
15. M. Fortin and R. Glowinski, Augmented Lagrangian Methods, Vol. 15 of Studies in Mathematics and its Applications, North-Holland Publishing Co., Amsterdam, 1983. Applications to the numerical solution of boundary value problems, Translated from the French by B. Hunt and D. C. Spicer.
16. R. Glowinski and P. Le Tallec, Augmented Lagrangian and Operator-splitting Methods in Nonlinear Mechanics, Vol. 9 of SIAM Studies in Applied Mathematics, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1989.
17. N.I.M. Gould, D. Orban, and Ph.L. Toint, Numerical methods for large-scale nonlinear optimization, Acta Numerica (2005), 299-361.
18. Grapiglia, G.N., Yuan, J.Y., and Yuan, Y.X., A subspace version of the Powell-Yuan trust-region algorithm for equality constrained optimization, J. Operations Research Society of China 1 (2013), 425- 451.
19. A. V. Knyazev, Toward the optimal preconditioned eigensolver: Locally optimal block preconditioned conjugate gradient method, SIAM J. Sci. Comput. 23 (2001), 517-541.
20. Lee, J.H., A Subspace Algorithm for Nonlinear Equality Constrained Optimization, Ph.D. thesis, ICMSEC, AMSS, Chinese Academy of Science, Beijing, 2009.
21. D.C. Liu and J. Nocedal, On the limited memory BFGS method for large scale optimization, Mathematical Programming 45 (1989), 503-528.
22. X. Liu, X. Wang, Z.W. Wen, and Y.X. Yuan, On the convergence of the self-consistent field iteration in Kohn-Sham density function theory, accepted by SIAM Journal on Matrix Analysis and Applications (2014).
23. X. Liu, Z.W. Wen and Y. Zhang, Limited memory block Krylov subspace optimization for computing dominant singular value decompositions, SIAM Journal on Scientific Computing 35 (2013), A1641-A1668
24. Y.Y. Li and S. Osher, Coordinate descent optimization for '1 minimization with application to compressed sensing; a greedy algorithm, Inverse Probl. Imaging 3 (2009), 487-503
25. Z. Q. Luo and P. Tseng, On the convergence of the coordinate descent method for convex differentiable minimization, J. Optim. Theory Appl. 72 (1992), 7-35
26. D.Q. Mayne and E. Polak, A superlinearly convergent algorithm for constrained optimization problems, Math. Prog. Study 16 (1982), 45-61.
27. Q. Ni and Y.X. Yuan, A subspace limited memory quasi-Newton algorithm for largescale nonlinear bound constrained optimization, Mathematics of Computations 66 (1997), 1509-1520.
28. M.J.D. Powell and Y.X. Yuan, A trust region algorithm for equality constrained optimization, Mathematical Programming 49 (1991), 189-211.
29. Y. Saad, Iterative Methods for Sparse Linear Systems: 2nd Ed., SIAM, Philedalphia, 2003.
30. A. Sit, Z.J. Wu, and Y.X. Yuan, A geometric buildup algorithm for the solution of the distance geometry problem using least-squares approximation, Bulletin of Mathematical Biology 71 (2009), 1914-1933.
31. J. Stoer and Y.X. Yuan, A subspace study on conjugate gradient algorithms, ZAMM Z. angew. Math. Mech. 75 (1995), 69-77.
32. W.Y. Sun and Y.X. Yuan, Optimization Theory and Methods: Nonlinear Programming, Springer Verlag, New York, 2006.
33. Z.H. Wang, Z.W. Wen, and Y.X. Yuan, A subspace trust region method for large scale unconstrained optimization, in: Y.X. Yuan eds. Numerical Linear Algebra and Optimization (Science Press, Beijing/NewYork, 2004), pp. 265-274.
34. Z.H. Wang and Y.X. Yuan, A subspace implementation of quasi-Newton trust region methods for unconstrained optimization, Numerische Mathematik 104 (2006), 241-269.
35. Z.W. Wen, D. Goldfarb, and K. Scheinberg, Block coordinate descent methods for semidefinite programming, in: Handbook on Semidefinite, Conic and Polynomial Optimization International Series in Operations Research & Management Science 166 (2012), 533-564.
36. Z.W. Wen, A. Milzarek, M. Ulbrich, and H.C. Zhang, Adaptive regularized selfconsistent field iteration with exact Hessian for electronic structure calculation, SIAM Journal on Scientific Computing 35 (2013), pp. A1299-A1324.
37. Z.W. Wen, D. Goldfarb, and W. Yin, Alternating direction augmented Lagrangian methods for semidefinite programming, Mathematical Programming Computation 2 (2010), 203-230
38. Z.W. Wen, W. Yin, D. Goldfarb, and Y. Zhang, A fast algorithm for sparse reconstruction based on shrinkage, subspace optimization, and continuation, SIAM Journal on Scientific Computing 32 (2010), 1832-1857.
39. X.C. Tai and J.C. Xu, Global and uniform convergence of subspace correction methods for some convex optimization problems, Math. Comp. 71 (2002), 105-124.
40. J. Yang and Y. Zhang, Alternating direction algorithms for l1-problems in compressive sensing, SIAM journal on scientific computing 33 (2011), 250-278.
41. Y.X. Yuan, Subspace techniques for nonlinear optimization, in: R. Jeltsch, D.Q. Li and I. H. Sloan, eds., Some Topics in Industrial and Applied Mathematics (Series in Contemporary Applied Mathematics CAM 8) (Higher Education Press. Beijing, 2007), pp. 206-218.
42. Subspace methods for large scale nonlinear equations and nonlinear least squares, Optimization and Engineering 10 (2009), 207-218.