Accelerated linearized Bregman method

Journal of Scientific Computing

Volumn 54, Issue 2-3, 2013, Pages 428-453

  Huang, Bo ^a   Ma, Shiqian ^b   Goldfarb, Donald ^a  

^a Department of Earth and Environmental Sciences   (United States)

^b UNIVERSITY OF MINNESOTA   (United States)

Author keywords

Accelerated linearized Bregman method; Basis pursuit; Compressed sensing; Convex optimization; Linearized Bregman method; Matrix completion

Indexed keywords

BASIS PURSUITS; BREGMAN METHODS; COMPUTATIONAL EFFORT; GRADIENT DESCENT METHOD; MATRIX COMPLETION; MATRIX COMPLETION PROBLEMS; NUMERICAL RESULTS; SPARSE OPTIMIZATIONS;

ALGORITHMS; COMPRESSED SENSING; CONVEX OPTIMIZATION; LINEARIZATION; NUMERICAL METHODS; OPTIMIZATION; SIGNAL RECONSTRUCTION;

ITERATIVE METHODS;

EID: 84875960294     PISSN: 08857474     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10915-012-9592-9     Document Type: Article

References (46)

1. Barzilai, J., Borwein, J.: Two point step size gradient methods. IMA J. Numer. Anal. 8, 141-148 (1988)
2. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2, 183-202 (2009)
3. Becker, S., Candès, E. J., Grant, M.: Templates for convex cone problems with applications to sparse signal recovery. Math. Program. Comput. 3, 165-218 (2011)
4. Bertsekas, D. P., Tsitsiklis, J. N.: Parallel and Distributed Computation: Numerical Methods. Prentice-Hall, Upper Saddle River (1989)
5. Bregman, L.: The relaxation method of finding the common points of convex sets and its application to the solution of problems in convex programming. U. S. S. R. Comput. Math. Math. Phys. 7, 200-217 (1967)
6. Cai, J., Candès, E. J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20, 1956-1982 (2010)
7. Cai, J.-F., Osher, S., Shen, Z.: Linearized Bregman iterations for compressed sensing. Math. Comput. 78, 1515-1536 (2009)
8. Cai, J.-F., Osher, S., Shen, Z.: Convergence of the linearized Bregman iteration for 1-norm minimization. Math. Comput. 78, 2127-2136 (2009)
9. Candès, E. J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9, 717-772 (2009)
10. Candès, E. J., Tao, T: The power of convex relaxation: near-optimal matrix completion. IEEE Trans. Inf. Theory 56, 2053-2080 (2009)
11. Candès, E. J., Romberg, J., Tao, T: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52, 489-509 (2006)
12. Donoho, D.: Compressed sensing. IEEE Trans. Inf. Theory 52, 1289-1306 (2006)
13. Friedlander, M. P, Tseng, P.: Exact regularization of convex programs. SIAM J. Optim. 18, 1326-1350 (2007)
14. Goldfarb, D., Ma, S.: Fast multiple splitting algorithms for convex optimization. Tech. report, Department of IEOR, Columbia University (2009). Preprint, available at http://arxiv.org/abs/0912.4570
15. Goldfarb, D., Ma, S.: Convergence of fixed point continuation algorithms for matrix rank minimization. Found. Comput. Math. 11, 183-210 (2011)
16. Goldfarb, D., Ma, S., Scheinberg, K.: Fast alternating linearization methods for minimizing the sum of two convex functions. Math. Program. Ser. A (2012). doi:10.1007/s10107-012-0530-2
17. Goldfarb, D., Scheinberg, K.: Fast first-order methods for composite convex optimization with line search. Technical Report, Department of IEOR, Columbia University (2011)
18. Gross, D.: Recovering low-rank matrices from few coefficients in any basis. IEEE Trans. Inf. Theory 57, 1548-1566 (2011)
19. Hale, E. T., Yin, W., Zhang, Y.: Fixed-point continuation for l1-minimization: methodology and convergence. SIAM J. Optim. 19, 1107-1130 (2008)
20. Hestenes, M. R.: Multiplier and gradient methods. J. Optim. Theory Appl. 4, 303-320 (1969)
21. Liu, D. C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Math. Program., Ser. B 45, 503-528 (1989)
22. Liu, Y, Sun, D., Toh, K.-C: An implementable proximal point algorithmic framework for nuclear norm minimization. Math. Program. (2011). doi:10.1007/s10107-010-0437-8
23. Ma, S., Goldfarb, D., Chen, L.: Fixed point and Bregman iterative methods for matrix rank minimization. Math. Program., Ser. A 128, 321-353 (2011)
24. Mangasarian, O. L., Meyer, R. R.: Nonlinear perturbation of linear programs. SIAM J. Control Optim. 17, 745-752 (1979)
25. Natarajan, B. K.: Sparse approximate solutions to linear systems. SIAM J. Comput. 24, 227-234 (1995)
26. Nemirovski, A.: Prox-method with rate of convergence O (1/t) for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM J. Optim. 15, 229-251 (2005)
27. Nesterov, Y. E.: A method for unconstrained convex minimization problem with the rate of convergence 0 (1/k). Dokl. Akad. Nauk SSSR 269, 543-547 (1983)
28. Nesterov, Y. E.: In: Introductory Lectures on Convex Optimization, vol. 87, p. xviii+236 (2004). A basic course
29. Nesterov, Y. E.: Smooth minimization for non-smooth functions. Math. Program., Ser. A 103, 127-152 (2005)
30. Nesterov, YE.: Gradient methods for minimizing composite objective function. CORE Discussion Paper 2007/76 (2007)
31. Osher, S., Burger, M., Goldfarb, D., Xu, J., Yin, W.: An iterative regularization method for total variationbased image restoration. Multiscale Model. Simul. 4, 460-489 (2005)
32. Osher, S., Mao, Y, Dong, B., Yin, W.: Fast linearized Bregman iteration for compressive sensing and sparse denoising. Commun. Math. Sci. 8, 93-111 (2010)
33. Powell, M. J. D.: A method for nonlinear constraints in minimization problems. In: Fletcher, R. (ed.) Optimization, pp. 283-298. Academic Press, New York (1972)
34. Recht, B.: A simpler approach to matrix completion. J. Mach. Learn. Res. 12, 3413-3430 (2011)
35. Recht, B., Fazel, M., Parrilo, P.: Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev. 52, 471-501 (2010)
36. Rockafellar, R. T.: Convex Analysis. Princeton University Press, Princeton (1970)
37. Rockafellar, R. T.: Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res. 1, 97-116 (1976)
38. ACM SIGKDD and Netflix: In: Proceedings of kdd Cup and Workshop, Proceedings available online at http://www.cs.uic.edu/~liub/KDD-cup-2007/ proceedings.html (2012)
39. Srebro, N.: Learning with matrix factorizations. PhD thesis, Massachusetts Institute of Technology (2004)
40. Srebro, N., Jaakkola, T: Weighted low-rank approximations. In: Proceedings of the Twentieth International Conference on Machine Learning (ICML-2003) (2003)
41. Toh, K.-C, Yun, S.: An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems. Pac. J. Optim. 6, 615-640 (2010)
42. Tseng, P.: On accelerated proximal gradient methods for convex-concave optimization. SIAM J. Optim. (2008, submitted)
43. Van Den Berg, E., Friedlander, M. P: Probing the Pareto frontier for basis pursuit solutions. SIAM J. Sci. Comput. 31, 890-912 (2008)
44. Wen, Z., Yin, W, Zhang, Y: Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm. Technical Report, Department of CAAM, Rice University (2010)
45. Yin, W: Analysis and generalizations of the linearized Bregman method. SIAM J. Imaging Sci. 3, 856-877 (2010)
46. Yin, W, Osher, S., Goldfarb, D., Darbon, J.: Bregman iterative algorithms for I1-minimization with applications to compressed sensing. SIAM J. Imaging Sci. 1, 143-168 (2008)