Analysis and generalizations of the linearized Bregman method

SIAM Journal on Imaging Sciences

Volumn 3, Issue 4, 2010, Pages 856-877

  Yin, Wotao ^a  

^a Rice University   (United States)

Author keywords

Basis pursuit; Bregman; Compressed sensing; L1 minimization; Linearized bregman

Indexed keywords

BASIS PURSUITS; BREGMAN; COMPRESSED SENSING; L1-MINIMIZATION; LINEARIZED BREGMAN;

CONJUGATE GRADIENT METHOD; OPTIMIZATION; SIGNAL RECONSTRUCTION;

LINEARIZATION;

EID: 78651547900     PISSN: None     EISSN: 19364954     Source Type: Journal    
DOI: 10.1137/090760350     Document Type: Article

References (70)

1. K. J. Arrow, L. Hurwicz, and H. Uzawa, Studies in Nonlinear Programming, Stanford University Press, Stanford, CA, 1958.
2. J. Barzilai and J. Borwein, Two point step size gradient methods, IMA J. Numer. Anal., 8 (1988), pp.141-148.
3. S. Becker, J. Bobin, and E. Candès, NESTA: A Fast and Accurate First-Order Method for Sparse Recovery, Technical report, California Institute of Technology, Pasadena, CA, 2009.
4. D. P. Bertsekas, Constrained Optimization and Lagrange Multiplier Methods, Athena Scientific, Belmont, MA, 1996.
5. J. M. Bioucas-Dias and M. A. Figueiredo, A new TwIST: Two-step iterative shrinkage/thresholding algorithms for image restoration, IEEE Trans. Image Process., 16 (2007), pp. 2992-3004.
6. P. Boufounos, M. Duarte, and R. Baraniuk, Sparse signal reconstruction from noisy compressive measurements using cross validation, in Proceedings of the IEEE Workshop on Statistical Signal Processing, 2007, pp. 299-303.
7. L. Bregman, The relaxation method of finding the common points of convex sets and its application to the solution of problems in convex programming, USSR Comput. Math. Math. Phys., 7 (1967), pp. 200-217.
8. M. Burger, G. Gilboa, S. Osher, and J. Xu, Nonlinear inverse scale space methods, Commun. Math. Sci., 4 (2006), pp. 175-208.
9. M. Burger, S. Osher, J. Xu, and G. Gilboa, Nonlinear inverse scale space methods for image restoration, in Variational, Geometric, and Level Set Methods in Computer Vision, Lecture Notes in Comput. Sci. 3752, Springer-Verlag, Berlin, 2005, pp. 25-36.
10. J.-F. Cai, E. J. Candès, and Z. Shen, A singular value thresholding algorithm for matrix completion, SIAM J. Optim., 20 (2010), pp. 1956-1982.
11. J.-F. Cai, H. Ji, C. Liu, and Z. Shen, Blind motion deblurring from a single image using sparse approximation, in Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2009, pp. 104-111
12. J.-F. Cai, H. Ji, C. Liu, and Z. Shen, Blind motion deblurring using multiple images, J. Comput. Phys., 228 (2009), pp. 5057-5071.
13. J.-F. Cai, S. Osher, and Z. Shen, Linearized Bregman iterations for compressed sensing, Math. Comp., 78 (2009), pp. 1515-1536.
14. J.-F. Cai, S. Osher, and Z. Shen, Convergence of the linearized Bregman iteration for _1-norm minimization, Math. Comp., 78 (2009), pp. 2127-2136.
15. J.-F. Cai, S. Osher, and Z. Shen, Linearized Bregman iterations for frame-based image deblurring, SIAM J. Imaging Sci., 2 (2009), pp. 226-252.
16. E. Candès, J. Romberg, and T. Tao, Stable signal recovery from incomplete and inaccurate information, Comm. Pure Appl. Math., 59 (2006), pp. 1207-1233.
17. A. Chambolle, Total variation minimization and a class of binary MRF models, in Energy Minimization Methods in Computer Vision and Pattern Recognition, Lecture Notes in Comput. Sci. 3757, Springer- Verlag, Berlin, 2005, pp. 136-152.
18. P. L. Combettes and J.-C. Pesquet, Proximal thresholding algorithm for minimization over orthonormal bases, SIAM J. Optim., 18 (2007), pp. 1351-1376.
19. Y. H. Dai and R. Fletcher, Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming, Numer. Math., 100 (2005), pp. 21-47.
20. J. Darbon and M. Sigelle, Image restoration with discrete constrained total variation. I. Fast and exact optimization, J. Math. Imaging Vision, 26 (2006), pp. 261-276.
21. I. Daubechies, M. Defrise, and C. De Mol, An iterative thresholding algorithm for linear inverse problems with a sparsity constraint, Comm. Pure Appl. Math., 57 (2004), pp. 1413-1457.
22. C. De Mol and M. Defrise, A note on wavelet-based inversion algorithms, in Inverse Problems, Image Analysis, and Medical Imaging, Contemp. Math. 313, AMS, Providence, RI, 2002, pp. 85-96.
23. D. Donoho, Compressed sensing, IEEE Trans. Inform. Theory, 52 (2006), pp. 1289-1306.
24. M. Elad, Why simple shrinkage is still relevant for redundant representations?, IEEE Trans. Inform. Theory, 52 (2006), pp. 5559-5569.
25. M. Elad and A. Bruckstein, A generalized uncertainty principle and sparse representations in pairs of bases, IEEE Trans. Inform. Theory, 48 (2002), pp. 2558-2567.
26. E. Esser, Applications of Lagrangian-based Alternating Direction Methods and Connections to Split Bregman, CAM Report 09-31, UCLA, Los Angeles, CA, 2009.
27. M. C. Ferris and O. L. Mangasarian, Finite perturbation of convex programs, Appl. Math. Optim., 23 (1991), pp. 263-273.
28. A. Feuer and A. Nemirovski, On sparse representation in pairs of bases, IEEE Trans. Inform. Theory, 49 (2003), pp. 1579-1581.
29. M. Figueiredo, R. Nowak, and S. J. Wright, GPSR: Gradient Projection for Sparse Reconstruction, http://www.lx.it.pt/~mtf/GPSR/ (2007).
30. M. P. Friedlander and P. Tseng, Exact regularization of convex programs, SIAM J. Optim., 18 (2007), pp. 1326-1350.
31. D. Goldfarb and W. Yin, Parametric maximum flow algorithms for fast total variation minimization, SIAM J. Sci. Comput., 31 (2009), pp. 3712-3743.
32. T. Goldstein, X. Bresson, and S. Osher, Geometric Applications of the Split Bregman Method: Segmentation and Surface Reconstruction, CAM Report 09-06, UCLA, Los Angeles, CA, 2009.
33. T. Goldstein and S. Osher, The Split Bregman Algorithm for L1 Regularized Problems, CAM Report 08-29, UCLA, Los Angeles, CA, 2008.
34. R. Gribonval, R. M. Figuerasi Ventura, and P. Vandergheynst, A simple test to check the optimality of a sparse signal approximation, Signal Process., 86 (2006), pp. 496-510.
35. W. Guo and F. Huang, A local mutual information guided denoising technique and its application to self-calibrated partially parallel imaging, in Medical Image Computing and Computer Assisted Intervention-MICCA, 2008, Lecture Notes in Comput. Sci. 5242, Springer-Verlag, Berlin, 2008, pp. 937-947.
36. E. T. Hale, W. Yin, and Y. Zhang, Fixed-point continuation applied to compressed sensing: Implementation and numerical experiments, J. Comput. Math., 28 (2010), pp. 170-194.
37. E. T. Hale, W. Yin, and Y. Zhang, Fixed-point continuation for l1-minimization: Methodology and convergence, SIAM J. Optim., 19 (2008), pp. 1107-1130.
38. L. He, T.-C. Chang, S. Osher, T. Fang, and P. Speier, MR Image Reconstruction by Using the Iterative Refinement Method and Nonlinear Inverse Scale Space Methods, CAM Report 06-35, UCLA, Los Angeles, CA, 2006.
39. L. He, A. Marquina, and S. Osher, Blind deconvolution using TV regularization and Bregman iteration, Int. J. Imaging Syst. Technol., 15 (2005), pp. 74-83.
40. M. R. Hestenes, Multiplier and gradient methods, J. Optim. Theory Appl., 4 (1969), pp. 303-320.
41. A. Juditsky and A. S. Nemirovski, On Verifiable Sufficient Conditions for Sparse Signal Recovery via l1 Minimization, 2008; available online from http://www.arxiv.org/abs/0809.2650.
42. S.-J. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, An interior-point method for large-scale l1-regularized least squares, IEEE J. Sel. Top. Signal Process., 1 (2007), pp. 606-617.
43. C. Lemaréchal and C. Sagastizábal, Practical aspects of the Moreau-Yosida regularization: Theoretical preliminaries, SIAM J. Optim., 7 (1997), pp. 367-385.
44. D. C. Liu and J. Nocedal, On the limited memory method for large scale optimization, Math. Programming Ser. B, 45 (1989), pp. 503-528.
45. S. Ma, D. Goldfarb, and L. Chen, Fixed point and Bregman iterative methods for matrix rank minimization, Math. Program., to appear.
46. O. L. Mangasarian, Normal solutions of linear programs, Math. Programming Stud., No. 22 (1984), pp. 206-216.
47. O. L. Mangasarian and R. R. Meyer, Nonlinear perturbation of linear programs, SIAM J. Control Optim., 17 (1979), pp. 745-752.
48. Y. Nesterov, Gradient Methods for Minimizing Composite Objective Function, CORE Discussion Paper 2007/76, 2007, http://www.optimization-online.org.
49. J. Nocedal and S. J. Wright, Numerical Optimization, Springer-Verlag, New York, 1999.
50. th Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, 2001, pp. 371-375.
51. S. Osher, M. Burger, D. Goldfarb, J. Xu, and W. Yin, An iterative regularization method for total variation-based image restoration, Multiscale Model. Simul., 4 (2005), pp. 460-489.
52. S. Osher, Y. Mao, B. Dong, and W. Yin, Fast linearized Bregman iteration for compressive sensing and sparse denoising, Commun. Math. Sci., 8 (2010), pp. 93-111.
53. M. J. D. Powell, A method for nonlinear constraints in minimization problems, in Optimization, R. Fletcher, ed., Academic Press, New York, 1972, pp. 283-298.
54. R. T. Rockafellar, Convex Analysis, Princeton University Press, Princeton, NJ, 1970.
55. R. T. Rockafellar, A dual approach to solving nonlinear programming problems by unconstrained optimization, Math. Programming, 5 (1973), pp. 354-373.
56. M. Rudelson and R. Vershynin, Geometric approach to error correcting codes and reconstruction of signals, Int. Math. Res. Not., no. 64 (2005), pp. 4019-4041.
57. X. C. Tai and C. Wu, Augmented Lagrangian Method, Dual Methods and Split Bregman Iteration for ROF Model, CAM Report 09-05, UCLA, Los Angeles, CA, 2009.
58. E. van den Berg and M. P. Friedlander, SPGL1: A Solver for Large-Scale Sparse Reconstruction, http://www.cs.ubc.ca/labs/scl/spgl1 (2007).
59. Y. Wang, J. Yang, W. Yin, and Y. Zhang, A new alternating minimization algorithm for total variation image reconstruction, SIAM J. Imaging Sci., 1 (2008), pp. 248-272.
60. R. Ward, Compressed sensing with cross validation, IEEE Trans. Inform. Theory, 55 (2009), pp. 5773-5782.
61. Z. Wen, W. Yin, D. Goldfarb, and Y. Zhang, A fast algorithm for sparse reconstruction based on shrinkage, subspace optimization, and continuation, SIAM J. Sci. Comput., 32 (2010), pp. 1832-1857.
62. S. Wright, R. Nowak, and M. Figueiredo, Sparse reconstruction by separable approximation, IEEE Trans. Signal Process., 57 (2009), pp. 2479-2493.
63. J. Xu and S. Osher, Iterative regularization and nonlinear inverse scale space applied to wavelet-based denoising, IEEE Trans. Image Process., 16 (2006), pp. 534-544.
64. J. Yang, W. Yin, Y. Zhang, and Y. Wang, A fast algorithm for edge-preserving variational multichannel image restoration, SIAM J. Imaging Sci., 2 (2009), pp. 569-592.
65. J. Yang, Y. Zhang, and W. Yin, A fast alternating direction method for TVL1-L2 signal reconstruction from partial Fourier data, IEEE J. Sel. Top. Signal Process., 4 (2008), pp. 288-297.
66. J. Yang, Y. Zhang, and W. Yin, An efficient TVL1 algorithm for deblurring multichannel images corrupted by impulsive noise, SIAM J. Sci. Comput., 31 (2009), pp. 2842-2865.
67. W. Yin, S. Osher, D. Goldfarb, and J. Darbon, Bregman iterative algorithms for l1-minimization with applications to compressed sensing, SIAM J. Imaging Sci., 1 (2008), pp. 143-168.
68. X. Zhang, M. Burger, X. Bresson, and S. Osher, Bregmanized nonlocal regularization for deconvolution and sparse reconstruction, SIAM J. Imaging Sci., 3 (2010), pp. 253-276.
69. Y. Zhang, A Simple Proof for Recoverability of l1-Minimization: Go Over or Under?, CAAM Technical report TR05-09, Department of Computational and Applied Mathematics, Rice University, Houston, TX, 2005.
70. Y. Zhang, When Is Missing Data Recoverable?, CAAM Technical report TR06-15, Department of Computational and Applied Mathematics, Rice University, Houston, TX, 2006.