Alternating direction algorithms for ℓ1-problems in compressive sensing

SIAM Journal on Scientific Computing

Volumn 33, Issue 1, 2011, Pages 250-278

  Yang, Junfeng ^a   Zhang, Yin ^b  

^a NANJING UNIVERSITY   (China)

^b Rice University   (United States)

Author keywords

Alternating direction method; Augmented lagrangian function; Compressive sensing; Dual; Primal

Indexed keywords

CONSTRAINED OPTIMIZATION; ITERATIVE METHODS; LAGRANGE MULTIPLIERS;

1-NORM MINIMIZATIONS; ALTERNATING DIRECTION METHODS; ALTERNATING DIRECTIONS ALGORITHMS; AUGMENTED LAGRANGIAN FUNCTIONS; BASIS PURSUITS; COMPRESSIVE SENSING; DUAL; MINIMIZATION PROBLEMS; PRIMAL; SPARSE SOLUTIONS;

COMPRESSED SENSING;

EID: 79952307213     PISSN: 10648275     EISSN: None     Source Type: Journal    
DOI: 10.1137/090777761     Document Type: Article

References (64)

1. M. V. Afonso, J. Bioucas-Dias, and M. Figueiredo, Fast image recovery using variable splitting and constrained optimization, IEEE Trans. Image Process., 19 (2010), pp. 2345-2356.
2. M. V. Afonso, J. Bioucas-Dias, and M. Figueiredo, A fast algorithm for the constrained formulation of compressive image reconstruction and other linear inverse problems, in IEEE International Conference on Acoustics, Speech, and Signal Processing, IEEE, Washington, DC, 2010, pp. 4034-4037.
3. D. Baron, M. Duarte, S. Sarvotham, M. B. Wakin, and R. G. Baraniuk, Distributed compressed sensing, available at http://dsp.rice.edu/cs/DCS112005. pdf, 2009.
4. J. Barzilai and J. Borwein, Two point step size gradient methods, IMA J. Numer. Anal., 8 (1988), pp. 141-148.
5. A. Beck and M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci., 2 (2009), pp. 183-202.
6. S. Becker, J. Bobin, and E. Candès, NESTA: A Fast and Accurate First-Order Method for Sparse Recovery, Technical report, California Institute of Technology, Pasadena, CA, 2009.
7. J. Cai, S. Osher, and Z. Shen, Linearized Bregman Iterations for Compressive Sensing, UCLA CAM TR08-06, UCLA, Los Angeles, CA, 2008.
8. 1-Norm Minimization, UCLA CAM TR08-52, UCLA, Los Angeles, CA, 2008.
9. E. Candès, J. Romberg, and T. Tao, Stable signal recovery from incomplete and inaccurate information, Commun. Pure Appl. Math., 59 (2005), pp. 1207-1233.
10. E. Candès and B. Recht, Exact matrix completion via convex optimization, Found. Comput. Math., 9 (2008), pp. 717-772.
11. E. Candès, J. Romberg, and T. Tao, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inform. Theory, 52 (2006), pp. 489-509. (Pubitemid 43174093)
12. E. Candès and T. Tao, The power of convex relaxation: Near-optimal matrix completion, IEEE Trans. Inform. Theory, 56 (2009), pp. 2053-2080.
13. V. Chandrasekaran, S. Sanghavi, P. A. Parrilo, and A. S. Willsky, Rank Sparsity Incoherence for Matrix Decomposition, http://arxiv.org/abs/0906.2220, 2009.
14. S. S. Chen, D. L. Donoho, and M. A. Saunders, Atomic decomposition by basis pursuit, SIAM J. Sci. Comput., 20 (1998), pp. 33-61. (Pubitemid 128689501)
15. I. Daubechies, M. Defrise, and C. De Mol, An iterative thresholding algorithm for linear inverse problems with a sparsity constraint, Commun. Pure Appl. Math., 57 (2004), pp. 1413-1457. (Pubitemid 39427442)
16. C. De Mol and M. Defrise, A note on wavelet-based inversion algorithms, Contemp. Math., 313 (2002), pp. 85-96.
17. D. Donoho, Compressed sensing, IEEE Trans. Inform. Theory, 52 (2006), pp. 1289-1306.
18. 1-norm solution is also the sparsest solution, Commun. Pure Appl. Math., 59 (2006), pp. 907-934.
19. J. Douglas and H. Rachford, On the numerical solution of heat conduction problems in two and three space variables, Trans. Amer. Math. Soc., 82 (1956), pp. 421-439.
20. J. Eckstein and D. Bertsekas, On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators, Math. Program., 55 (1992), pp. 293-318.
21. M. Elad, Why simple shrinkage is still relevant for redundant representations?, IEEE Trans. Inform. Theory, 52 (2006), pp. 5559-5569. (Pubitemid 46445399)
22. E. Esser, Applications of Lagrangian-Based Alternating Direction Methods and Connections to Split Bregman, CAM report TR09-31, UCLA, Los Angeles, CA, 2009.
23. M. Figueiredo and R. Nowak, An EM algorithm for wavelet-based image restoration, IEEE Trans. Image Process., 12 (2003), pp. 906-916.
24. M. Figueiredo, R. Nowak, and S. J. Wright, Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems, IEEE J. Sel. Top. Signa., 1 (2007), pp. 586-597. (Pubitemid 351276802)
25. M. Fukushima, Application of the alternating direction method of multipliers to separable convex programming, Comput. Optim. Appl., 1 (1992), pp. 93-111.
26. D. Gabay and B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite-element approximations, Comp. Math. Appl., 2 (1976), pp. 17-40.
27. R. Glowinski, Numerical Methods for Nonlinear Variational Problems, Springer-Verlag, New York, 1984.
28. R. Glowinski and P. Le Tallec, Augmented Lagrangian and Operator Splitting Methods in Nonlinear Mechanics, SIAM Stud. Appl. Math. 9, SIAM, Philadelphia, 1989.
29. R. Glowinski and A. Marrocco, Sur l'approximation par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires, Rev. Française Automat. Informat. Rech. Opér., 9 (1975), pp. 41-76.
30. T. Goldstein and S. Osher, The split Bregman method for L1-regularized problems, SIAM J. Imaging Sci., 2 (2009), pp. 323-343.
31. 1-minimization: Methodology and convergence, SIAM J. Optim., 19 (2008), pp. 1107-1130.
32. E. Hale, W. Yin, and Y. Zhang, Fixed-point continuation applied to compressed sensing: Implementation and numerical experiments, J. Comput. Math., 28 (2010), pp. 170-194.
33. M. Hanke, Conjugate Gradient Type Methods for Ill-Posed Problems, Longman Scientific & Technical, Harlow, UK, 1995.
34. B. He, L. Liao, D. Han, and H. Yang, A new inexact alternating directions method for monotone variational inequalities, Math. Progam. Ser. A, 92 (2002), pp. 103-118. (Pubitemid 44769742)
35. B. He and H. Yang, Some convergence properties of a method of multipliers for linearly constrained monotone variational inequalities, Oper. Res. Lett., 23 (1998), pp. 151-161. (Pubitemid 128676387)
36. M. R. Hestenes, Multiplier and gradient methods, J. Optim. Theory Appl., 4 (1969), pp. 303-320.
37. C. Li, W. Yin, and Y. Zhang, User's Guide for TVAL3: TV Minimization by Augmented Lagrangian and Alternating Direction Algorithms, CAAM report, Rice University, Houston, TX, 2009.
38. V. A. Morozov, On the solution of functional equations by the method of regularization, Soviet Math. Dokl., 7 (1966), pp. 414-417.
39. Y. Nesterov, Smooth minimization of non-smooth functions, Math. Program. Ser. A, 103 (2005), pp. 127-152. (Pubitemid 40537674)
40. Y. Nesterov, Gradient methods for minimizing composite objective function, CORE discussion paper 2007/76, Université Catholique de Louvain, Louvain-la-Neuve, Belgium, 2007.
41. R. Nowak and M. Figueiredo, Fast wavelet-based image deconvolution using the EM algorithm, in Proceedings of the 35th Asilomar Conference on Signals, Systems and Computers, Vol. 1, IEEE, Washington, DC, 2001, pp. 371-375. (Pubitemid 34244663)
42. S. Osher, M. Burger, D. Goldfarb, J. Xu, and W. Yin, An iterative regularization method for total variation-based image restoration, Multiscale Model. Simul., 4 (2005), pp. 460-489. (Pubitemid 43885248)
43. G. Plonka and J. Ma, Curvelet-wavelet regularized split Bregman iteration for compressed sensing, Int. J. Wavelets Multiresolut. Inf. Process., to appear.
44. M. J. D. Powell, A method for nonlinear constraints in minimization problems, in Optimization, R. Fletcher, ed., Academic Press, New York, 1969, pp. 283-298.
45. B. Recht, M. Fazel, and P. Parrilo, Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization, SIAM Rev., 52 (2010), pp. 471-501.
46. R. T. Rockafellar, The multiplier method of Hestenes and Powell applied to convex programming, J. Optim. Theory Appl., 12 (1973), pp. 555-562.
47. L. Rudin, S. Osher, and E. Fatemi, Nonlinear total variation based noise removal algorithms, Phys. D, 60 (1992), pp. 259-268.
48. J.-L. Starck, E. Candès, and D. Donoho, Astronomical image representation by the curvelet transform, Astron. Astrophys., 398 (2003), pp. 785-800.
49. J.-L. Starck, M. Nguyen, and F. Murtagh, Wavelets and curvelets for image deconvolution: A combined approach, Signal Process., 83 (2003), pp. 2279-2283.
50. M. Tao, J.-F. Yang, and B. He, Alternating direction algorithms for total variation deconvolution in image reconstruction, available from http://www.optimizationonline. org/DB FILE/2009/11/2463.pdf., 2009.
51. R. Tibshirani, Regression shrinkage and selection via the Lasso, J. Roy. Statist. Soc. Ser. B., 58 (1996), pp. 267-288.
52. J. A. Tropp and A. C. Gilbert, Signal recovery from random measurements via orthogonal matching pursuit, IEEE Trans. Inform. Theory, 53 (2007), pp. 4655-4666.
53. P. Tseng, Applications of a splitting algorithm to decomposition in convex programming and variational inequalities, SIAM J. Control Optim., 29 (1991), pp. 119-138.
54. E. Van den Berg and M. Friedlander, Probing the pareto frontier for basis pursuit solutions, SIAM J. Sci. Comput., 31 (2008), pp. 890-912.
55. Y. Wang, J.-F. Yang, W. Yin, and Y. Zhang, A new alternating minimization algorithm for total variation image reconstruction, SIAM J. Imaging Sci., 1 (2008), pp. 248-272.
56. Z. Wen, W. Yin, and D. Goldfarb, Alternating direction augmented Lagrangian methods for semidefinite programming, Math. Prog. Comp., 2 (2010), pp. 203-230.
57. 1- minimization, IEEE Trans. Inform. Theory, to appear.
58. S. Wright, R. Nowak, and M. Figueiredo, Sparse reconstruction by separable approximation, IEEE Trans. Signal Process., 57 (2009), pp. 2479-2493.
59. J.-F. Yang, Y. Zhang, and W. Yin, A fast alternating direction method for TVL1-L2 signal reconstruction from partial Fourier data, IEEE J. Sel. Top. Signa., 4 (2010), pp. 288-297.
60. 1-minimization with applications to compressed sensing, SIAM J. Imaging Sci., 1 (2008), pp. 143-168.
61. W. Yin, The linearized Bregman method: Reviews, analysis, and generalizations, CAAM TR09-02, Rice University, Houston, TX, 2009.
62. X. Yuan and J.-F. Yang, Sparse and low-rank matrix decomposition via alternating direction methods, available from http://www.optimization-online. org/DB FILE/ 2009/11/2447.pdf., 2009.
63. 1-regularized convex minimization, Comput. Optim. Appl., to appear.
64. Y. Zhang, J.-F. Yang, and W. Yin, Your ALgorithm for L1, available from http://yall1.blogs. rice.edu/, 2009.