NESTA: A fast and accurate first-order method for sparse recovery

SIAM Journal on Imaging Sciences

Volumn 4, Issue 1, 2011, Pages 1-39

  Becker, Stephen ^a   Bobin, Jérôme ^a   Candès, Emmanuel J ^a,b  

^a CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^b STANFORD UNIVERSITY   (United States)

Author keywords

Compressed sensing; Continuation methods; Duality in convex optimization; l1 Minimization; Nesterov's method; Smooth approximations of nonsmooth functions; Total variation minimization

Indexed keywords

COMPRESSED SENSING; CONTINUATION METHODS; DUALITY IN CONVEX OPTIMIZATION; L1 MINIMIZATION; NESTEROV'S METHOD; SMOOTH APPROXIMATIONS OF NONSMOOTH FUNCTIONS; TOTAL-VARIATION;

ALGORITHMS; CONVEX OPTIMIZATION; IMAGE RECONSTRUCTION; NUMERICAL METHODS; OPTIMIZATION; QUEUEING NETWORKS;

PROBLEM SOLVING;

EID: 79551550744     PISSN: None     EISSN: 19364954     Source Type: Journal    
DOI: 10.1137/090756855     Document Type: Article

References (61)

1. M. V. Afonso, J. M. Bioucas-Dias, and M. A. T. Figueiredo, Fast image recovery using variable splitting and constrained optimization, IEEE Trans. Image Process., 19 (2010), pp. 2345-2356.
2. J.-F. Aujol, Some first-order algorithms for total-variation based image restoration, J. Math. Imaging Vision, 34 (2009), pp. 307-327.
3. J. Barzilai and J. Borwein, Two point step size gradient method, IMA J. Numer. Anal., 8 (1988), pp.141-148.
4. A. Beck and M. Teboulle, Fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci., 2 (2009), pp. 183-202.
5. J. M. Bioucas-Dias and M. A. T. Figueiredo, A new TwIST: Two-step iterative shrinkage/ thresholding algorithms for image restoration, IEEE Trans. Image Process., 16 (2007), pp. 2992-3004.
6. E. G. Birgin, J. M. Martínez, and M. Raydan, Nonmonotone spectral projected gradient methods on convex sets, SIAM J. Optim., 10 (2000), pp. 1196-1211.
7. J. Bobin and E. J. Candès, A fast and accurate first-order algorithm for compressed sensing, in Proceedings of the 16th IEEE International Conference on Image Processing (ICIP'09), 2009, pp. 1457-1460.
8. J. Bobin, J.-L. Starck, and R. Ottensamer, Compressed sensing in astronomy, IEEE J. Sel. Top. Signal Process., 2 (2008), pp. 718-726.
9. J.-F. Cai, S. Osher, and Z. Shen, Convergence of the linearized Bregman iterations for l1-norm minimization, Math. Comp., 78 (2009), pp. 2127-2136.
10. E. J. Candès, L1-Magic, Technical report, Caltech, Pasadena, CA, 2007.
11. E. Candès, L. Demanet, D. Donoho, and L. Ying, Fast discrete curvelet transforms, Multiscale Model. Simul., 5 (2006), pp. 861-899.
12. E. J. Candès and Y. Plan, Near-ideal model selection by l1 minimization, Ann. Statist., 37 (2007), pp. 2145-2177.
13. E. J. Candès, J. Romberg, and T. Tao, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inform. Theory, 52 (2006), pp. 489-509.
14. E. J. Candès and T. Tao, Near optimal signal recovery from random projections: Universal encoding strategies?, IEEE Trans. Inform. Theory, 52 (2006), pp. 5406-5425.
15. E. J. Candès and T. Tao, The Dantzig selector: Statistical estimation when p is much larger than n, Ann. Statist., 35 (2007), pp. 2313-2351.
16. E. J. Candès, M. B. Wakin, and S. P. Boyd, Enhancing sparsity by reweighted l1 minimization, J. Fourier Anal. Appl., 14 (2008), pp. 877-905.
17. A. Chambolle, An algorithm for total-variation minimization and applications, J. Math. Imaging Vision, 20 (2004), pp. 89-97.
18. S. S. Chen, D. L. Donoho, and M. A. Saunders, Atomic decomposition by basis pursuit, SIAM J. Sci. Comput., 20 (1998), pp. 33-61.
19. P. L. Combettes and J. C. Pesquet, A proximal decomposition method for solving convex variational inverse problems, Inverse Problems, 24 (2008), 065014.
20. P. L. Combettes and V. R. Wajs, Signal recovery by proximal forward-backward splitting, Multiscale Model. Simul., 4 (2005), pp. 1168-1200.
21. J. Dahl, C. Hansen, S. H. Jensen, and T. L. Jensen, Algorithms and software for total variation image reconstruction via first-order methods, Numer. Algorithms, 53 (2010), pp. 67-92.
22. J. Darbon and M. Sigelle, A fast and exact algorithm for total-variation minimization, in Proceedings of the 2nd Iberian Conference on Pattern Recognition and Image Analysis (IbPRIA), Lecture Notes in Comput. Sci. 3522, Springer-Verlag, Berlin, 2005, pp. 351-359.
23. DARPA, Analog-to-Information Receiver Development Program (A-to-I), DARPA Broad Agency Announcement BAA08-03, 2007; available online from http://www.darpa.mil/mto/Solicitations/baa08-03/index.html.
24. I. Daubechies, M. Defrise, and C. De Mol, An iterative thresholding algorithm for linear inverse problems with a sparsity constraint, Comm. Pure Appl. Math., 57 (2004), pp. 1413-1457.
25. I. Daubechies, M. Fornasier, and I. Loris, Accelerated projected gradient method for linearinverse problems with sparsity constraints, J. Fourier Anal. Appl., 14 (2008), pp. 764-792.
26. D. L. Donoho, Compressed sensing, IEEE Trans. Inform. Theory, 52 (2006), pp. 1289-1306.
27. D. L. Donoho, A. Maleki, I. Rahman, M. Shahram, and V. Stodden, Reproducible research in computational harmonic analysis, Computing in Science and Engineering, 11 (2009), pp. 8-18.
28. D. L. Donoho and Y. Tsaig, Fast solution of l1 minimization problems when the solution may be sparse, IEEE Trans. Inform. Theory, 54 (2008), pp. 4789-4812.
29. D. L. Donoho, Y. Tsaig, I. Drori, and J.-L. Starck, Sparse solution of underdetermined linear equations by stagewise orthogonal matching pursuit, IEEE Trans. Inform. Theory, submitted.
30. M. Elad, B. Matalon, and M. Zibulevsky, Coordinate and subspace optimization methods for linear least squares with non-quadratic regularization, Appl. Comput. Harmon. Anal., 23 (2007), pp. 346-367.
31. M. Elad, P. Milanfar, and R. Rubinstein, Analysis versus synthesis in signal priors, Inverse Problems, 23 (2007), pp. 947-968.
32. M. A. T. Figueiredo and R. D. Nowak, An EM algorithm for wavelet-based image restoration, IEEE Trans. Image Process., 12 (2003), pp. 906-916.
33. M. A. T. Figueiredo, R. D. Nowak, and S. J. Wright, Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems, IEEE J. Sel. Top. Signal Process., 1 (2007), pp. 586-597.
34. J. Friedman, T. Hastie, and R. Tibshirani, Regularization paths for generalized linear models via coordinate descent, J. Stat. Softw., 33 (2010), pp. 1-22.
35. T. Goldstein and S. Osher, The split Bregman method for L1 regularized problems, SIAM J. Imaging Sci., 2 (2009), pp. 323-343.
36. E. T. Hale, W. Yin, and Y. Zhang, A Fixed-Point Continuation Method for l1-Regularized Minimization with Applications to Compressed Sensing, CAAM Technical Report TR07-07, Rice University, Houston, TX, 2007.
37. E. T. Hale, W. Yin, and Y. Zhang, Fixed-point continuation for l1-minimization: Methodology and convergence, SIAM J. Optim., 19 (2008), pp. 1107-1130.
38. S. J. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, An interior-point method for large-scale l1-regularized least squares, IEEE J. Sel. Top. Signal Process., 1 (2007), pp. 606-617.
39. T. Lin and F. J. Herrmann, Compressed wavefield extrapolation, Geophysics, 72 (2007), pp. SM77-SM93.
40. I. Loris, On the performance of algorithms for the minimization of l1-penalized functionals, Inverse Problems, 25 (2009), 035008.
41. I. Loris, M. Bertero, C. De Mol, R. Zanella, and L. Zanni, Accelerating gradient projection methods for _1-constrained signal recovery by steplength selection, Appl. Comput. Harmon. Anal., 27 (2009), pp. 247-254.
42. M. Lustig, D. L. Donoho, and J. M. Pauly, Sparse MRI: The application of compressed sensing for rapid MR imaging, Magn. Reson. Med., 58 (2007), pp. 1182-1195.
43. S. Mallat, A Wavelet Tour of Signal Processing, Academic Press, San Diego, 1998.
44. D. Needell and J. A. Tropp, CoSaMP: Iterative signal recovery from incomplete and inaccurate samples, Appl. Comput. Harmon. Anal., 26 (2009), pp. 301-321.
45. Y. Nesterov, A method for solving the convex programming problem with convergence rate O(1/k2), Dokl. Akad. Nauk SSSR, 269 (1983), pp. 543-547 (in Russian); Soviet Math. Dokl., 27 (1983), pp 372-376 (in English).
46. Y. Nesterov, Introductory Lectures on Convex Optimization. A Basic Course, Kluwer Academic Publishers, Boston, 2004.
47. Y. Nesterov, Smooth minimization of non-smooth functions, Math. Program. Ser. A, 103 (2005), pp. 127-152.
48. Y. Nesterov, Gradient Methods for Minimizing Composite Objective Function, Technical report 2007/76, CORE, Université catholique de Louvain, Louvain-la-Neuve, Belgium, 2007.
49. M. R. Osborne, B. Presnell, and B. A. Turlach, A new approach to variable selection in least squares problems, IMA J. Numer. Anal., 20 (2000), pp. 389-403.
50. S. Osher, Y. Mao, B. Dong, and W. Yin, Fast linearized Bregman iteration for compressive sensing and sparse denoising, Commun. Math. Sci., 8 (2010), pp. 93-111.
51. R. T. Rockafellar, Convex Analysis, Princeton Math. Ser. 28, Princeton University Press, Princeton, NJ, 1970.
52. M. A. Saunders and B. Kim, PDCO: Primal-Dual Interior Method for Convex Objectives, Technical report, Stanford University, Stanford, CA, 2002; available online from http://www.stanford.edu/group/SOL/software/pdco.html.
53. R. Tibshirani, Regression shrinkage and selection via the lasso, J. Roy. Statist. Soc. Ser. B, 58 (1996), pp. 267-288.
54. P. Tseng, On Accelerated Proximal Gradient Methods for Convex-Concave Optimization, Technical report, Department of Mathematics, University of Washington, Seattle, WA; available online from http://www.math.washington.edu/~tseng/papers.html.
55. E. van den Berg and M. P. Friedlander, Probing the Pareto frontier for basis pursuit solutions, SIAM J. Sci. Comput., 31 (2008), pp. 890-912.
56. P. Weiss, L. Blanc-Féraud, and G. Aubert, Efficient schemes for total variation minimization under constraints in image processing, SIAM J. Sci. Comput., 31 (2009), pp. 2047-2080.
57. Z. Wen, W. Yin, D. Goldfarb, and Y. Zhang, A fast algorithm for sparse reconstruction based on shrinkage, subspace optimization, and continuation, SIAM J. Sci. Comput., 32 (2010), pp. 1832-1857.
58. S. J. Wright, R. D. Nowak, and M. A. T. Figueiredo, Sparse reconstruction by separable approximation, IEEE Trans. Signal Process., 57 (2009), pp. 2479-2493.
59. J. Yang, Y. Zhang, and W. Yin, A fast alternating direction method for TV l1-l2 signal reconstruction from partial Fourier data, IEEE J. Sel. Top. Signal Process., 4 (2010), pp. 288-297.
60. W. Yin, S. Osher, D. Goldfarb, and J. Darbon, Bregman iterative algorithms for l1 minimization with applications to compressed sensing, SIAM J. Imaging Sci., 1 (2008), pp. 143-168.
61. M. Zhu and T. Chan, An Efficient Primal-Dual Hybrid Gradient Algorithm for Total Variation Image Restoration, UCLA CAM Report [08-34], UCLA, Los Angeles, CA, 2008; available online from ftp://ftp.math.ucla.edu/pub/camreport/cam08-34.pdf