A fast algorithm for sparse reconstruction based on shrinkage, subspace optimization, and continuation

SIAM Journal on Scientific Computing

Volumn 32, Issue 4, 2010, Pages 1832-1857

  Wen, Zaiwen ^a   Yin, Wotao ^b   Goldfarb, Donald ^a   Zhang, Yin ^b  

^a Columbia University ^*  (United States)

^b Rice University   (United States)

Author keywords

Active set; Basis pursuit; Compressive sensing; Continuation; L1 minimization; Shrinkage; Subspace optimization

Indexed keywords

COMPRESSED SENSING; ITERATIVE METHODS;

ACTIVE SETS; ALGORITHM FOR SOLVING; BASIS PURSUITS; COMPRESSIVE SENSING; CONTINUATION; FAST ALGORITHMS; L1-MINIMIZATION; MINIMISATION; SPARSE RECONSTRUCTION; SUBSPACE OPTIMIZATION;

SHRINKAGE;

EID: 77955666600     PISSN: 10648275     EISSN: None     Source Type: Journal    
DOI: 10.1137/090747695     Document Type: Article

References (67)

1. S. AYBAT AND G. IYENGAR, A first-order smoothed penalty method for compressed sensing, SIAM J. Optim., submitted.
2. J. Barzilai AND J. M. Borwein, Two-point step size gradient methods, IMA J. Numer. Anal., 8 (1988), pp. 141-148.
3. S. BECKER, J. BOBIN, AND E. CANDÈS, NESTA: A Fast and Accurate First-order Method for Sparse Recovery, Technical report, California Institute of Technology, Pasadena, 2009; available online from http://arxiv.org/abs/0904. 3367.
4. 1-unified variational framework for image restoration, in Proceedings of the European Conference on Computer Vision, Prague, Lecture Notes in Comput. Sci. 3024, Springer, Berlin, Heidelberg, 2004, pp. 1-13.
5. J. M. BIOUCAS-DIAS AND M. A. T. FIGUEIREDO, A new twist: Two-step iterative shrinkage/thresholding algorithms for image restoration, IEEE Trans. Image Process., 16 (2007), pp. 2992-3004.
6. J. V. BURKE AND J. J. MORÉ, On the identification of active constraints, SIAM J. Numer. Anal., 25 (1988), pp. 1197-1211.
7. J. V. BURKE AND J. J. MORÉ, Exposing constraints, SIAM J. Optim., 4 (1994), pp. 573-595.
8. R. H. BYRD, N. I. M. GOULD, J. NOCEDAL, AND R. A. WALTZ, An algorithm for nonlinear optimization using linear programming and equality constrained subproblems, Math. Program., 100 (2004), pp. 27-48.
9. R. H. BYRD, N. I. M. GOULD, J. NOCEDAL, AND R. A. WALTZ, On the convergence of successive linear-quadratic programming algorithms, SIAM J. Optim., 16 (2005), pp. 471-489.
10. E. Candes AND S. Becker, private communication, 2008.
11. E. Candès AND J. Romberg, Quantitative robust uncertainty principles and optimally sparse decompositions, Found. Comput. Math., 6 (2006), pp. 227-254.
12. E. CANDÈS, J. ROMBERG, AND T. TAO, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inform. Theory, 52 (2006), pp. 489-509.
13. E. CANDÈS AND T. TAO, Near optimal signal recovery from random projections: Universal encoding strategies, IEEE Trans. Inform. Theory, 52 (2004), pp. 5406-5425.
14. P. L. COMBETTES AND J.-C. PESQUET, Proximal thresholding algorithm for minimization over orthonormal bases, SIAM J. Optim., 18 (2007), pp. 1351-1376.
15. P. L. COMBETTES AND V. R. WAJS, Signal recovery by proximal forward-backward splitting, Multiscale Model. Simul., 4 (2005), pp. 1168-1200.
16. A. R. CONN AND J. W. SINCLAIR, Quadratic Programming via a Nondifferentiable Penalty Function, Technical report CORR 75/15, Faculty of Mathematics, University of Waterloo, Waterloo, ON, Canada, 1975.
17. W. DAI AND M. OLGICA, Subspace Pursuit for Compressive Sensing: Closing the Gap between Performance and Complexity, Technical report, Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, IL, 2008.
18. Y. H. DAI, On the nonmonotone line search, J. Optim. Theory Appl., 112 (2002), pp. 315-330.
19. C. DE MOL AND M. DEFRISE, A note on wavelet-based inversion algorithms, in Inverse Problems, Image Analysis, and Medical Imaging, Contemp. Math. 313, AMS, Providence, RI, 2002, pp. 85-96.
20. E. D. DOLAN AND J. J. MORÉ, Benchmarking optimization software with performance profiles, Math. Program., 91 (2002), pp. 201-213.
21. D. DONOHO, Compressed sensing, IEEE Trans. Inform. Theory, 52 (2006), pp. 1289-1306.
22. D. Donoho, I. Drori, V. Stodden, AND Y. Tsaig, Sparse Lab, 2008, http://sparselab. stanford.edu/.
23. D. DONOHO, Y. TSAIG, I. DRORI, AND J.-C. STARCK, Sparse solution of underdetermined linear equations by stagewise orthogonal matching pursuit, IEEE Trans. Inform. Theory, submitted.
24. M. ELAD, Why simple shrinkage is still relevant for redundant representations?, IEEE Trans. Inform. Theory, 52 (2006), pp. 5559-5569.
25. M. ELAD, B. MATALON, J. SHTOK, AND M. ZIBULEVSKY, A wide-angle view at iterated shrinkage algorithms, in Proceedings of the SPIE (Wavelets XII), Vol. 6701, San Diego, 2007, 670102.
26. M. ELAD, B. MATALON, AND M. ZIBULEVSKY, Coordinate and subspace optimization methods for linear least squares with non-quadratic regularization, Appl. Comput. Harmon. Anal., 23 (2006), pp. 346-367.
27. F. FACCHINEI, A. FISCHER, AND C. KANZOW, On the accurate identification of active constraints, SIAM J. Optim., 9 (1998), pp. 14-32.
28. M. FIGUEIREDO AND R. NOWAK, An EM algorithm for wavelet-based image restoration, IEEE Trans. Image Process., 12 (2003), pp. 906-916.
29. M. A. T. FIGUEIREDO, R. D. NOWAK, AND S. J. WRIGHT, Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems, IEEE J. Sel. Topics Signal Process., 1 (2007), pp. 586-598.
30. N. I. M. GOULD AND P. L. TOINT, An iterative working-set method for large-scale nonconvex quadratic programming, Appl. Numer. Math., 43 (2002), pp. 109-128.
31. L. Grippo, F. Lampariello, AND S. Lucidi, A nonmonotone line search technique for Newton's method, SIAM J. Numer. Anal., 23 (1986), pp. 707-716.
32. W. W. HAGER AND H. ZHANG, A new active set algorithm for box constrained optimization, SIAM J. Optim., 17 (2006), pp. 526-557.
33. 1-minimization: Methodology and convergence, SIAM J. Optim., 19 (2008), pp. 1107-1130.
34. E. T. HALE, W. YIN, AND Y. ZHANG, A numerical study of fixed-point continuation applied to compressed sensing, J. Comput. Math., 28 (2010), pp. 170-194.
35. Ilog, Inc., ILOG CPLEX 10.0, 2006, http://www-01.ibm.com/software/ integration/optimization/cplex-optimizer/.
36. 1-regularized least squares, IEEE J. Sel. Topics Signal Process., 1 (2007), pp. 606-617. (Pubitemid 351266626)
37. S. KIROLOS, J. LASKA, M. WAKIN, M. DUARTE, D. BARON, T. RAGHEB, Y. MASSOUD, AND R. BARANIUK, Analog-to-information conversion via random demodulation, in Proceedings of the IEEE Dallas Circuits and Systems Workshop (DCAS), Dallas, 2006.
38. J. LASKA, S. KIROLOS, M. DUARTE, T. RAGHEB, R. BARANIUK, AND Y. MASSOUD, Theory and implementation of an analog-to-information converter using random demodulation, in Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS), New Orleans, 2007.
39. J. LASKA, S. KIROLOS, Y. MASSOUD, R. BARANIUK, A. GILBERT, M. IWEN, AND M. STRAUSS, Random sampling for analog-to-information conversion of wideband signals, in Proceedings of the IEEE Dallas Circuits and Systems Workshop (DCAS), Dallas, 2006.
40. H. LEE, A. BATTLE, R. RAINA, AND A. Y. NG, Efficient sparse coding algorithms, in Advances in Neural Information Processing Systems, Vol. 19, B. Schölkopf, J. Platt, and T. Hoffman, eds., MIT Press, Cambridge, MA, 2007, pp. 801-808.
41. M. LUSTIG, D. DONOHO, AND J. M. PAULY, Sparse MRI: The application of compressed sensing for rapid MR imaging, Magn. Reson. Med., 58 (2007), pp. 1182-1195.
42. S. G. MALLAT AND Z. ZHANG, Matching pursuits with time-frequency dictionaries, IEEE Trans. Signal Process., 41 (1993), pp. 3397-3415.
43. J. J. MORÉ AND G. TORALDO, Algorithms for bound constrained quadratic programming problems, Numer. Math., 55 (1989), pp. 377-400.
44. J. J. MORÉ AND G. TORALDO, On the solution of large quadratic programming problems with bound constraints, SIAM J. Optim., 1 (1991), pp. 93-113.
45. D. NEEDELL AND J. A. TROPP, Cosamp: Iterative signal recovery from incomplete and inaccurate samples, Appl. Comput. Harmon. Anal., 26 (2008), pp. 301-321.
46. Y. NESTEROV, Gradient Methods for Minimizing Composite Objective Function, CORE Discussion Paper 2007/76, 2007, http://www.optimization-online. org.
47. J. NOCEDAL AND S. J. WRIGHT, Numerical Optimization, Springer Ser. Oper. Res., 2nd ed., Springer, New York, 2006.
48. R. NOWAK AND M. FIGUEIREDO, Fast wavelet-based image deconvolution using the EM algorithm, in Proceedings of the 35th IEEE Asilomar Conference on Signals, Systems, and Computers, Monterey, CA, 2001, pp. 371-375.
49. S. OSHER, Y. MAO, B. DONG, AND W. YIN, Fast linearized Bregman iteration for compressive sensing and sparse denoising, Commun. Math. Sci., 8 (2010), pp. 93-111.
50. W. SUN AND Y.-X. YUAN, Optimization Theory and Methods, Nonlinear Programming, Springer Optimization and Its Applications 1, Springer, New York, 2006.
51. D. TAKHAR, J. LASKA, M. WAKIN, M. DUARTE, D. BARON, S. SARVOTHAM, K. KELLY, AND R. BARANIUK, A new compressive imaging camera architecture using optical-domain compression, in Proceedings of Computational Imaging IV at SPIE Electronic Imaging, San Jose, CA, 2006, pp. 43-52.
52. P. L. TOINT, An assessment of nonmonotone linesearch techniques for unconstrained optimization, SIAM J. Sci. Comput., 17 (1996), pp. 725-739.
53. J. TROPP, Greed is good: Algorithmic results for sparse approximation, IEEE Trans. Inform. Theory, 50 (2006), pp. 2231-2342.
54. J. TROPP, Just relax: Convex programming methods for identifying sparse signals, IEEE Trans. Inform. Theory, 51 (2006), pp. 1030-1051.
55. J. TROPP, M. WAKIN, M. DUARTE, D. BARON, AND R. BARANIUK, Random filters for compressive sampling and reconstruction, in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Toulouse, France, 2006, pp. 872-875.
56. P. Tseng AND S. Yun, Block-coordinate gradient descent method for linearly constrained nonsmooth separable optimization, J. Optim. Theory Appl., 140 (2009), pp. 513-535.
57. P. TSENG AND S. YUN, A coordinate gradient descent method for nonsmooth separable minimization, Math. Program., 117 (2009), pp. 387-423.
58. E. van den Berg AND M. P. Friedlander, Probing the Pareto frontier for basis pursuit solutions, SIAM J. Sci. Comput., 31 (2008), pp. 890-912.
59. E. VAN DEN BERG, M. P. FRIEDLANDER, G. HENNENFENT, F. J. HERRMANN, R. SAAB, AND O. YILMAZ, Algorithm 890: Sparco: A testing framework for sparse reconstruction, ACM Trans. Math. Software, 35 (2009), pp. 1-16.
60. M. WAKIN, J. LASKA, M. DUARTE, D. BARON, S. SARVOTHAM, D. TAKHAR, K. KELLY, AND R. BARANIUK, An architecture for compressive imaging, in Proceedings of the IEEE International Conference on Image Processing (ICIP), Atlanta, 2006, pp. 1273-1276.
61. M. WAKIN, J. LASKA, M. DUARTE, D. BARON, S. SARVOTHAM, D. TAKHAR, K. KELLY, AND R. BARANIUK, Compressive imaging for video representation and coding, in Proceedings of the Picture Coding Symposium (PCS), Beijing, China, 2006.
62. 1 Minimization, Technical report, Department of Industrial Engineering and Operations Research, Columbia University, New York, 2008.
63. S. J. WRIGHT, R. D. NOWAK, AND M. A. T. FIGUEIREDO, Sparse reconstruction by separable approximation, IEEE Trans. Signal Proc, 57 (2009), pp. 2479-2493.
64. J. YANG, Y. ZHANG, AND W. YIN, A fast alternating direction method for TVL1-L2 signal reconstruction from partial Fourier data, IEEE J. Sel. Top. Signal Process., 4 (2007), pp. 288-297.
65. 1minimization with applications to compressed sensing, SIAM J. Imaging Sci., 1 (2008), pp. 143-168.
66. H. ZHANG AND W. W. HAGER, A nonmonotone line search technique and its application to unconstrained optimization, SIAM J. Optim., 14 (2004), pp. 1043-1056.
67. C. ZHU, R. H. BYRD, P. LU, AND J. NOCEDAL, Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization, ACM Trans. Math. Software, 23 (1997), pp. 550-560.