Fixed-point continuation applied to compressed sensing: Implementation and numerical experiments

Journal of Computational Mathematics

Volumn 28, Issue 2, 2010, Pages 170-194

  Hale, Elaine T ^a   Yin, Wotao ^a   Zhang, Yin ^a  

^a Rice University   (United States)

Author keywords

Compressed sensing; Continuation; Fixed point algorithm; l1 regularization; Numerical experiments

Indexed keywords

EID: 77950363235     PISSN: 02549409     EISSN: None     Source Type: Journal    
DOI: 10.4208/jcm.2009.10-m1007     Document Type: Article

References (81)

1. T. Adeyemi and M. Davies, Sparse representations of images using overcomplete complex wavelets, 2005 IEEE/SP 13th Workshop on Statistical Signal Processing, 2005.
2. 1-unified variational framework for image restoration, European Conference on Computer Vision, Prague, Lecture Notes in Computer Sciences 3024, IV, 2004.
3. D. Baron, M. Wakin, M. Duarte, S. Sarvotham, and R. Baraniuk, Distributed compressed sensing, Preprint, 2005.
4. J. Barzilai and J. Borwein, Two point step size gradient methods, IMA J. Numer. Anal, 8 (1988), 141-148.
5. D. P. Bertsekas, Nonlinear Programming, Athena Scientific, Belmont, Massachusetts, second edition, 1999.
6. J. Bioucas-Dias, Bayesian wavelet-based image deconvolution: a GEM algorithm exploiting a class of heavy-tailed priors, IEEE T. Image Process., 15:4 (2006), 937-951.
7. J. Bioucas-Dias and M. Figueiredo, A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration, IEEE T. Image Process., 16:12 (2007), 2992-3004.
8. T. Blumensath and M. Davies, Iterative thresholding for sparse approximations, Preprint, 2007.
9. T. Blumensath and M. E. Davies, Gradient pursuits, IEEE T. Signal Proces., 56:6 (2008), 2370-2382.
10. S. Bourguignon, H. Carfantan, and J. Idier, A sparsity-based method for the estimation of spectral lines from irregularly sampled data, IEEE Journal of Selected Topics in Signal Processing, 1 (2007), 575-585.
11. J.-F. Cai, S. Osher, and Z. Shen, Linearized Bregman iterations for frame-based image deblurring, To appear in SIAM Journal on Imaging Sciences, 2009.
12. E. Candes and J. Romberg, Quantitative robust uncertainty principles and optimally sparse decompositions, Found. Comput. Math., 6:2 (2006), 227-254.
13. E. Candes, J. Romberg, and T. Tao, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE T. Inform. Theory, 52 (2006), 489-509.
14. E. Candes and T. Tao, Near optimal signal recovery from random projections: universal encoding strategies, IEEE T. Inform. Theory, 52:1 (2006), 5406-5425.
15. A. Chambolle, An algorithm for total variation minimization and applications, J. Math. Imaging Vis., 20 (2004), 89-97.
16. A. Chambolle, R. A. DeVore, N.-Y. Lee, and B. J. Lucier, Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage, IEEE T. Image Process., 7:3 (1998), 319-335.
17. S. Chen, D. Donoho, and M. A. Saunders, Atomic decomposition by basis pursuit, SIAM J. Sci. Comput, 20 (1998), 33-61.
18. J. Claerbout and F. Muir, Robust modelling of erratic data, Geophysics, 38 (1973), 826-844.
19. P. L. Combettes and J.-C. Pesquet, Proximal thresholding algorithm for minimization over orthonormal bases, To appear in SIAM J. Optimiz., 2007.
20. Y. Dai and R. Fletcher, Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming, Numer. Math., 100 (2005), 21-47.
21. J. Darbon and S. Osher, Fast discrete approximations for sparse approximation and deconvolution, Technical Report (to appear), UCLA, 2007.
22. I. Daubechies, M. Defrise, and C. De Mol, An iterative thresholding algorithm for linear inverse problems with a sparsity constraint, Commun. Pur. Appl. Math., 57 (2004), 1413-1457
23. I. Daubechies, M. Fornasier, and I. Loris, Accelerated projected gradient method for linear inverse problems with sparsity constraints, arXiv:0706:4297, 2007.
24. C. De Mol and M. Defrise, A note on wavelet-based inversion algorithms, Contemporary Mathematics, 313 (2002), 85-96.
25. D. Donoho, Compressed sensing, IEEE T. Inform. Theory, 52 (2006), 1289-1306.
26. D. Donoho and J. Tanner, Neighborliness of randomly-projected simplices in high dimensions, Proc. National Academy of Sciences, 102:27 (2005), 9452-9457.
27. D. Donoho, Y. Tsaig, I. Drori, and J.-C. Starck, Sparse solution of underdetermined linear equations by stagewise orthogonal matching pursuit, Submitted to IEEE T. Inform. Theory, 2006.
28. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, Least angle regression, Ann. Stat., 32 (2004), 407-499.
29. M. Elad, Why simple shrinkage is still relevant for redundant representations? IEEE T. Inform. Theory, 52:12 (2006), 5559-5569.
30. M. Elad, B. Matalon, J. Shtok, and M. Zibulevsky, A wide-angle view at iterated shrinkage algorithms, SPIE (Wavelet XII), San-Diego CA, August 26-29, 2007., 2007.
31. M. Elad, B. Matalon, and M. Zibulevsky, Coordinate and subspace optimization methods for linear least squares with non-quadratic regularization, Journal on Applied and Computational Harmonic Analysis, 2006.
32. M. J. Fadili and J. L. Starck, Sparse representation-based image deconvolution by iterative thresholding, Astronomical Data Analysis ADA'06, Marseille, France, 2006.
33. M. Figueiredo, J. Bioucas-Dias, and R. Nowak, Majorization-minimization algorithms for wavelet-based image restoration, IEEE T. Image Process., 16:12 (2007), 2980-2991.
34. M. Figueiredo and R. Nowak, An EM algorithm for wavelet-based image restoration. IEEE T. Image Process., 12 (2003), 906-916.
35. M. Figueiredo and R. Nowak, A bound optimization approach to wavelet-based image deconvolution, Proceedings of the IEEE International Conference on Image Processing (ICIP), 2005.
36. M. Figueiredo, R. Nowak, and S. J. Wright, Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems, To appear in the IEEE Journal of Selected Topics in Signal Processing: Special Issue on Convex Optimization Methods for Signal Processing, 2007.
37. J. Friedman, T. Hastie, H. Honing, and R. Tibshirani, Pathwise coordinate optimization, Annals of Applied Statistics, 1 (2007), 302-332.
38. T. Goldstein and S. Osher, The split Bregman algorithm for L1 regularized problems, UCLA CAM Report 08-29, 2008.
39. 1-regularization with application to compressed sensing, Rice University CAAM Technical Report TR07-07, 2007.
40. 1-minimization: Methodology and convergence, SIAM J. Optimiz., 19 (2008), 1107-1130.
41. E. T. Hale, W. Yin, and Y. Zhang, A numerical study of fixed-point continuation applied to compressed sensing, Technical Report TR08-24, Rice University, Department of Computational and Applied Mathematics, 2008.
42. 1-regularized least squares problems with applications in signal processing and statistics, http://www.Stanford.edu/~boyd/11_1s.html. 2007.
43. S. Kirolos, J. Laska, M. Wakin, M. Duarte, D. Baron, T. Ragheb, Y. Massoud, and R. Baraniuk, Analog-to-information conversion via random demodulation, In Proceedings of the IEEE Dallas Circuits and Systems Workshop (DCAS), Dallas, Texas, 2006.
44. J. Laska, S. Kirolos, M. Duarte, T. Ragheb, R. Baraniuk, and Y. Massoud, Theory and imple- mentaion of an analog-to information converter using random demodulation, In Proceedings of the IEEE International Symposium on Circuites and Systems (ISCAS), New Orleans, Louisiana, 2007.
45. J. Laska, S. Kirolos, Y. Massoud, R. Baraniuk, A. Gilbert, M. Iwen, and M. Strauss, Radnom sampling for analog-to-information conversion of wideband signals, In Proceedings of the IEEE Dallas Circuits and Systems Workshop, Dallas, Texas, 2006.
46. S. Levy and P. Fullagar, Reconstruction of a sparse spike train from a portion of its spectrum and application to high-resolution deconvolution, Geophysics, 46 (1981), 1235-1243.
47. Z.-Q. Luo and P. Tseng, On the linear convergence of descent methods for convex essentially smooth minimization, SIAM J. Control Optimn, 30:2 (1990), 408-425.
48. M. Lustig, D. Donoho, and J. Pauly, Sparse MRI: The application of compressed sensing for rapid MR imaging, Preprint, 2007.
49. S. Ma, W. Yin, Y. Zhang, and A. Chakraborty, An efficient algorithm for compressed MR imaging using total variation and wavelets, IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2008, (1-8), 2008.
50. S. G. Mallat and Z. Zhang, Matching pursuits with time-frequency dictionaries, IEEE T. Signal Proces., 41:12 (1993), 3397-3415.
51. A. Miller, Subset Selection in Regression, Chapman and Hall, 2002.
52. Y. Nesterov, Gradient methods for minimizing composite objective function, CORE Discussion Paper 2007/76, 2007. http: //www. caam. rice, edu/~optimization/L1/fpc/.
53. R,. Nowak and M. Figueiredo, Fast wavelet-based image deconvolution using the EM algorithm, Proceedings of the 35th Asilomar Conference on Signals, Systems, and Computers, Monterey, CA, 2001.
54. S. Osher, M. Burger, D. Goldfarb, J. Xu, and W. Yin, An iterated regularization method for total variation based image restoration, SIAM Journal on Multiscale Modeling and Simulation, 4:2 (2005), 460-489.
55. S. Osher, Y. Mao, B. Dong, and W. Yin, Fast linearized Bregman iteration for compressive sensing and sparse denoising, Rice University CAAM Technical Report TR08-07, 2008.
56. C. C. Paige and M. A. Saunders, Lsqr: An algorithm for sparse linear equations and sparse least squares, ACM T. Math. Soft-mart., 8:1 (1982), 43-71.
57. J.-S. Pang, A posteriori error bounds for the linearly-constrained variational inequality problem, Math. Method. Oper. Res., 12 (1987), 474-484.
58. T. H. Reeves and N. G. Kingsbury, Overcomplete image coding using iterative projection-based noise shaping, 2002 International Conference on Image Processing, 3 (2002), 24-28.
59. M. Rudelson and R. Vershynin, Geometric approach to error correcting codes and reconstruction of signals, International Mathematical Research Notices, 64 (2005), 4019-4041.
60. F. Santosa and W. Symes, Linear inversion of band-limited reflection histograms, SIAM J. Sci. Comput, 7 (1986), 1307-1330.
61. 1-regularized logistic regression, In preparation, 2008.
62. D. Takhar, J. Laska, M. Wakin, M. Duarte, D. Baron, S. Sarvotham, K. Kelly, and R. Baraniuk, A new compressive imaging camera architecture using optical-domain compression, In Proceedings of Computational Imaging IV at SPIE Electronic Image, San Jose, California, 2006.
63. 1 norm, Geophysics, 44 (1979), 39-52.
64. R,. Tibshirani, Regression shrinkage and selection via the lasso, J. Roy. Stat. Soc. B, 58 (1996), 267-288.
65. J. Tropp, Just relax: Convex programming methods for identifying sparse signals, IEEE T. Inform. Theory, 51 (2006), 1030-1051.
66. J. Tropp and A. Gilbert, Signal recovery from partial information via orthogonal matching pursuit, IEEE T. Inform. Theory, 53:12 (2007), 4655-4666.
67. J. Tropp, M. Wakin, M. Duarte, D. Baron, and R. Baraniuk, Random filters for compressive sampling and reconstruction, In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Toulouse, France, 2006.
68. Y. Tsaig and D. Donoho, Extensions of compressed sensing, Signal Process., 86:3 (2005), 533-548.
69. P. Tseng, Convergence of block coordinate descent method for nondifferentiable minimization, J. Optimiz. Theory Appl., 109:3 (2001), 1573-2878.
70. P. Tseng and S. Yun, A coordinate gradient descent method for nonsmooth separable minimization, Math. Program., 117:1-2 (2009), 387-423.
71. E. Van den Berg and M. P. Friedlander, SPGL1: A MATLAB solver for large-scale sparse reconstruction, http://www. cs .ubc.ca/labs/sc1/index.php/main/spg11. 2007.
72. E. van den Berg, M. P. Friedlander, G. Hennenfent, F. J. Herrmann, Y. Saad, and P. Yilmaz, Sparco: A testing framework for sparse reconstruction. UBC Computer Science Technical Report TR-2007-20, 2007.
73. M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, and R. Baraniuk, An architecture for compressiving image, In Proceedings of the International Conference on Image Processing (ICIP), Atlanta, Georgia, 2006.
74. M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, and R. Baraniuk, Compressive imaging for video representation and coding, In Proceedings of Picture Coding Symposium (PCS), Beijing, China, 2006.
75. Z. Wen, W. Yin, D. Goldfarb, and Y. Zhang, A fast algorithm for sparse reconstruction based on shrinkage, subspace optimization and continuation, Submitted to SIAM Journal on Scientific Computing, Rice University CAAM Technical Report TR09-01, 2009.
76. S. Wright, R. Nowak, and M. Figueiredo, Sparse reconstruction by separable approximation, Submitted, 2008.
77. J. Yang, Y. Zhang, and W. Yin, A fast TVL1-L2 algorithm for image reconstruction from partial fourier data, Submitted to IEEE Journal of Selected Topics in Signal Processing Special Issue on Compressed Sensing. Rice University CAAM Technical Report TR08-27, 2008.
78. 1-minimization with applications to compressed sensing, SIAM Journal on Imaging Sciences, 1:1 (2008), 143-168.
79. S. Yun and K.-C. Toh, A coordinate gradient descent method for L1-regularized convex minimization, Optimization-Online, 2008.
80. H. Zhang and W. W. Hager, A nonmonotone line search technique and its application to unconstrained optimization, SIAM Journal on Optimization, 14:4 (2004), 1043-1056.
81. Y. Zhang, When is missing data recoverable? Rice University CAAM Technical Report TR06-15, 2006.