Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems

IEEE Journal on Selected Topics in Signal Processing

Volumn 1, Issue 4, 2007, Pages 586-597

  Figueiredo, Mário A T ^a   Nowak, Robert D ^b   Wright, Stephen J ^c  

^a INSTITUTO SUPERIOR TÉCNICO   (Portugal)

^b Wisconsin Electric Machines and Power Electronics Consortium   (United States)

^c University of Wisconsin   (United States)

Author keywords

Compressed sensing; Convex optimization; Deconvolution; Gradient projection; Quadratic programming; Sparse reconstruction; Sparseness

Indexed keywords

DECONVOLUTION; INVERSE PROBLEMS; LINEAR EQUATIONS; LINEAR SYSTEMS; QUADRATIC PROGRAMMING; WAVELET ANALYSIS;

COMPRESSED SENSING; GRADIENT PROJECTION; SPARSE RECONSTRUCTION;

SIGNAL PROCESSING;

EID: 39449126969     PISSN: 19324553     EISSN: None     Source Type: Journal    
DOI: 10.1109/JSTSP.2007.910281     Document Type: Article

References (61)

1. 2 norms with application to Bayesian estimation," IEEE Trans. Signal Processing, vol. 42, pp. 618-627, Mar. 1994.
2. J. Barzilai and J. Borwein, "Two point step size gradient methods," IMA J. Numer. Anal., vol. 8, pp. 141-148, 1988.
3. D. P. Bertsekas, Nonlinear Programming, 2nd ed. Boston, MA: Athena, 1999.
4. E. G. Birgin, J. M. Martinez, and M. Raydan, "Nonmonotone spectral projected gradient methods on convex sets," SIAM J. Optim., vol. 10, pp. 1196-1211, 2000.
5. T. Blumensath and M. Davies, Gradient Pursuits 2007 [Online]. Available: http://www.see.ed.ac.uk/~tblumens/papers
6. E. Candès, J. Romberg, and T. Tao, "Stable signal recovery from incomplete and inaccurate information," Commun. Pure Appl. Math., vol. 59, pp. 1207-1233, 2005.
7. E. Candès and T. Tao, "Near optimal signal recovery from random projections: Universal encoding strategies," IEEE Trans. Inform. Theory, vol. 52, pp. 5406-5425, Dec. 2006.
8. E. Candès and T. Tao, "The Dantzig selector: Statistical estimation when p is much larger than n," Ann. Statist. 2007 [Online]. Available: http://arxiv.org/abs/math.ST/0506081
9. E. Candès, J. Romberg, and T. Tao, "Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information," IEEE Trans. Inform. Theory, vol. 52, pp. 489-509, Feb. 2006.
10. A. Chambolle, "An algorithm for total variation minimization and applications," J. Math. Imag. Vis., vol. 20, pp. 89-97, 2004.
11. S. Chen, D. Donoho, and M. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Sci. Comput., vol. 20, pp. 33-61, 1998.
12. J. Claerbout and F. Muir, "Robust modelling of erratic data," Geophys., vol. 38, pp. 826-844, 1973.
13. P. Combettes and V. Wajs, "Signal recovery by proximal forward-backward splitting," SIAM J. Multiscale Model. Simul., vol. 4, pp. 1168-1200, 2005.
14. R. W. Cottle, J.-S. Pang, and R. E. Stone, The Linear Complementarity Problem. New York: Academic, 1993.
15. Y.-H. Dai and R. Fletcher, "Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming," Numerische Mathematik, vol. 100, pp. 21-47, 2005.
16. I. Daubechies, M. De Friese, and C. De Mol, "An iterative thresholding algorithm for linear inverse problems with a sparsity constraint," Commun. Pure Appl. Math., vol. 57, pp. 1413-1457, 2004.
17. G. Davis, S. Mallat, and M. Avellaneda, "Greedy adaptive approximation," J. Construct. Approx., vol. 12, pp. 57-98, 1997.
18. D. Donoho, "Compressed sensing," IEEE Trans. Inform. Theory, vol. 52, pp. 1289-1306, Apr. 2006.
19. D. Donoho, "De-noising by soft-thresholding," IEEE Trans. Inform. Theory, vol. 41, pp. 613-627, May 1995.
20. D. Donoho, M. Elad, and V. Temlyakov, "Stable recovery of sparse overcomplete representations in the presence of noise," IEEE Trans. Inform. Theory, vol. 52, pp. 6-18, Jan. 2006.
21. D. Donoho and Y. Tsaig, Fast Solution of L1-Norm Minimization Problems When the Solution May be Sparse Inst. Comput. Math. Eng., Stanford Univ., Stanford, CA, 2006 [Online]. Available: http://www.stanford.edu/ ~tsaig, Tech. Rep.
22. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, "Least angle regression," Ann. Statist., vol. 32, pp. 407-499, 2004.
23. M. Elad, "Why simple shrinkage is still relevant for redundant representations?," IEEE Trans. Inform. Theory, vol. 52, pp. 5559-5569, Dec. 2006.
24. M. Elad, B. Matalon, and M. Zibulevsky, "Image denoising with shrinkage and redundant representations," in Proc. IEEE Computer Soc. Conf. Computer Vision and Pattern Recognition - CVPR'2006, New York, 2006.
25. M. Figueiredo and R. Nowak, "An EM algorithm for wavelet-based image restoration," IEEE Trans. Image Processing, vol. 12, pp. 906-916, Aug. 2003.
26. M. Figueiredo and R. Nowak, "A bound optimization approach to wavelet-based image deconvolution," in Proc. IEEE Int. Conf. Image Processing-ICIP'2005, 2005.
27. M. Frank and P. Wolfe, "An algorithm for quadratic programming," Naval Res. Logist. Quar., vol. 3, pp. 95-110, 1956.
28. J. J. Fuchs, "Multipath time-delay detection and estimation," IEEE Trans. Signal Processing, vol. 47, pp. 237-243, Jan. 1999.
29. J. J. Fuchs, "More on sparse representations in arbitrary bases," IEEE Trans. Inform. Theory, vol. 50, pp. 1341-1344, 2004.
30. J. Gondzio and A. Grothey, A New Unblocking Technique to Warm-start Interior Point Methods Based on Sensitivity Analysis School Math., Univ. Edinburgh, Edinburgh, U.K., Tech. Rep. MS-06-005, 2006.
31. 1-Regularized Minimization With Applications to Compresses Sensing Dept. Computational and Applied Mathematics, Rice Univ., Houston, TX, Tech. Rep. TR07-07, 2007.
32. J. Haupt and R. Nowak, "Signal reconstruction from noisy random projections," IEEE Trans. Inform. Theory, vol. 52, pp. 4036-4048, Sep. 2006.
33. D. Hunter and K. Lange, "A tutorial on MM algorithms," Amer. Statistician, vol. 58, pp. 30-37, 2004.
34. A. N. Iusem, "On the convergence properties of the projected gradient method for convex optimization," Comput. Appl. Math., vol. 22, pp. 37-52, 2003.
35. E. John and E. A. Yildrrim, Implementation of Warm-Start Strategies in Interior-Point Methods for Linear Programming in Fixed Dimension Dept. Ind. Eng., Bilkent Univ., Ankara, Turkey, 2006.
36. 1 -Regularized Least Squares Problems With Applications in Signal Processing and Statistics Dept. Elect. Eng., Stanford Univ., 2007 [Online]. Available: http://www.stanford.edu/~boyd/111.html
37. S. Levy and P. Fullagar, "Reconstruction of a sparse spike train from a portion of its spectrum and application to high-resolution deconvolution," Geophys., vol. 46, pp. 1235-1243, 1981.
38. M. Lustig, D. Donoho, and J. Pauly, Sparse MRI: The Application of Compressed Sensing for Rapid MR Imaging 2007 [Online]. Available: http://www.stanford.edu/~mlustig/SparseMRI.pdf
39. D. Malioutov, M. Çetin, and A. Willsky, "Homotopy continuation for sparse signal representation," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing, Philadelphia, PA, 2005, vol. 5, pp. 733-736.
40. S. Mallat, A Wavelet Tour of Signal Processing. San Diego, CA: Academic, 1998.
41. A. Miller, Subset Selection in Regression. London, U.K.: Chapman & Hall, 2002.
42. B. Moghaddam, Y. Weiss, and S. Avidan, "Spectral bounds for sparse PCA: Exact and greedy algorithms," in Advances in Neural Information Processing Systems 18. Cambridge, MA: MIT Press, 2006, pp. 915-922.
43. J. Moré and G. Toraldo, "On the solution of large quadratic programming problems with bound constraints," SIAM J. Optim., vol. 1, pp. 93-113, 1991.
44. J. Nocedal and S. J. Wright, Numerical Optimization, 2nd ed. New York: Springer-Verlag, 2006.
45. R. Nowak and M. Figueiredo, "Fast wavelet-based image deconvolution using the EM algorithm," in Proc. 35th Asilomar Conf. Signals, Systems, Computers, Monterey, CA, 2001.
46. M. Osborne, B. Presnell, and B. Turlach, "A new approach to variable selection in least squares problems," IMA J. Numer. Anal., vol. 20, pp. 389-403, 2000.
47. S. Osher, L. Rudin, and E. Fatemi, "Nonlinear total variation based noise removal algorithms," Phys. D., vol. 60, pp. 259-268, 1992.
48. C. Paige and M. A. Saunders, "LSQR: An algorithm for sparse linear equations and sparse least squares," ACM Trans. Math. Softw., vol. 8, pp. 43-71, 1982.
49. R. T. Rockafellar, Convex Analysis. Princeton, NJ: Princeton Univ. Press, 1970.
50. F. Santosa and W. Symcs, "Linear inversion of band-limited reflection histograms," SIAM J. Sci. Statist. Comput., vol. 7, pp. 1307-1330, 1986.
51. M. A. Saunders, PDCO: Primal-Dual Interior-Point Method for Convex Objectives Systems Optimization Laboratory, Stanford Univ.. Stanford, CA, 2002 [Online]. Available: http://www.stanford.edu/group/SOL/
52. T. Serafini, G. Zanghirati, and L. Zanni, "Gradient projection methods for large quadratic programs and applications in training support vector machines," Optim. Meth. Softw., vol. 20, pp. 353-378, 2004.
53. 1 norm," Geophys., vol. 44, pp. 39-52, 1979.
54. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. R. Statist. Soc. B, vol. 58, pp. 267-288, 1996.
55. J. Tropp, "Just relax: Convex programming methods for identifying sparse signals," IEEE Trans. Inform. Theory, vol. 51, pp. 1030-1051, 2006.
56. J. Tropp, "Greed is good: Algorithmic results for sparse approximation," IEEE Trans. Inform. Theory, vol. 50, pp. 2231-2242, Oct. 2004.
57. 1 constraint," in Proc. American Statistical Association; Statistical Computing Section, Alexandria, VA, 2005, pp. 2572-2577.
58. B. Turlach, W. N. Venables, and S. J. Wright, "Simultaneous variable selection," Technometrics, vol. 27, pp. 349-363, 2005.
59. S. J. Wright, "Implementing proximal point methods for linear programming," J. Optim. Theory Applicat., vol. 65, pp. 531-554, 1990.
60. S. J. Wright, Primal-Dual Interior-Point Methods. Philadelphia, PA: SIAM, 1997.
61. E. A. Yildirim and S. J. Wright, "Warm-start strategies in interior-point methods for linear programming," SIAM J. Optim., vol. 12, pp. 782-810, 2002.