Sparse reconstruction by separable approximation

IEEE Transactions on Signal Processing

Volumn 57, Issue 7, 2009, Pages 2479-2493

  Wright, Stephen J ^a   Nowak, Robert D ^b   Figueiredo, Mário A T ^c  

^a University of Wisconsin   (United States)

^b Wisconsin Electric Machines and Power Electronics Consortium   (United States)

^c INSTITUTO SUPERIOR TÉCNICO   (Portugal)

Author keywords

Compressed sensing; Optimization; Reconstruction; Sparse approximation

Indexed keywords

ALGORITHMIC FRAMEWORK; APPROXIMATE SOLUTION; BASIS PURSUITS; COMMON PROBLEMS; COMPRESSED SENSING; CONVEX FUNCTIONS; ERROR TERMS; ITERATIVE ALGORITHM; LEAST ABSOLUTE SHRINKAGE AND SELECTION OPERATORS; LINEAR SYSTEMS OF EQUATIONS; NON-SMOOTH; NONCONVEX; OBJECTIVE FUNCTIONS; RECONSTRUCTION; REGULARIZER; SEPARABLE APPROXIMATION; SOLUTION TECHNIQUES; SPARSE APPROXIMATION; SPARSE RECONSTRUCTION; WAVELET-BASED DECONVOLUTION;

CONVERGENCE OF NUMERICAL METHODS; LINEAR SYSTEMS; OPTIMIZATION; REPAIR; SIGNAL RECONSTRUCTION; STANDARDS;

ITERATIVE METHODS;

EID: 67650178787     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2009.2016892     Document Type: Article

References (81)

1. J. Barzilai and J. Borwein, "Two point step size gradient methods," IMA J. Numer. Anal., vol. 8, no. 1, pp. 141-148, 1988.
2. J. Bioucas-Dias and M. Figueiredo, "A new TwIST: Two-step iterative shrinkage/thresholding algorithms for image restoration," IEEE Trans. Image Process., vol. 16, no, 12, pp. 2992-3004, Dec. 2007.
3. E. Birgin, J. Martinez, and M. Raydan, "Nonmonotone spectral projected gradient methods on convex sets," SIAM J. Optim., vol. 10, no. 4, pp. 1196-1211, 2000.
4. T. Blumensath and M. Davies," Gradient pursuits," IEEE Trans. Signal Process., 2008, submitted for publication.
5. A. Bolstad, B. Van Veen, and R. Nowak, "Space-time sparsity regularization for the magnetoencephalography inverse problem," in Proc. IEEE Int. Conf. Biomed. Imag., Arlington, VA, 2007, pp. 984-987.
6. A. Bolstad, B. Van Veen, R. Nowak, and R. Wakai, "An expectation-maximization algorithm for space-time sparsity regularization of the MEG inverse problem," in Proc. Int. Conf. Biomagn., Vancouver, BC, Canada, 2006.
7. A. Bolstad, B. Van Veen, and R, Nowak, "Magneto-/electroencephalography with space-time sparse priors," in Proc. IEEE Statist. Signal Process. Workshop, Madison, WI, 2007, pp. 190-194.
8. S. Bourguignon, H. Carfantan, and J. Idier, "A sparsity-based method for the estimation of spectral lines from irregularly sampled data," IEEE J. Sel. Topics Signal Process., vol. 1, no. 4, pp. 575-585, Dec. 2007.
9. E. Candès, J. Romberg, and T. Tao, "Stable signal recovery from incomplete and inaccurate information," Commun. Pure Appl. Math., vol. 59, no. 8, pp. 1207-1233, Aug. 2005.
10. E. Candès, J. Romberg, and T. Tao, "Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information," IEEE Trans. Inf. Theory, vol. 52, no. 2, pp. 489-509, Feb. 2006.
11. E. Candès and T. Tao, "The Dantzig selector: Statistical estimation when p is much larger than n," Ann. Stat., vol. 35, no. 6, pp. 2313-2351, Dec. 2007.
12. A. Chambolle, "An algorithm for total variation minimization and applications," J. Math. Imag. Vision, vol. 20, no. 1-2, pp. 89-97, Jan.-Mar. 2004.
13. T. Chan, S. Esedoglu, F. Park, and A. Yip, "Recent developments in total variation image restoration," in Mathematical Models of Computer Vision, N, Paragios, Y. Chen, and O. Faugeras, Eds. New York: Springer-Verlag, 2005.
14. C. Chaux, P. Combettes, J.-C. Pesquet, and V. Wajs, "A variational formulation for frame-based inverse problems," Inverse Probl., vol. 23, no. 4, pp. 1495-1518, Aug, 2007.
15. S. Chen, D. Donoho, and M. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Sci. Comput., vol. 20, no. 1, pp. 33-61, 1998.
16. J. Claerbout and F. Muir, "Robust modelling of erratic data," Geophysics, vol. 38, no. 5, pp. 826-844, 1973.
17. P. Combettes and V. Wajs, "Signal recovery by proximal forward-backward splitting," SIAM J. Multiscale Model. Simul., vol. 4, no. 4, pp. 1168-1200, 2005.
18. Y.-H. Dai and R. Fletcher, "Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming," Numer. Math., vol. 100, no. 1, pp. 21-47, Mar. 2005.
19. Y.-H. Dai, W. Hager, K. Schittkowski, and H. Zhang, "The cyclic Barzilai-Borwein method for unconstrained optimization," IMA J. Numer. Anal., vol. 26, no. 3, pp. 604-427, Mar. 2006.
20. J. Darbon and M. Sigelle, "Image restoration with discrete constrained total variation; part I: Fast and exact optimization," J. Math. Imag. Vision, vol. 26, no. 3, pp. 261-276, Dec. 2006.
21. I. Daubechies, M. Defrise, and C. De Mol, "An iterative thresholding algorithm for linear inverse problems with a sparsity constraint," Commun. Pure Appl. Math., vol. 57, pp. 1413-1457, Nov. 2004.
22. I. Daubechies, M. Fornasier, and I. Loris, "Accelerated projected gradient method for linear inverse problems with sparsity constraints," J. Fourier Anal. Appl., vol. 14, no. 5-6, pp. 764-792, Dec. 2008.
23. G. Davis, S. Mallat, and M. Avellaneda, "Aadaptive greedy approximations," Constr. Approx., vol. 13, no. 1, pp. 57-98, Mar. 1997.
24. A. Dempster, N. Laird, and D. Rubin, "Maximum likelihood estimation from incomplete data via the EM algorithm," J. Roy. Stat. Soc. B. vol. 39, no. 1, pp. 1-38, Jan. 1977.
25. P. Djurić, "A model selection role for sinusoids in white Gaussian noise," IEEE Trans. Signal Process., vol. 44, no. 7, pp. 1744-1751, Jul. 1996.
26. D. Donoho, "De-noising by soft thresholding," IEEE Trans. Inf. Theory, vol. 41, no. 3, pp. 613-627, May 1995.
27. D. Donobo, "Compressed sensing," IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289-1306, Apr. 2006.
28. D. Donoho, M. Elad, and V. Temlyakov, "Stable recovery of sparse overcomplete representations in the presence of noise," IEEE Trans. Inf. Theory, vol. 52, no. 1, pp. 6-18, Jan. 2006.
29. D. Donoho and Y. Tsaig, "Fast solution of L1-norm minimization problems when the solution may be sparse," Dept. Statistics, Stanford Univ., Stanford, CA, Tech. Rep. 2006-18, 2006.
30. D. Donoho, Y. Tsaig, I. Drori, and J.-L. Starck, "Sparse solution of underdetermined linear equations by stagewise orthogonal matching pursuit," IEEE Trans. Inf. Theory, 2007, submitted for publication.
31. J. Duchi, S. Shalev-Shwartz, Y. Singer, and T. Chandra, "Efficient projections onto the L1-ball for learning in high dimensions," in Proc. Int. Conf. Mach. Learn. (ICML), Helsinki, Finland, 2008.
32. M. Elad, "Why simple shrinkage is still relevant for redundant representations?," IEEE Trans. Inf. Theory, vol. 52, no. 12, pp. 5559-5569, Dec. 2006.
33. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, "Least angle regression" Ann. Stat., vol. 32, no. 2, pp. 407-499, Apr. 2004.
34. M. Elad, B. Matalon, and M. Zibulevsky, "Image denoising with shrinkage and redundant representations," in Proc. IEEE Conf. Comput. Vision Pattern Recogn. (CVPR' 2006), New York, 2006.
35. M. Figueiredo and R. Nowak, "Wavelet-based image estimation: An empirical Bayes' approach using Jeffreys' noninformative prior," IEEE Trans. Image Process., vol. 10, no. 9, pp. 1322-1331, Sep. 2001.
36. M. Figueiredo and R. Nowak, "An EM algorithm for wavelet-based image restoration," IEEE Trans. Image Process., vol. 12, no. 8, pp. 906-916, Aug. 2003.
37. M. Figueiredo, J. Bioucas-Dias, and R. Nowak, "Majorization-minimization algorithms for wavelet-based image restoration," IEEE Trans. Image Process., vol. 16, no. 12, pp. 2980-2991, Dec. 2007.
38. M. Figueiredo, J. Bioucas-Dias, J. Oliveira, :aNd R. Nowak, "On total-variation denoising: A new majorization-minimization algorithm and an experimental comparison with wavelet denoising," in Proc. IEEE Int. Conf. Image Process. (ICIP), 2006.
39. M. Figueiredo, R. Nowak, and S. Wright, "Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems," IEEE J. Sel. Top. Signal Process., vol. 1, no. 4, pp. 586-598, Dec. 2007.
40. J. Friedman, T. Hastie, H. Hofling, and R. Tibshirani, "Pathwise coordinate optimization," Ann. Appl. Stat., vol. 1, no. 2, pp. 302-332, 2007.
41. J. Fuchs, "More on sparse representations in arbitrary bases," in Proc. 13th IFAC-IFORS Symp. Identification System Parameter Estimation Rotterdam, 2003, vol. 2, pp. 1357-1362.
42. J. Fuchs, "Convergence of a sparse representations algorithm applicable to real or complex data," IEEE J. Sel. Topics Signal Process., vol. 1, no. 12, pp. 598-605, Dec. 2007.
43. H. Gao, "Wavelet shrinkage denoising using the non-negative garrote," J. Comput. Graph. Stat., vol. 7, no. 4, pp. 469-488, 1998.
44. H. Gao and A. Bruce, "Waveshrink with firm shrinkage," Statistica Sinica, vol. 7, no. 4, pp. 855-874, Oct. 1997.
45. D. Goldfarb and W. Yin, "Parametric maximum flow algorithms for fast total variation minimization," Dept. Computat. Appl. Math., Rice Univ., Houston, TX, Tech. Rep. TR07-09, 2007.
46. G. Golub and C. Van Loan, Matrix Computations, 3rd ed. Baltimore, MD: Johns Hopkins Univ. Press, 1996.
47. I. Gorodnitsky and B. Rao, "Sparse signal reconstruction from limited data using FOCUSS: A recursive weighted norm minimization algorithm" IEEE Trans. Signal Process., vol. 45, no. 3, pp. 600-616, Mar. 1997.
48. R. Griesse and D. Lorenz, "A semismooth Newton method for Tikhonov functionals with sparsity constraints," Inverse Probl., vol. 24, no. 3, 2008.
49. L. Grippo, F. Lampariello, and S. Lucidi, "A nonmonotone line search technique for Newton's method," SIAM J. Numer. Anal., vol. 23, no. 4, pp. 706-716, 1986.
50. L. Grippo and M. Sciandrone, "Nonmonotone globalization techniques for the Barzilai-Borwein method," Comput. Optim. Appl., vol. 23, no. 2, pp. 143-169, Nov. 2002.
51. 1-regularized minimization with applications to compressed sensing," Dept. Computat. Appl. Math., Rice Univ., Houston, TX, Tech. Rep. TR07-07, 2007.
52. J. Haupt and R. Nowak, "Signal reconstruction from noisy random projections," IEEE Trans. Inf. Theory, vol. 52, no. 9, pp. 4036-4048, Sep. 2006.
53. 1-regularized least squares," IEEE J. Sel. Topics Signal Process., vol. 1, no. 4, pp. 606-617, Dec. 2007.
54. Y. Kim, J. Kim, and Y. Kim, "Blockwise sparse regression," Statistica Sinica, vol. 16, no. 2, pp. 375-390, Apr. 2006.
55. S. Levy and P. Fullagar, "Reconstruction of a sparse spike train from a portion of its spectrum and application to high-resolution deconvolution" Geophysics, vol. 46, no. 9, pp. 1235-1243, 1981.
56. D. Malioutov, M. Çetin, and A. Willsky, "Homotopy continuation for sparse signal representation," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., Philadelphia, PA, 2005, vol. 5, pp. 733-736.
57. D. Malioutov, M. Cetin, and A. Willsky, "Sparse signal reconstruction perspective for source localization with sensor arrays," IEEE Trans. Signal Process., vol. 53, no. 8, pp. 3010-3022, Aug. 2005.
58. L. Meier, S. van de Geer, and P. Buhlmann, "The group LASSO for logistic regression," J. Roy. Stat. Soc. B, vol. 70, no. 1, pp. 53-71, Jan. 2008.
59. A. Miller, Subset Selection in Regression. London, U.K.: Chapman and Hall, 2002,
60. P. Moulin and J. Liu, "Analysis of multiresolution image denoising schemes using generalized-Gaussian and complexity priors," IEEE Trans. Inf. Theory, vol. 45, no. 3, pp. 909-919, Apr. 1999.
61. Y. Nesterov, "Gradient methods for minimizing composite objective function," Center for Operations Research and Econometrics (CORE), Catholic Univ. Louvain, Louvain-la-Neuve, Belgium, CORE Discussion Paper 2007/76, 2007.
62. J. Nocedal and S. J. Wright, Numerical Optimization, 2nd ed. New York: Springer, 2006.
63. M. Osborne, B. Presnell, and B. Turlach, "A new approach to variable selection in least squares problems," IMA J. Numer. Anal., vol. 20, no. 3, pp. 389-403, 2000.
64. S. Osher, L. Rudin, and E. Fatemi, "Nonlinear total variation based noise removal algorithms," Physica D, vol. 60, no. 1-4, pp, 259-268, 1992.
65. R. T. Rockafellar, Convex Analysis. Princeton, NJ: Princeton Univ. Press, 1970.
66. F. Santosa and W. Symes, "Linear invesion of band-limited reflection histograms," SIAM J. Sci. Stat. Comput., vol. 7, no. 4, pp. 1307-1330, 1986.
67. M. A. Saunders, PDCO: Primal-Dual Interior-Point Method for Convex Objectives. Stanford, CA: Systems Optimization Lab., Stanford Univ. Press, 2002.
68. T. Serafini, G. Zanghirati, and L. Zanni, "Gradient projection methods for large quadratic programs and applications in training support vector machines," Optim. Meth. Softw., vol. 20, no. 2-3, pp. 353-378, Apr. 2005.
69. 1 norm," Geophysics, vol. 44, no. 1, pp. 39-52, 1979.
70. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Roy. Stat. Soc. B, vol. 58, no. 1, pp. 267-288, 1996.
71. J. Tropp, "Just relax: Convex programming methods for identifying sparse signals," IEEE Trans. Inf. Theory, vol. 51, no. 3, pp. 1030-1051, Apr. 2006.
72. J. Tropp, "Greed is good: Algorithmic results for sparse approximation," IEEE Trans. Inf. Theory, vol. 50, no. 10, pp. 2231-2242, Oct. 2004.
73. P. Tseng,"Convergence of a block coordinate descent method for nondifferentiable minimization," J. Optim. Theory Appl., vol. 109, no. 3, pp. 475-494, Jun. 2001.
74. B. Turlach, W. N. Venables, and S. J. Wright, "Simultaneous variable selection," Technometrics, vol. 47, no. 3, pp. 349-363, Aug. 2005.
75. E. van den Berg and M. P. Friedlander, "In pursuit of a root," Dept. Comput. Sci., Univ. British Columbia, BC, Canada, Tech. Rep. TR-20077-19, 2007.
76. S. Weisberg, Applied Linear Regression. New York: Wiley, 1980.
77. 1-minimization with applications to compressed sensing" SIAM J. Imag. Sci., vol. 1, no. 1, pp. 143-168, 2008.
78. M. Yuan and Y. Lin, "Model selection and estimation in regression with grouped variables," J. Roy. Stat. Soc. B, vol. 68, no, 1, pp. 49-67, Feb. 2006.
79. P. Zhao, G. Rocha, and B. Yu, "Grouped and hierarchical model selection through composite absolute penalties," Statistics Dept., Univ, California, Berkeley, 2007, TR 703.
80. P. Zhao and B. Yu, "Boosted LASSO," Statistics Dept., Univ. California, Berkeley, 2004.
81. C. Zhu, "Stable recovery of sparse signals via regularized minimization" IEEE Trans. Inf. Theory, vol. 54, no. 7, pp. 3364-3367, Jul. 2008.