Sparse signal estimation by maximally sparse convex optimization

IEEE Transactions on Signal Processing

Volumn 62, Issue 5, 2014, Pages 1078-1092

  Selesnick, Ivan W ^a   Bayram, Ilker ^b  

^a POLYTECHNIC UNIVERSITY   (United States)

^b ISTANBUL TECHNICAL UNIVERSITY   (Turkey)

Author keywords

basis pursuit; Convex optimization; deconvolution; L1 norm; lasso; non convex optimization; sparse optimization; sparse regularization; threshold function

Indexed keywords

EID: 84894593844     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2014.2298839     Document Type: Article

References (73)

1. A. Achim, P. Tsakalides, and A. Bezerianos, "SAR image denoising via Bayesian wavelet shrinkage based on heavy-tailed modeling," IEEE Trans. Geosci. Remote Sens., vol. 41, no. 8, pp. 1773-1784, Aug. 2003.
2. A. Antoniou and W. S. Lu, Practical Optimization: Algorithms and Engineering Applications. New York, NY, USA: Springer, 2007.
3. A.M. Atto, D. Pastor, and G. Mercier, "Wavelet shrinkage: Unification of basic thresholding functions and thresholds," Signal, Image, Video Process., vol. 5, pp. 11-28, 2011.
4. F. Bach, R. Jenatton, J. Mairal, and G. Obozinski, "Optimization with sparsity-inducing penalties," Found. Trends Mach. Learn., vol. 4, no. 1, pp. 1-106, 2012.
5. R. G. Baraniuk, E. Candes, M. Elad, and Y. Ma, "Special issue on applications of sparse representation and compressive sensing," Proc. IEEE, vol. 98, no. 6, Jun. 2010.
6. A. Blake andA. Zisserman, Visual Reconstruction. Cambridge,MA, USA: MIT Press, 1987.
7. A. Blumensath, "Accelerated iterative hard thresholding," Signal Process., vol. 92, no. 3, pp. 752-756, 2012.
8. T. Blumensath and M. E. Davies, "Normalized iterative hard thresholding: Guaranteed stability and performance," IEEE. J. Sel. Top. Signal Process., vol. 4, no. 2, pp. 298-309, Apr. 2010.
9. S. Boyd,N. Parikh, E. Chu, B. Peleato, and J. Eckstein, "Distributed optimization and statistical learning via the alternating direction method of multipliers," Found. Trends Mach. Learn., vol. 3, no. 1, pp. 1-122, 2011.
10. M. S. O'Brien, A. N. Sinclair, and S. M. Kramer, "Recovery of a sparse spike time series by L1 norm deconvolution," IEEE Trans. Signal Process., vol. 42, no. 12, pp. 3353-3365, Dec. 1994.
11. E. J. Candès, M. B. Wakin, and S. Boyd, "Enhancing sparsity by reweighted l1 minimization," J. Fourier Anal. Appl., vol. 14, no. 5, pp. 877-905, Dec. 2008.
12. P. Charbonnier, L. Blanc-Feraud, G. Aubert, and M. Barlaud, "Deterministic edge-preserving regularization in computed imaging," IEEE Trans. Image Process., vol. 6, no. 2, pp. 298-311, Feb. 1997. (Pubitemid 127766386)
13. R. Chartrand, "Fast algorithms for nonconvex compressive sensing: MRI reconstruction from very few data," in Proc. IEEE Int. Symp. Biomed. Imag. (ISBI), July 2009, pp. 262-265.
14. P.-Y. Chen and I. W. Selesnick, "Group-sparse signal denoising: Non-convex regularization, convex optimization," Aug. 2013 [Online]. Available: http://arxiv.org/abs/1308.5038
15. S. Chen, D. L. Donoho, and M. A. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Sci. Comput., vol. 20, no. 1, pp. 33-61, 1998. (Pubitemid 128689501)
16. J. F. Claerbout and F. Muir, "Robust modeling of erratic data," Geophysics, vol. 38, no. 5, pp. 826-844, 1973.
17. M. Clyde and E. I. George, "Empirical Bayes estimation in wavelet nonparametric regression," in Bayesian Inference in Wavelet Based Models, P. Muller and B. Vidakovic, Eds. New York, NY, USA: Springer-Verlag, 1999, pp. 309-322.
18. P. L. Combettes and J.-C. Pesquet, "Proximal thresholding algorithm for minimization over orthonormal bases," SIAM J. Optim., vol. 18, no. 4, pp. 1351-1376, 2008.
19. P. L. Combettes and J.-C. Pesquet, "Proximal splitting methods in signal processing," in Fixed-Point Algorithms for Inverse Problems in Science and Engineering, H.H. Bauschke, Ed. et al. New York, NY, USA: Springer-Verlag, 2011, pp. 185-212.
20. D. Donoho, A. Maleki, and M. Shahram,Wavelab 850 [Online]. Available: http://www-stat.stanford.edu/%7Ewavelab/
21. D. L. Donoho and I. M. Johnstone, "Ideal spatial adaptation by wavelet shrinkage," Biometrika, vol. 81, no. 3, pp. 425-455, 1994.
22. E. Esser, X. Zhang, and T. F. Chan, "A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science," SIAM J. Imag. Sci., vol. 3, no. 4, pp. 1015-1046, 2010.
23. J. M. Fadili and L. Boubchir, "Analytical form for a Bayesian wavelet estimator of images using the Bessel K form densities," IEEE Trans. Image Process., vol. 14, no. 2, pp. 231-240, Feb. 2005. (Pubitemid 40181981)
24. J. Fan and R. Li, "Variable selection via nonconcave penalized likelihood and its oracle properties," J. Amer. Statist. Assoc., vol. 96, no. 456, pp. 1348-1360, 2001. (Pubitemid 33695585)
25. M. Figueiredo, J. Bioucas-Dias, and R. Nowak, "Majorization- minimization algorithms forwavelet-based image restoration," IEEE Trans. Image Process., vol. 16, no. 12, pp. 2980-2991, Dec. 2007. (Pubitemid 350201231)
26. M. Figueiredo and R. Nowak, "Wavelet-based image estimation: An empirical Bayes approach using Jeffrey's noninformative prior," IEEE Trans. Image Process., vol. 10, no. 9, pp. 1322-1331, Sept. 2001. (Pubitemid 32949956)
27. M. A. T. Figueiredo, R. D. Nowak, and S. J.Wright, "Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems," IEEE J. Sel. Top. Signal Process., vol. 1, no. 4, pp. 586-598, Dec. 2007.
28. S. Foucart, "Hard thresholding pursuit: An algorithm for compressive sensing," SIAM J. Numer. Anal., vol. 49, no. 6, pp. 2543-2563, 2010.
29. J.-J. Fuchs, "On sparse representations in arbitrary redundant bases," IEEE Trans. Inf. Theory, vol. 50, no. 6, pp. 1341-1344, 2004.
30. J. J. Fuchs, "Identification of real sinusoids in noise, the globalmatched filter approach," in Proc. 15th IFAC Symp. Syst. Identif., Saint-Malo, France, Jul. 2009, pp. 1127-1132.
31. H. Gao, "Wavelet shrinkage denoising using the nonnegative garrote," J. Comput. Graph. Statist., vol. 7, pp. 469-488, 1998. (Pubitemid 128635037)
32. H.-Y. Gao andA.G. Bruce, "Waveshrinkwith firm shrinkage," Statistica Sinica, vol. 7, pp. 855-874, 1997.
33. G. Gasso, A. Rakotomamonjy, and S. Canu, "Recovering sparse signals with a certain family of nonconvex penalties and DC programming," IEEE Trans. Signal Process., vol. 57, no. 12, pp. 4686-4698, Dec. 2009.
34. D. Geman and Y. Chengda, "Nonlinear image recovery with half-quadratic regularization," IEEE Trans. Image Process., vol. 4, no. 7, pp. 932-946, 1995.
35. A. Gholami and S. M. Hosseini, "A general framework for sparsitybased denoising and inversion," IEEE Trans. Signal Process., vol. 59, no. 11, pp. 5202-5211, Nov. 2011.
36. T. Goldstein and S. Osher, "The split Bregman method for L1-regularized problems," SIAM J. Imag. Sci., vol. 2, no. 2, pp. 323-343, 2009.
37. I. F. Gorodnitsky and B. D. Rao, "Sparse signal reconstruction from limited data using FOCUSS: A re-weighted minimum norm algorithm," IEEE Trans. Signal Process., vol. 45, no. 3, pp. 600-616, Mar. 1997. (Pubitemid 127765947)
38. G. Harikumar and Y. Bresler, "A new algorithm for computing sparse solutions to linear inverse problems," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), May 1996, vol. 3, pp. 1331-1334.
39. A. Hyvärinen, "Sparse code shrinkage: Denoising of non-Gaussian data by maximum likelihood estimation," Neural Comput., vol. 11, pp. 1739-1768, 1999.
40. S. Ji, Y. Xue, and L. Carin, "Bayesian compressive sensing," IEEE Trans. Signal Process., vol. 56, no. 6, pp. 2346-2356, June 2008. (Pubitemid 351795888)
41. K. F. Kaaresen, "Deconvolution of sparse spike trains by iterated window maximization," IEEE Trans. Signal Process., vol. 45, no. 5, pp. 1173-1183, May 1997. (Pubitemid 127766066)
42. N. Kingsbury and T. Reeves, "Redundant representation with complex wavelets: How to achieve sparsity," in Proc. IEEE Int. Conf. Image Processing, 2003, vol. 1, pp. 45-48.
43. I. Kozlov andA. Petukhov, ,W. Freeden, Ed. et al., "Sparse solutions of underdetermined linear systems," in Handbook of Geomathematics. New York, NY, USA: Springer, 2010.
44. D. A. Lorenz, "Non-convex variational denoising of images: Interpolation between hard and soft wavelet shrinkage," Current Develop. Theory Appl. Wavelets, vol. 1, no. 1, pp. 31-56, 2007.
45. S. Mallat, AWavelet Tour of Signal Processing. NewYork,NY,USA: Academic, 1998.
46. H. Mohimani, M. Babaie-Zadeh, and C. Jutten, "A fast approach for overcomplete sparse decomposition based on smoothed l0 norm," IEEE Trans. Signal Process., vol. 57, no. 1, pp. 289-301, Jan. 2009.
47. N. Mourad and J. P. Reilly, "Minimizing nonconvex functions for sparse vector reconstruction," IEEE Trans. Signal Process., vol. 58, no. 7, pp. 3485-3496, Jul. 2010.
48. S. Nadarajah and S. Kotz, "The BKF Bayesian wavelet estimator," Signal Process., vol. 87, no. 9, pp. 2268-2271, Sep. 2007. (Pubitemid 46686801)
49. M. Nikolova, "Estimation of binary images by minimizing convex criteria," in Proc. IEEE Int. Conf. Image Process., 1998, vol. 2, pp. 108-112.
50. M. Nikolova, "Markovian reconstruction using a GNC approach," IEEE Trans. Image Process., vol. 8, no. 9, pp. 1204-1220, 1999.
51. M. Nikolova, J. Idier, and A. Mohammad-Djafari, "Inversion of largesupport ill-posed linear operators using a piecewise Gaussian MRF," IEEE Trans. Image Process., vol. 7, no. 4, pp. 571-585, 1998. (Pubitemid 128745372)
52. M. Nikolova, M. Ng, S. Zhang, andW. Ching, "Efficient reconstruction of piecewise constant images using nonsmooth nonconvex minimization," SIAM J. Imag. Sci., vol. 1, no. 1, pp. 2-25, 2008.
53. M. Nikolova and M. K. Ng, "Analysis of half-quadratic minimization methods for signal and image recovery," SIAM J. Sci. Comput., vol. 27, no. 3, pp. 937-966, Oct. 2005.
54. M. Nikolova, M. K. Ng, and C.-P. Tam, "Fast nonconvex nonsmooth minimization methods for image restoration and reconstruction," IEEE Trans. Image Process., vol. 19, no. 12, pp. 3073-3088, Dec. 2010.
55. J. Portilla and L. Mancera, "L0-based sparse approximation: Two alternative methods and some applications," in Proc. SPIE, San Diego, CA, USA, 2007, vol. 6701, (Wavelets XII).
56. J. Portilla, V. Strela, M. J. Wainwright, and E. P. Simoncelli, "Image denoising using scale mixtures of Gaussians in the wavelet domain," IEEE Trans. Image Process., vol. 12, no. 11, pp. 1338-1351, Nov. 2003.
57. K. Qiu and A. Dogandzic, "Sparse signal reconstruction via ECME hard thresholding," IEEE Trans. Signal Process., vol. 60, no. 9, pp. 4551-4569, Sep. 2012.
58. B. D. Rao, K. Engan, S. F. Cotter, J. Palmer, and K. Kreutz-Delgado, "Subset selection in noise based on diversity measure minimization," IEEE Trans. Signal Process., vol. 51, no. 3, pp. 760-770, Mar. 2003.
59. I. Selesnick, "Penalty and shrinkage functions for sparse signal processing," Connexions Web site, 2012 [Online]. Available: http://cnx. org/content/m45134/1.1/
60. I. Selesnick, Sparse deconvolution (an MM algorithm), Connexions Website, 2012 [Online]. Available: http://cnx.org/content/m44991/1.4/
61. I. W. Selesnick, S. Arnold, and V. R.Dantham, "Polynomial smoothing of time series with additive step discontinuities," IEEE Trans. Signal Process., vol. 60, no. 12, pp. 6305-6318, Dec. 2012.
62. I.W. Selesnick, H. L. Graber, D. S. Pfeil, and R. L. Barbour, "Simultaneous low-pass filtering and total variation denoising," IEEE Trans. Signal Process., 2014, preprint, doi: 10.1109/TSP.2014.2298836.
63. C. Soussen, J. Idier, D. Brie, and J. Duan, "From Bernoulli-Gaussian deconvolution to sparse signal restoration," IEEE Trans. Signal Process., vol. 59, no. 10, pp. 4572-4584, Oct. 2011.
64. J. F. Sturm, "Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones," Optim. Methods Softw., vol. 11-12, pp. 625-653, 1999 [Online]. Available: http://coral.ie.lehigh.edu/%7Enewsedumi/, Version 1.05 available from
65. X. Tan, W. Roberts, J. Li, and P. Stoica, "Sparse learning via iterative minimization with application to MIMO radar imaging," IEEE Trans. Signal Process., vol. 59, no. 3, pp. 1088-1101, Mar. 2011.
66. H. L. Taylor, S. C. Banks, and J. F. McCoy, "Deconvolution with the l1 norm," Geophysics, vol. 44, no. 1, pp. 39-52, 1979.
67. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Roy. Statist. Soc., Ser. B, vol. 58, no. 1, pp. 267-288, 1996.
68. Y. Wang and W. Yin, "Sparse signal reconstruction via iterative support detection," SIAM J. Imag. Sci., vol. 3, no. 3, pp. 462-491, 2010.
69. D.Wipf and S. Nagarajan, "Iterative reweighted and methods for finding sparse solutions," IEEE. J. Sel. Top. Signal Process., vol. 4, no. 2, pp. 317-329, Apr. 2010.
70. X.-P. Zhang, "Thresholding neural network for adaptive noise reduction," IEEE Trans. Neural Netw., vol. 12, no. 3, pp. 567-584, May 2001. (Pubitemid 32583556)
71. X.-P. Zhang and M. D. Desai, "Adaptive denoising based on SURE risk," IEEE Signal Process. Lett., vol. 5, no. 10, pp. 265-267, Oct. 1998. (Pubitemid 128556770)
72. Z.-D. Zhao, "Wavelet shrinkage denoising by generalized threshold function," in Proc. Int. Conf.Mach. Learn. Cybern., Aug. 2005, vol. 9, pp. 5501-5506. (Pubitemid 41734891)
73. H. Zou and R. Li, "One-step sparse estimates in nonconcave penalized likelihood models," Ann. Statist., vol. 36, no. 4, pp. 1509-1533, 2008.