The convergence guarantees of a non-convex approach for sparse recovery

IEEE Transactions on Signal Processing

Volumn 62, Issue 15, 2014, Pages 3754-3767

  Chen, Laming ^a   Gu, Yuantao ^a  

^a TSINGHUA UNIVERSITY   (China)

Author keywords

approximate projection; convergence analysis; non convex optimization; projected generalized gradient method; Sparse recovery; sparseness measure; weak convexity

Indexed keywords

CONVEX OPTIMIZATION; GRADIENT METHODS; OPTIMIZATION;

APPROXIMATE PROJECTION; CONVERGENCE ANALYSIS; GENERALIZED GRADIENTS; NONCONVEX OPTIMIZATION; SPARSE RECOVERY; SPARSENESS MEASURE; WEAK CONVEXITY;

RECOVERY;

EID: 84904365506     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2014.2330349     Document Type: Article

References (51)

1. E. Candès, J. Romberg, and T. Tao, "Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, " IEEE Trans. Inf. Theory, vol. 52, no. 2, pp. 489-509, Feb. 2006. (Pubitemid 43174093)
2. D. Donoho, "Compressed sensing, " IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289-1306, Apr. 2006.
3. E. Candès and T. Tao, "Near-optimal signal recovery fromrandom projections: Universal encoding strategies?, " IEEE Trans. Inf. Theory, vol. 52, no. 12, pp. 5406-5425, Dec. 2006. (Pubitemid 46445381)
4. M. Elad and M. Aharon, "Image denoising via sparse and redundant representations over learned dictionaries, " IEEE Trans. Image Process., vol. 15, no. 12, pp. 3736-3745, Dec. 2006. (Pubitemid 44811686)
5. M. Lustig, D. Donoho, J. Santos, and J. Pauly, "Compressed sensing MRI, " IEEE Signal Process. Mag., vol. 25, no. 2, pp. 72-82, Mar. 2008.
6. M. Herman and T. Strohmer, "High-resolution radar via compressed sensing, " IEEE Trans. Signal Process., vol. 57, no. 6, pp. 2275-2284, Jun. 2009.
7. J. Tropp, J. Laska, M. Duarte, J. Romberg, and R. Baraniuk, "Beyond Nyquist: Efficient sampling of sparse bandlimited signals, " IEEE Trans. Inf. Theory, vol. 56, no. 1, pp. 520-544, Jan. 2010.
8. M. Mishali and Y. Eldar, "From theory to practice: Sub-Nyquist sampling of sparse wideband analog signals, " IEEE J. Sel. Topics Signal Process., vol. 4, no. 2, pp. 375-391, Apr. 2010.
9. S. Chen, D. Donoho, and M. Saunders, "Atomic decomposition by basis pursuit, " SIAM J. Sci. Comput., vol. 20, no. 1, pp. 33-61, Aug. 1998. (Pubitemid 128689501)
10. E. Candès, "The restricted isometry property and its implications for compressed sensing, " Comptes Rendus Mathematique, vol. 346, no. 9-10, pp. 589-592, May 2008.
11. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
12. A. Aldroubi, X. Chen, and A. Powell, "Stability and robustness of minimization using null space property, " in Proc. Int. Conf. Sampling Theory Appl., Singapore, May 2011.
13. I. Gorodnitsky and B. Rao, "Sparse signal reconstruction from limited data using FOCUSS: A re-weighted minimum norm algorithm, " IEEE Trans. Signal Process., vol. 45, no. 3, pp. 600-616, Mar. 1997. (Pubitemid 127765947)
14. R. Chartrand and W. Yin, "Iteratively reweighted algorithms for compressive sensing, " Proc. Int. Conf. Acoust., Speech, Signal Process. (ICASSP), pp. 3869-3872, Apr. 2008.
15. E. Candès, M.Wakin, and S. Boyd, "Enhancing sparsity by reweighted minimization, " J. Fourier Anal. Appl., vol. 14, no. 5-6, pp. 877-905, Dec. 2008.
16. H. Mohimani, M. Babaie-Zadeh, and C. Jutten, "A fast approach for overcomplete sparse decomposition based on smoothed norm, " IEEE Trans. Signal Process., vol. 57, no. 1, pp. 289-301, Jan. 2009.
17. G. Gasso, A. Rakotomamonjy, and S. Canu, "Recovering sparse signals with a certain family of nonconvex penalties and DC programming, " IEEE Trans. Signal Process., vol. 57, no. 12, pp. 4686-4698, Dec. 2009.
18. M. Hyder and K. Mahata, "An improved smoothed approximation algorithm for sparse representation, " IEEE Trans. Signal Process., vol. 58, no. 4, pp. 2194-2205, Apr. 2010.
19. J. Jin, Y. Gu, and S. Mei, "A stochastic gradient approach on compressive sensing signal reconstruction based on adaptive filtering framework, " IEEE J. Sel. Topics Signal Process., vol. 4, no. 2, pp. 409-420, Apr. 2010.
20. R. Gribonval and M. Nielsen, "Highly sparse representations from dictionaries are unique and independent of the sparseness measure, " Appl. Comput. Harmon. Anal., vol. 22, no. 3, pp. 335-355, May 2007. (Pubitemid 46528317)
21. R. Chartrand, "Exact reconstruction of sparse signals via nonconvex minimization, " IEEE Signal Process. Lett., vol. 14, no. 10, pp. 707-710, Oct. 2007. (Pubitemid 47434090)
22. S. Foucart and M. Lai, "Sparsest solutions of underdetermined linear systems via-minimization for, " Appl. Comput. Harmon. Anal., vol. 26, no. 3, pp. 395-407, May 2009.
23. R. Saab and Ö. Yilmaz, "Sparse recovery by non-convex optimization-instance optimality, " Appl. Comput. Harmon. Anal., vol. 29, no. 1, pp. 30-48, Jul. 2010.
24. I. Daubechies, R. DeVore, M. Fornasier, and C. Güntürk, "Iteratively reweighted least squares minimization for sparse recovery, " Commun. Pure Appl. Math., vol. 63, no. 1, pp. 1-38, Jan. 2010.
25. E. Chouzenoux, A. Jezierska, J. Pesquet, and H. Talbot, "A majorizeminimize subspace approach for image regularization, " SIAM J. Imag. Sci., vol. 6, no. 1, pp. 563-591, Mar. 2013.
26. H. Mohimani, M. Babaie-Zadeh, I. Gorodnitsky, and C. Jutten, "Sparse recovery using smoothed (SL0): Convergence analysis, " [Online]. Available: http://arxiv.org/abs/1001.5073, arXiv:1001.5073v1[cs.IT]
27. J. Vial, "Strong and weak convexity of sets and functions, " Math. Oper. Res., vol. 8, no. 2, pp. 231-259, May 1983.
28. B. Polyak, "Minimization of unsmooth functionals, " USSR Comput. Math. Math. Phys., vol. 9, no. 3, pp. 14-29, 1969.
29. D. Bertsekas, Nonlinear Programming, 2nd ed. Belmont, MA, USA: Athena Scientific, 1999.
30. S. Boyd, L. Xiao, and A. Mutapcic, "Subgradient methods, lecture notes for EE392o, " Stanford Univ.. Stanford, CA, USA, Autumn Quart. 2003-2004.
31. A. Goldstein, "Convex programming in Hilbert space, " Bull. Amer. Math. Soc., vol. 70, no. 5, pp. 709-710, May 1964.
32. A. Cohen, W. Dahmen, and R. Devore, "Compressed sensing and best-term approximation, " J. Amer. Math. Soc., vol. 22, no. 1, pp. 211-231, Jan. 2009.
33. R. Janin, "Sur la dualité et la sensibilité dans les problèmes de programme mathématique, " (in French) Ph.D. dissertation, Univ. of Paris, Paris, France, 1974.
34. A. Colesanti and D. Hug, "Hessian measures of semi-convex functions and applications to support measures of convex bodies, " Manuscripta Mathematica, vol. 101, no. 2, pp. 209-238, Feb. 2000.
35. F. Clarke, "Generalized gradients and applications, " Trans. Amer. Math. Soc., vol. 202, pp. 247-262, Apr. 1975.
36. D. Lorenz, M. Pfetsch, and A. Tillmann, "An infeasible-point subgradient method using adaptive approximate projections, " Comput. Optim. Appl., vol. 56, no. 1, pp. 1-36, Sept. 2013.
37. K. Kiwiel, "Convergence of approximate and incremental subgradient methods for convex optimization, " SIAM J. Optim., vol. 14, no. 3, pp. 807-840, 2004. (Pubitemid 39073314)
38. X.Wang, Y. Gu, and L. Chen, "Proof of convergence and performance analysis for sparse recovery via zero-point attracting projection, " IEEE Trans. Signal Process., vol. 60, no. 8, pp. 4081-4093, Aug. 2012.
39. V. Jeyakumar, "On subgradient duality with strong and weak convex functions, " J. Aust. Math. Soc. (Series A), vol. 40, no. 2, pp. 143-152, Apr. 1986.
40. V. Jeyakumar and B. M. Glover, "Characterizing global optimality for DC optimization problems under convex inequality constraints, " J. Global Optim., vol. 8, no. 2, pp. 171-187, Mar. 1996.
41. Z.Wu, "Sufficient global optimality conditions forweakly convexminimization problems, " J. Global Optim., vol. 39, no. 3, pp. 427-440, Nov. 2007. (Pubitemid 47551829)
42. M. Bayati and A. Montanari, "The dynamics of message passing on dense graphs, with applications to compressed sensing, " IEEE Trans. Inf. Theory, vol. 57, no. 2, pp. 764-785, Feb. 2011.
43. J. Trzasko and A. Manduca, "Highly undersampled magnetic resonance image reconstruction via homotopic-minimization, " IEEE Trans. Med. Imag., vol. 28, no. 1, pp. 106-121, Jan. 2009.
44. J. Tropp and A. Gilbert, "Signal recovery from random measurements via orthogonal matching pursuit, " IEEE Trans. Inf. Theory, vol. 53, no. 12, pp. 4655-4666, Dec. 2007.
45. CVX: Matlab Software for Disciplined Convex Programming, Version 2.0 Beta. ver. 2.0 Beta, CVX Research, Inc., Sep. 2012 [Online]. Available: http://cvxr.com/cvx
46. N. Shinozaki, M. Sibuya, and K. Tanabe, "Numerical algorithms for the Moore-Penrose inverse of a matrix: Direct methods, " Ann. Inst. Statist. Math., vol. 24, no. 1, pp. 193-203, Dec. 1972.
47. N. Shinozaki, M. Sibuya, and K. Tanabe, "Numerical algorithms for the Moore-Penrose inverse of a matrix: Iterative methods, " Ann. Inst. Statist. Math., vol. 24, no. 1, pp. 621-629, Dec. 1972.
48. G. Golub and W. Kahan, "Calculating the singular values and pseudoinverse of a matrix, " J. SIAM Numer. Anal. Ser. B, vol. 2, no. 2, pp. 205-224, 1965.
49. G. Golub and C. Reinsch, "Singular value decomposition and least squares solutions, " Numer. Math., vol. 14, no. 5, pp. 403-420, Apr. 1970.
50. A. Ben-Israel and D. Cohen, "On iterative computation of generalized inverses and associated projections, " SIAM J. Numer. Anal., vol. 3, no. 3, pp. 410-419, Sep. 1966.
51. A. Ben-Israel and T. Greville, Generalized Inverses, 2nd ed. New York, NY, USA: Springer, 2003.