L1/2 regularization: Convergence of iterative half thresholding algorithm

IEEE Transactions on Signal Processing

Volumn 62, Issue 9, 2014, Pages 2317-2329

  Zeng, Jinshan ^a,b   Lin, Shaobo ^a   Wang, Yao ^a   Xu, Zongben ^a  

^a XI'AN JIAOTONG UNIVERSITY   (China)

^b BEIJING CENTER FOR MATHEMATICS AND INFORMATION INTERDISCIPLINARY SCIENCES   (China)

Author keywords

Convergence; iterative half thresholding algorithm; nonconvex regularization

Indexed keywords

ALGORITHMS;

CONVERGENCE; CONVEX REGULARIZATIONS; ITERATIVELY REWEIGHTED LEAST SQUARES; LINEAR CONVERGENCE RATE; NONCONVEX; REGULARIZATION APPROACH; THRESHOLDING ALGORITHMS;

ITERATIVE METHODS;

EID: 84898680921     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2014.2309076     Document Type: Article

References (53)

1. D. L. Donoho, "Compressed sensing," IEEE Trans. Inf. Theory, vol.52, no. 4, pp. 1289-1306, 2006.
2. E. J. Candès, J. Romberg, and T. Tao, "Robust uncertainty principles:Exact signal reconstruction from highly incomplete frequency information,"IEEE Trans. Inf. Theory, vol. 52, no. 2, pp. 489-509, 2006. (Pubitemid 43174093)
3. E. J. Candès and T. Tao, "Decoding by linear programming," IEEETrans. Inf. Theory, vol. 51, no. 12, pp. 4203-4215, 2005. (Pubitemid 41800353)
4. M. Lustig, D. L. Donoho, J. M. Santos, and J. M. Pauly, "Compressedsensing MRI," IEEE Signal Process. Mag., vol. 25, no. 2, pp. 72-82,2008.
5. M. F. Duarte, M. A. Davenport, D. Takhar, J. N. Laska, T. Sun, K.F. Kelly, and R. G. Baraniuk, "Single pixel imaging via compressivesampling, " IEEE Signal Process. Mag., vol. 25, no. 2, pp. 83-91, 2008.
6. M. F. Duarte and Y. C. Eldar, "Structured compressed sensing: Fromtheory to applications," IEEE Trans. Signal Process., vol. 59, no. 9, pp.4053-4085, 2011.
7. B. C. Zhang, W. Hong, and Y. R. Wu, "Sparse microwave imaging:Principles and applications," Sci. China Inf. Sci., vol. 55, pp.1722-1754, 2012.
8. B. K. Natarajan, "Sparse approximate solutions to linear systems,"SIAM J. Comput., vol. 24, pp. 227-234, 1995.
9. R. Tibshirani, "Regression shrinkage and selection via the lasso," J.Roy. Statist. Soc, Series B (Methodolog.), vol. 58, no. 1, pp. 267-288,1996.
10. S. S. Chen,D. L.Donoho, andM.A. Saunders, "Atomic decompositionby basis pursuit," SIAM J. Sci. Comput., vol. 20, pp. 33-61, 1998.
11. D. L. Donoho, "High-dimensional centrally symmetric polytopes withneighborliness proportional to dimension," Discrete Comput. Geometry,vol. 35, no. 4, pp. 617-652, 2006. (Pubitemid 43701356)
12. A. Beck and M. Teboulle, "A fast iterative shrinkage-thresholding algorithmfor linear inverse problems," SIAM J. Image Sci., vol. 2, no. 1,pp. 183-202, 2009.
13. J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, "Robustface recognition via sparse representation," IEEE Trans. Pattern Anal.Mach. Intell., vol. 31, no. 2, pp. 210-227, 2008.
14. J. Ye and J. Liu, "Sparse methods for biomedical data," ACM SIGKDDExplorat. Newslett., vol. 14, no. 1, pp. 4-15, 2012.
15. E. J. Candès, M. B. Wakinm, and S. P. Boyd, "Enhancing sparsity byreweighted minimization," J. Fourier Anal. Appl., vol. 14, no. 5, pp.877-905, 2008.
16. T. Zhang, "Analysis of multi-stage convex relaxation for sparse regularization,"J. Mach. Learn. Res., vol. 11, pp. 1081-1107, 2010.
17. R. Chartrand, "Exact reconstruction of sparse signals via nonconvexminimization," IEEE Signal Process. Lett., vol. 14, no. 10, pp.707-710, 2007. (Pubitemid 47434090)
18. R. Chartrand and V. Staneva, "Restricted isometry properties and nonconvexcompressive sensing," Inverse Problems, vol. 24, pp. 1-14,2008.
19. Z. B. Xu, H. Zhang, Y. Wang, X. Y. Chang, and Y. Liang, "regularizater," Sci. in China, Series F, vol. 53, pp. 1159-1169, 2010.
20. Z. B. Xu, X. Y. Chang, F. M. Xu, and H. Zhang, " regularization:A thresholding representation theory and a fast solver," IEEE Trans.Neural Netw. Learn. Syst., vol. 23, no. 7, pp. 1013-1027, 2012.
21. Z. B. Xu, H. L. Guo, Y. Wang, and H. Zhang, "The representationof regularizer among regularizer: An experimentalstudy based on phase diagram," Acta Autom. Sin., vol. 38, pp.1225-1228, 2012.
22. D. Krishnan and R. Fergus, "Fast image deconvolution using hyper-Laplacian priors," Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2009.
23. W. F. Cao, J. Sun, and Z. B. Xu, "Fast image deconvolution usingclosed-form thresholding formulas of Lq regularization,"J. Vis. Commun. Image Represent., vol. 24, no. 1, pp. 1529-1542,2013.
24. Y. T. Qian, S. Jia, J. Zhou, and A. Robles-Kelly, " Hyperspectralunmixing via sparsity-constrained nonnegative matrix factorization,"IEEE Trans. Geosci. Remote Sens., vol. 49, no. 11, pp.4282-4297, 2011.
25. J. S. Zeng, J. Fang, and Z. B. Xu, "Sparse SAR imaging based onregularization," Sci. China Inf. Sci., vol. 55, pp. 1755-1775,2012.
26. J. S. Zeng, Z. B. Xu, B. C. Zhang, W. Hong, and Y. R.Wu, "Acceleratedregularization based SAR imaging via BCR and reducedNewton skills," Signal Process., vol. 93, pp. 1831-1844, 2013.
27. A. Kaban, "Fractional normregularization: Learning with very few relevantfeatures," IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 6,pp. 953-963, 2013.
28. C. Xu, Z. M. Peng, and W. F. Jing, "Sparse kernel logistic regressionbased on regularization," Sci. China Inf. Sci., vol. 56, no. 4, p.042308(16), 2013, 10.1007/s11432-012-4679-3.
29. Y. Liang, C. Liu, X. Z. Luan, K. S. Leung, T. M. Chan, Z. B. Xu,and H. Zhang, "Sparse logistic regression with a L1/2 penalty for geneselection in cancer classification," in Proc. BMC Bioinformatics 2013,vol. 14, p. 198, 10.1186/1471-2105-14-198.
30. W. F. Jing, D. Y. Meng, C. Qiao, and Z.M. Peng, "Eliminating verticalstrip defects on silicon steel surface by regularization," Math.Problems Eng., p. 13, 2011, 10.1155/2011/854674, Article ID 854674.
31. K. Bredies and D. A. Lorenz, "Linear convergence of iterative softthresholding,"J. Fourier Anal. Appl., vol. 14, pp. 813-837, 2008.
32. T. Blumensath and M. E. Davies, "Iterative thresholding for sparse approximation,"J. Fourier Anal. Appl., vol. 14, no. 5, pp. 629-654, 2008.
33. I. Daubechies, M. Defries, and C. De Mol, "An iterative thresholdingalgorithm for linear inverse problems with a sparsity constraint,"Comm. Pure Appl. Math., vol. 57, pp. 1413-1457, 2004. (Pubitemid 39427442)
34. S. Foucart, "Sparse recovery algorithms: Sufficient conditions in termsof restricted isometry constants," in Proc. Approximation Theory XIII,Springer Proceedings in Mathematics, San Antonio, TX, USA, 2010,vol. 13, pp. 65-77.
35. A. S. Bandeira, E. Dobriban, D. G. Mixon, and W. F. Sawin, "Certifyingthe restricted isometry property is hard," IEEE Trans. Inf. Theory,vol. 59, no. 6, pp. 3448-3450, 2013.
36. P. Koiran and A. Zouzias, "On the certification of the restricted isometryproperty," arXiv e-prints
37. [Online]. Available: http://arxiv.org/abs/1103.4984
38. P. Koiran and A. Zouzias, "Hidden cliques and the certification of therestricted isometry property," arXiv e-print
39. [Online]. Available: http://arxiv.org/abs/1211.0665
40. M. Rudelson and R. Vershynin, "On sparse reconstruction from Fourierand Gaussian measurements," Comm. Pure Appl. Math., vol. 61, pp.1025-1045, 2008.
41. R. Baraniuk, M. Davenport, R. DeVore, and M. B. Wakin, "A simpleproof of the restricted isometry property for random matrices," Constr.Approx., vol. 28, pp. 253-263, 2008.
42. E. J. Candès and T. Tao, "The Dantzig selector: Stastical estimationwhen is much larger than ," Ann. Statist., vol. 35, no. 6, pp.2313-2351, 2007.
43. T. T. Cai and A. Zhang, "Compressed sensing and affine rank minimizationunder restricted isometry," IEEE Trans. Signal Process., vol.61, no. 13, pp. 3279-3290, 2013.
44. D. L. Donoho andM. Elad, "Optimally sparse representation in general(nonorthogonal) dictionaries via minimization," Proc. Nat. Acad.Sci., vol. 100, no. 5, pp. 2197-2202, 2003. (Pubitemid 36297476)
45. F. M. Xu and S. H.Wang, "A hybrid simulated annealing thresholdingalgorithm for compressed sensing," Signal Process., vol. 93, pp.1577-1585, 2013.
46. S. Foucart and M. J. Lai, "Sparsest solutions of undertermined linearsystems via-minimization for ," Appl. Comput. Harmon.Anal., vol. 26, pp. 395-407, 2009.
47. K. Bredies and D. A. Lorenz, "Minimization of non-smooth,non-convex functionals by iterative thresholding," 2009
48. [Online].Available: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10. 1.1.156.9058
49. P. H. Gong, C. S. Zhang, Z. S. Lu, J. H. Huang, and J. P. Ye, "A generaliteartive shrinkage and thresholding algorithm for non-convex regularizedoptimization problems," presented at the 30th Int. Conf. Mach.Learn. (ICML), Atlanta, GA, USA, 2013.
50. I. F. Gorodnitsky and B. D. Rao, "Sparse signal reconstruction fromlimited data using FOCUSS: A re-weighted minimum norm algorithm,"IEEE Trans. Signal Process., vol. 45, no. 3, pp. 600-616, 1997. (Pubitemid 127765947)
51. I. Daubechies, R. Devore,M. Fornasier, and C. S. Gunturk, "Iterativelyreweighted least squares minimization for sparse recovery," Comm.Pure Appl. Math., vol. 63, pp. 1-38, 2010.
52. X. Chen and W. Zhou, "Convergence of the reweighted minimizationalgorithm for minimization," Comput. Optim. Appl., 2013,to be published.
53. T. Blumensath andM. E. Davies, "Iterative hard thresholding for compressedsensing," Appl. Comput. l Harmon. Anal., vol. 27, pp. 265-274,2008.