Recovering sparse signals with a certain family of nonconvex penalties and DC programming

IEEE Transactions on Signal Processing

Volumn 57, Issue 12, 2009, Pages 4686-4698

  Gasso, Gilles ^a   Rakotomamonjy, Alain ^a   Canu, Stéphane ^a  

^a UNIVERSITÉ DE ROUEN   (France)

Author keywords

DC programming; Lasso; Nonconvex regularization; Signal representation; Sparsity; Variable selection

Indexed keywords

COMPUTER PROGRAMMING; FUNCTIONS; INVERSE SYNTHETIC APERTURE RADAR; ITERATIVE METHODS; LEAST SQUARES APPROXIMATIONS; RECOVERY;

D-C PROGRAMMING; LASSO; NONCONVEX; SIGNAL REPRESENTATIONS; SPARSITY; VARIABLE SELECTION;

SIGNAL RECONSTRUCTION;

EID: 70450245260     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2009.2026004     Document Type: Article

References (51)

1. L. T. H. An and P. D. Tao, DC Programming Approach and Solution Algorithm to Multidimensional Scaling Problem. Boston, MA: Kluwer Academic, 2001, Nonconvex Optimization and its Application, pp. 231-276.
2. A. Antoniadis and J. Fan, "Regularization of wavelet approximation," J. Amer. Statist. Assoc., vol.96, no.455, pp. 939-967, 2001.
3. S. Balakrishnan and D. Madigan, "Algorithms for sparse liner classifiers in the massive data setting," J. Mach. Learn. Res., vol.9, pp. 313-337, 2008.
4. L. Bottou and O. Bousquet, "The tradeoffs of large scale learning," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2008, vol.20, to be published.
5. P. Bühlmann and L. Meier, "Discussion on one-step sparse estimates in nonconcave penalized likelihood models," Ann. Statist., vol.36, no.4, pp. 1534-1541, 2008.
6. F. Bunea, "Honest variable selection in linear and logistic regression models via - and - - penalization," Electron. J. Statist., vol.2, pp. 1153-1194, 2008.
7. E. Candès and J. Romberg, "Sparsity and incoherence in compressive sampling," Inverse Problems, vol.23, no.3, pp. 969-985, 2006.
8. E. Candès and T. Tao, "The Dantzig selector: Statistical estimation when - is much larger than ," Ann. Statist., vol.35, no.6, pp. 2392-2404, 2007.
9. E. Candès, M.Wakin, and S. Boyd, "Enhancing sparsity by reweighted - minimization," J. Fourier Anal. Appl., 2008.
10. R. Chartrand, "Exact reconstruction of sparse signals via nonconvex minimization," IEEE Signal Process. Lett., vol.14, pp. 707-710, 2007.
11. R. Chartrand and V. Staneva, "Restricted isometry properties and nonconvex compressive sensing," Inverse Problems, vol.24, pp. 1-14, 2008.
12. R. Chartrand and W. Yin, "Iteratively reweighted algorithms for compressive sensing," in Proc. 33rd Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2008.
13. S. Chen, D. Donoho, and M. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Scientif. Comput., vol.20, no.1, pp. 33-61, 1998.
14. 1 -norm minimization," in Proc. Nat. Acad. Sci., USA, 2002, vol.1005, pp. 2197-2202.
15. D. Donoho and X. Huo, "Uncertainty principles and ideal atomic decompositions," IEEE Trans. Inf. Theory, vol.47, no.2845-2863, 2002.
16. D. Donoho and Y. Tsaig, "Extensions of compressed sensing," Signal Process., vol.86, no.3, pp. 533-548, 2006.
17. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, "Least angle regression," Ann. Statist., vol.32, no.2, pp. 407-499, 2004.
18. J. Fan and R. Li, "Variable selection via nonconcave penalized likelihood and its oracle properties," J. Amer. Statist. Assoc., vol.96, no.456, pp. 1348-1360, 2001.
19. M. Figueiredo, R. Nowak, and S. Wright, "Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems," IEEE J. Sel. Topics Signal Process.(Special Issue on Convex Optimization Methods for Signal Processing), vol.1, no.4, pp. 586-598, 2007.
20. 1 -," Appl. Computat. Harmon. Anal., vol.26, no.3, pp. 395-407, 2009.
21. I. Frank and J. Friedman, "A statistical view of som chemometrics regression tools (with discussion)," Technometrics, vol.35, no.109-148, 1993.
22. J. Friedman, T. Hastie, H. Höfling, and R. Tibshirani, "Pathwise coordinate optimization," Ann. Appl. Statist., vol.1, no.2, pp. 302-332, 2007.
23. W. J. Fu, "Penalized regression: The bridge versus the lasso," J. Computat. Graph. Statist., vol.7, pp. 397-416, 1998.
24. R. Gribonval and M. Nielsen, "Highly sparse representations from dictionaries are unique and independent of the sparseness measure," Appl. Computat. Harmon. Anal., vol.22, no.3, pp. 335-355, 2007.
25. R. Horst and N. V. Thoai, "Dc programming: Overview," J. Optimiz. Theory Appl., vol.103, pp. 1-41, 1999.
26. J. Huang, J. Horowitz, and S. Ma, "Asymptotic properties of bridge estimators in sparse high-dimensional regression models," Ann. Statist., vol.36, no.2, pp. 587-613, 2008.
27. S.-J. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, "A method for large-scale l1-regularized least squares," IEEE J. Sel. Topics Signal Process., vol.4, no.1, pp. 606-617, 2007.
28. K. Knight and W. Fu, "Asymptotics for lasso-type estimators," Ann. Statist., vol.28, pp. 1356-1378, 2000.
29. R. M. Leahy and B. D. Jeffs, "On the design of maximally sparse beamforming arrays," IEEE Trans. Antennas Propag., vol.39, pp. 1178-1187, 1991.
30. Y. Liu, X. Shen, and H. Doss, "Multicategory --learning and support vector machines: Computational tools," J. Computat. Graph. Statist., vol.14, no.1, pp. 219-236, 2005. (Pubitemid 40447834)
31. N. Meinshausen and P. Bühlmann, "High dimensional graphs and variable selection with the lasso," Ann. Statist., vol.34, no.3, pp. 1436-1462, 2006.
32. M. Nikolova, "Local strong homogeneity of a regularized estimator," SIAM J. Appl. Math., vol.61, no.2, pp. 633-658, 2000.
33. M. Osborne, B. Presnell, and B. Turlach, "A new approach to variable selection in least squares problems," IMA J. Numer. Anal., vol.20, no.3, pp. 389-403, 2000.
34. M. Osborne, B. Presnell, and B. Turlach, "On the lasso and its dual," J. Computat. Graph. Statist., vol.9, no.2, pp. 319-337, 2000.
35. R. Rockafellar, Convex Analysis. Princeton, NJ: Princeton Univ. Press, 1996.
36. R. Saab, R. Chartrand, and Y. Özgür, "Stable sparse approximations via nonconvex optimization," in Proc. 33rd Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2008.
37. S. Shevade and S. Keerthi, "A simple and efficient algorithm for gene selection using sparse logistic regression," Bioinformatics, vol.19, no.17, pp. 2246-2253, 2003.
38. P. D. Tao and L. T. H. An, "Dc optimization algorithms for solving the trust region subproblem," SIAM J. Optimiz., vol.8, no.2, pp. 476-505, 1998.
39. H. L. Thi and P. Tao, "A continuous approach for the concave cost supply problem via DC programming and DCA," Discrete Appl. Math., vol.156, pp. 325-338, 2008.
40. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Royal Statist. Soc., vol.46, pp. 431-439, 1996.
41. M. E. Tipping, "Sparse Bayesian learning and the relevance vector machine," J. Mach. Learn. Res., vol.1, pp. 211-244, 2001.
42. J. Tropp, "Just relax: Convex programming methods for identifying sparse signals," IEEE Trans. Inf. Theory, vol.51, no.3, pp. 1030-1051, 2006.
43. B. A. Turlach, W. N. Venables, and S. J. Wright, "Simultaneous variable selection," Technometrics, vol.27, pp. 349-363, 2005.
44. M. J. Wainwright, "Sharp thresholds for noisy and high-dimensional recovery of sparsity using - -constrained quadratic programming," Tech. Rep. UC Berkeley, Dep. Statist., 2006.
45. J. Weston, A. Elisseeff, B. Scholkopf, and M. Tipping, "The use of zero-norm with linear models and kernel methods," J. Mach. Learn. Res., vol.3, pp. 1439-1461, 2003.
46. A. L. Yuille and A. Rangarajan, "The concave-convex procedure," in Proc. Adv. Neural Inf. Process. Syst., 2001.
47. T. Zhang, "Some sharp performance bounds for least squares regression with - regularization," Tech. Rep. Rutgers Univ., Dep. Statist., New Brunswick, NJ, 2007, Ann. Statist., to be published.
48. Z. Zhang, J. T.Kwok, and D.-Y.Yeung, "Surrogate maximization/ minimization algorithms," Mach. Learn., pp. 1-33, 2007.
49. P. Zhao and B. Yu, "On model selection consistency of lasso," J. Mach. Learn. Res., vol.7, pp. 2541-2563, 2006.
50. H. Zou, "The adaptive lasso and its oracle properties," J. Amer. Statist. Assoc., vol.101, no.476, pp. 1418-1429, 2006.
51. H. Zou and R. Li, "One-step sparse estimates in nonconcave penalized likelihood models," Ann. Statist., vol.36, no.4, pp. 1509-1533, 2008.