Iterative reweighted ℓ 1and ℓ 2 methods for finding sparse solutions

IEEE Journal on Selected Topics in Signal Processing

Volumn 4, Issue 2, 2010, Pages 317-329

  Wipf, David ^a   Nagarajan, Srikantan ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Compressive sensing (CS); Iterative reweighting algorithms; Sparse representations; Underdetermined inverse problems

Indexed keywords

COMPUTATIONAL TASK; CONVERGENCE RESULTS; EMPIRICAL TEST; EXISTING METHOD; ITERATIVE REWEIGHTING ALGORITHMS; NATURAL EXTENSION; NON-NEGATIVE SPARSE CODING; OVERCOMPLETE DICTIONARIES; PENALTY FUNCTION; PERFORMANCE IMPROVEMENTS; PRACTICAL METHOD; SENSING APPLICATIONS; SPARSE ESTIMATION; SPARSE RECOVERY; SPARSE REPRESENTATION; SPARSE REPRESENTATIONS; SPARSE SOLUTIONS; SPARSITY PROBLEMS;

ALGORITHMS; DIFFERENTIAL EQUATIONS; INDEPENDENT COMPONENT ANALYSIS; INVERSE PROBLEMS; NONLINEAR FILTERING; SIGNAL RECONSTRUCTION;

ITERATIVE METHODS;

EID: 77949736709     PISSN: 19324553     EISSN: None     Source Type: Journal    
DOI: 10.1109/JSTSP.2010.2042413     Document Type: Article

References (31)

1. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
2. A. M. Bruckstein, M. Elad, and M. Zibulevsky, "A non-negative and sparse enough solution of an underdetermined linear system of equations is unique," IEEE Trans. Inf. Theory, vol.54, no.11, pp. 4813-4820, Nov. 2008.
3. 1 minimization," J. Fourier Anal. Appl., vol.14, no.5, pp. 877-905, Dec. 2008.
4. R. Chartrand and W. Yin, "Iteratively reweighted algorithms for compressive sensing," in Proc. Int. Conf. Accoust., Speech, Signal Process., 2008, pp. 3869-3872.
5. p Sparse recovery can fail for 0 < p ≤ 1," Tech. Rep. IRISA, no.1899, Jul. 2008.
6. I. Daubechies, R. DeVore, M. Fornasier, and S. Gunturk, "Iteratively re-weighted least squares minimization for sparse recovery," Commun. Pure Appl. Math., vol.63, no.1, pp. 1-38, 2010.
7. 1 -norm solution is also the sparsest solution," Stanford Univ. Tech. Rep., Stanford, CA, Sep. 2004.
8. 1 minimization," Proc. National Acad. Sci., vol.100, no.5, pp. 2197-2202, Mar. 2003.
9. M. Fazel, H. Hindi, and S. Boyd, "Log-Det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices," in Proc. Amer. Control Conf., Jun. 2003, vol.3, pp. 2156-2162.
10. M. Figueiredo, "Adaptive sparseness for supervised learning," IEEE Trans. Pattern Anal. Mach. Intell., vol.25, no.9, pp. 1150-1159, Sep. 2003.
11. M. Figueiredo, J. Bioucas-Dias, and R. Nowak, "Majorization- minimization algorithms for wavelet-based image restoration," IEEE Trans. Image Process., vol.16, no.12, pp. 2980-2991, Dec. 2007.
12. J. Fuchs, "On sparse representations in arbitrary redundant bases," IEEE Trans. Inf. Theory, vol.50, no.6, pp. 1341-1344, Jun. 2004.
13. R. Gribonval and M. Nielsen, "Sparse representations in unions of bases," IEEE Trans. Inf. Theory, vol.49, no.12, pp. 3320-3325, Dec. 2003.
14. G. Luenberger, Linear and Nonlinear Programming, 2nd ed. Reading, MA: Addison-Wesley, 1984.
15. D. Malioutov, M. Çetin, and A. Willsky, "Optimal sparse representations in general overcomplete bases," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., May 2004, vol.2, pp. II-793-II-796.
16. R. Neal, Bayesian Learning for Neural Networks. New York: Springer-Verlag, 1996.
17. J. Palmer, D. Wipf, K. Kreutz-Delgado, and B. Rao, "Variational EM algorithms for non-gaussian latent variable models," Adv. Neural Inf. Process. Syst., vol.18, pp. 1059-1066, 2006.
18. B. Rao, K. Engan, S. F. Cotter, J. Palmer, and K. Kreutz-Delgado, "Subset selection in noise based on diversity measure minimization," IEEE Trans. Signal Process., vol.51, no.3, pp. 760-770, Mar. 2003.
19. B. D. Rao and K. Kreutz-Delgado, "Basis selection in the presence of noise," in Proc. 32nd Asilomar Conf. Signals, Syst. Comput., Nov. 1998, vol.1, pp. 752-756.
20. B. Rao and K. Kreutz-Delgado, "An affine scaling methodology for best basis selection," IEEE Trans. Signal Process., vol.47, no.1, pp. 187-200, Jan. 1999.
21. R. Showalter, "Monotone operators in Banach space and nonlinear partial differential equations," Mathematical Surveys and Monographs vol. 49, 1997, AMS, Providence, RI.
22. J. Silva, J. Marques, and J. Lemos, "Selecting landmark points for sparse manifold learning," Adv. Neural Inf. Process. Syst., vol.18, pp. 1241-1248, 2006.
23. M. Tipping, "Sparse Bayesian learning and the relevance vector machine," J. Mach. Learn. Res., vol.1, pp. 211-244, 2001.
24. J. Tropp, "Greed is good: Algorithmic results for sparse approximation," IEEE Trans. Inf. Theory, vol.50, no.10, pp. 2231-2242, Oct. 2004.
25. J. Tropp, "Algorithms for simultaneous sparse approximation. Part II: Convex relaxation," Signal Process., vol.86, pp. 589-602, Apr. 2006.
26. M. Wakin, M. Duarte, S. Sarvotham, D. Baron, and R. Baraniuk, "Recovery of jointly sparse signals from a few random projections," Adv. Neural Inf. Process. Syst., vol.18, pp. 1433-1440, 2006.
27. D. Wipf and S. Nagarajan, "A new view of automatic relevance determination," Adv. Neural Inf. Process. Syst., vol.20, 2008.
28. D. Wipf and S. Nagarajan, "Latent variable Bayesian models for promoting sparsity," , submitted for publication, 2009.
29. D. Wipf, J. Owen, H. Attias, K. Sekihara, and S. Nagarajan, "Robust Bayesian estimation of the location, orientation, and timecourse of mutliple correlated neural sources using MEG," NeuroImage, 2009, to be published.
30. M. Yuan and Y. Lin, "Model selection and estimation in regression with grouped variables," J. R. Statist. Soc. B, vol.68, pp. 49-67, 2006.
31. W. Zangwill, Nonlinear Programming: A Unified Approach. Englewood Cliffs, NJ: Prentice-Hall, 1969.