Fast solution of ℓ1-Norm minimization problems when the solution may be sparse

IEEE Transactions on Information Theory

Volumn 54, Issue 11, 2008, Pages 4789-4812

  Donoho, David L ^a   Tsaig, Yaakov ^b  

^a STANFORD UNIVERSITY   (United States)

^b Stanford University   (United States)

Author keywords

1 minimization; Basis pursuit; Homotopy methods; LASSO; Least Angle Regression (Lars); Orthogonal matching pursuit; Polytope faces pursuit; Sparse representations; Underdetermined systems of linear equations

Indexed keywords

ARTIFICIAL INTELLIGENCE; COMMUNICATION SYSTEMS; INFORMATION THEORY; OPTIMIZATION; SIGNAL PROCESSING;

BASIS PURSUIT; HOMOTOPY METHODS; LASSO; LEAST ANGLE REGRESSION (LARS); ORTHOGONAL MATCHING PURSUIT; POLYTOPE FACES PURSUIT; SPARSE REPRESENTATIONS; UNDERDETERMINED SYSTEMS OF LINEAR EQUATIONS;

LINEAR EQUATIONS;

EID: 55349134896     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2008.929958     Document Type: Article

References (55)

1. M. Berkelaar, K. Eikland, and P. Notebaert, Lp_solve: (Mixed-Integer) Linear Programming System [Online]. Available: http:// lpsolve.sourceforge.net
2. D. P. Bertsekas, Convex Analysis and Optimization. Nashua, NH: Athena Scientific, 2003.
3. W. L. Briggs and Van Emden Henson, The DFT: An Owner's Manual for the Discrete Fourier Transform. Philadelphia, PA: SIAM, 1995.
4. J. Buckheit and D. L. Donoho, "Wavelab and reproducible research," in Wavelets and Statistics, A. Antoniadis and G. Oppenheim, Eds. New York: Springer-Verlag, 1995, pp. 55-81.
5. E. Candès and T. Tao, "Decoding by linear programming," IEEE Trans. Inf. Theory, vol. 51, no. 12, pp. 4203-4215, Dec. 2005.
6. E. J. Candès and J. Romberg, "Practical signal recovery from random projections," 2005, unpublished manuscript.
7. E. J. Candès, J. Romberg, and T. Tao, "Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information," IEEE Trans. Inf. Theory, vol. 52, no. 2, pp. 489-509, Feb. 2006.
8. E. J. Candès and T. Tao, "Stable signal recovery from incomplete and inaccurate observations," Commun. Pure Appl. Math., vol. 59, no. 8, pp. 1207-1223, Aug. 2006.
9. S. S. Chen, D. L. Donoho, and M. A. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Sci Comp., vol. 20, no. 1, pp. 33-61, 1999.
10. S. C. Choi, D. L. Donoho, A. G. Flesia, X. Huo, O. Levi, and D. Slu, Beamlab Web Site Online. Available: http://www-stat.stanford.edu/ ~beamlab/
11. Daubechies, M. Defrise, and C. De Mol, "An iterative thresholding algorithm for linear inverse problems with a sparsity constraint," Commun. Pure Appl. Math., vol. 57, no. 11, pp. 1413-1541, Aug. 2004.
12. G. Davis, S. Mallat, and M. Avellaneda, "Adaptive greedy approximations," J. Constr. Approx., vol. 13, pp. 57-98, 1997.
13. D. L. Donoho, Neighborly Polytopes and Sparse Solution of Underdetermined Linear Equations Dept. Statist., Stanford Uni., Stanford, CA, 2005, Tech. Rep..
14. D. L. Donoho, "Compressed sensing," IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289-1306, Apr. 2006.
15. 1 solution is also the sparsest solution," Commun. Pure Appl. Math., vol. 59, no. 7, pp. 907-934, Jul. 2006.
16. 1-norm near-solution approximates the sparsest near-solution," Commun. Pure Appl. Math., vol. 59, no. 6, pp. 797-829, Jun. 2006.
17. D. L. Donoho, "High-dimensional centrosymmetric polytopes with neighborliness proportional to dimension," Discr. Comput. Geom., vol. 35, no. 4, pp. 617-652, May 2006.
18. 1 norm minimization," Proc. Nat. Acad. Sci. USA, vol. 100, no. 5, pp. 2197-2002, Mar. 2003.
19. D. L. Donoho, M. Elad, and V. N. Temlyakov, "Stable recovery of sparse overcomplete representations in the presence of noise," IEEE Trans. Inf. Theory, vol. 52, no. 1, pp. 6-18, Jan. 2006.
20. D. L. Donoho and X. Huo, "Uncertainty principles and ideal atontic decomposition," IEEE Trans. Inf. Theory, vol. 47, no. 7, pp. 2845-2862, Nov. 2001.
21. D. L. Donoho and B. F. Logan, "Signal recovery and the large sieve," SIAM J. Appl. Math., vol. 52, no. 2, pp. 577-591, 1992.
22. D. L. Donoho and J. Tanner, "Counting faces of randomly-projected polytopes when the projection radically lowers dimension," J. Amer. Math. Soc., posted online July 10, 2008, to be published.
23. D. L. Donoho, Y. Tsaig, 1. Drori, and J.-L. Starck, "Sparse Solution of Underdetermined Linear Equations by Stagewise Orthogonal Matching Pursuit," 2006, submitted for publication.
24. 1 minimization problems by LARS/homotopy methods," in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing, Toulose, France, May 2006.
25. B. Efron and T. Hastie, LARS Softare Web Site Online. Available: http:// www-stat.stanford.edu/~hastie/Papers/LARS/
26. B. Efron, T. Hastie, I. M. Johnstone, and R. Tibshwani, "Least angle regression," Ann. Statist., vol. 32, no. 2, pp. 407-499, 2004.
27. M. Elad, B. Matalon, and M. Zibulevsky, "Coordinate and subspace optimization methods for linear least squares with nonquadratic regularization," Appl. Comp. Harm. Anal., to be published.
28. M. Elad and A. M. Bruckstein, "A generalized uncertainty principle and sparse representations in pairs of bases," IEEE Trans. Inf. Theory vol. 48, no. 9, pp. 2558-2567, Sep. 2002.
29. J. J. Fuchs, "On sparse representations in arbitrary redundant bases," IEEE Trans. Inf. Theory, vol. 50, no. 6, pp. 1341-1344, Jun. 2004.
30. A. Y. Garnaev and E. D. Gluskin, "On widths of the Euclidean ball," in Sov. Math. - Dok.. :, 1984, vol. 30, pp. 200-203.
31. A. C. Gilbert, S. Guha, P. Indyk, S. Muthukrishnan, and M. Strauss, "Near-optimal sparse fourier representations via sampling," in Proc. 34th ACM Symp. Theory of Computing, 2002, pp. 152-161.
32. G. H. Golub and C. F. Van Loan, Matrix Computations, 3rd ed. Baltimore, MD: Johns Hopkins Univ. Press, 1996.
33. R. Gribonval and M. Nielsen, "Sparse representations in unions of bases," IEEE Trans. Inf. Theory, vol. 49, no. 12, pp. 1320-1325, Dec. 2003.
34. R. Gribonval and M. Nielsen, "Highly sparse representations from dictionaries are unique and independent of the sparseness measure," 2006, to be published.
35. M. Harwit and N. J. A. Sloane, Hadamard Transform Optics. San Diego, CA: Academic, 1979.
36. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning. New York: Springer-Verlag, 2001.
37. B. S. Kashin, "Diameters of certain finite-dimensional sets in classes of smooth functions," Izv. Akad. Nauk SSSR, Ser. Mat., vol. 41, no. 2, pp. 334-351, 1977.
38. 1 reconstruction," in Proc. 13th Int. Soc. Magnetic Resources in Medicine, Kyoto, Japan, May 2004.
39. M. Lustig, J. M. Santos, D. L. Donoho, and J. M. Pauly, "Rapid MR angiography with randomlyunder-sampled 3DFT trajectories and nonlinear reconstruction," in Proc. 9th Annu. Meeting, Socielyfor Cardiovascular Magnetic Resonance, Miami, FL, 2006.
40. D. M. Malioutov, M. __ceCetin, and A. S. Willsky, "Homotopy continuation for sparse signal representation," in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing, Philadelphia, PA, Mar. 2005, vol. 5, pp. 733-736.
41. M. R. Osborne, B. Presnell, and B. A. Turlach, "A new approach to variable selection in least squares problems," IMA J. Numer. Anal. vol. 20, pp. 389-403, 2000.
42. M. R. Osborne, B. Presnell, and B. A. Turlach, "On the LASSO and its dual," J. Comput. Graph. Stat., vol. 9, pp. 319-337, 2000.
43. 1 sparse representations," in Proc. Conf. Signal Processing with Adaptive Sparse Structured Representations (SPARS'05), Rennes, France, Nov. 2005, pp. 206-213.
44. M. D. Plumbley, "Recovery of sparse representations by polytope faces pursuit," in Proc. 6th Int. Conf. Independent Component Analysis and Blind Source Separation (ICA 2006), Charleston, SC, Mar. 2006, pp. 206-213.
45. M. Rudelson and R. Vershynin, Geometric Approach to Error-Correcting Codes and Reconstruction of Signals, Dept. Math., Univ. California, Davis, 2005, Tech. Rep.
46. M. A. Saunders and B. Kim, PDCO: Primal-Dual Interior Method for Convex Objectives [Online]. Available: http://www.stanford.edu/group/SOL/ software/Pdco.html
47. J.-L. Starck, M. Elad, and D. L. Donoho, "Image decomposition via the combination of sparse representation and a variational approach," IEEE Trans. Image Process., vol. 14, no. 10, pp. 1570-1582, Oct. 2005.
48. J.-L. Starck, M. Elad, and D. L. Donoho, "Redundant multiscale transforms and their application for morphological component analysis," Adv. Imag. Electron Phys., vol. 132, pp. 287-348, 2004.
49. T. Strohmer and R. Heath, Jr, "Grassmannian frames with applications to coding and communcations," Appl. Comp. Harm. Anal., vol. 14, no. 3, pp.257-275, 2003.
50. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Roy. Statist. Soc., vol. 58, no. 1, pp. 267-288, 1996.
51. J. A. Tropp, "Greed is good: Algorithmic results for sparse approximation," IEEE Trans. Inf. Theory, vol. 50, no. 10, pp. 2231-2242, Oct. 2004.
52. J. A. Tropp, "Just relax: Convex programming methods for subset selection and sparse approximation," IEEE Trans. Inf. Theory, vol. 51, no. 3, pp. 1030-1051, Mar. 2006.
53. J. A. Tropp and A. C. Gilbert, Signal recovery from partial information by orthogonal matching pursuit, 2005, unpublished manuscript.
54. 1-norm solution and the sparsest solution," Signal Process., vol. 86, no. 3, pp. 533-548, Mar. 2006.
55. Y. Tsaig and D. L. Donoho, "Extensions of compressed sensing," Signal Process., vol. 86, no. 3, pp. 549-571, Mar. 2006.