Just relax: Convex programming methods for identifying sparse signals in noise

IEEE Transactions on Information Theory

Volumn 52, Issue 3, 2006, Pages 1030-1051

  Tropp, Joel A ^a,b,c  

^a IEEE   (United States)

^b UNIVERSITY OF TEXAS AT AUSTIN   (United States)

^c UNIVERSITY OF MICHIGAN   (United States)

Author keywords

Algorithms; Approximation methods; Basis pursuit; Convex program; Linear regression; Optimization methods; Orthogonal matching pursuit; Sparse representations

Indexed keywords

BASIS PURSUIT; CONVEX PROGRAMMING METHODS; ORTHOGONAL MATCHING PURSUIT; SPARSE REPRESENTATIONS;

CHANNEL CODING; COMPUTATIONAL COMPLEXITY; LINEAR REGRESSION; NUMERICAL ANALYSIS; OPTIMIZATION; PROBLEM SOLVING;

SPURIOUS SIGNAL NOISE;

EID: 33645712308     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2005.864420     Document Type: Article

References (67)

1. R. Gribonval and E. Bacry, "Harmonic decomposition of audio signals with matching pursuit," IEEE Trans. Signal Process., vol. 51, no. 1, pp. 101-111, Jan. 2003.
2. P. Frossard, P. Vandergheynst, R. M. F. i. Ventura, and M. Kunt, "A posteriori quantization of progressive matching pursuit streams," IEEE Trans. Signal Process., vol. 52, no. 2, pp. 525-535, Feb. 2004.
3. T. Nguyen and A. Zakhor, "matching pursuits based multiple description video coding for lossy environments," presented at the 2003 IEEE Int. Conf. Image Processing, Barcelona, Catalonia, Spain, Sep. 2003.
4. A. J. Miller, Subset Selection in Regression, 2nd ed. London, U.K.: Chapman and Hall, 2002.
5. 1 norm," Geophysics, vol. 44, no. 1, pp. 39-52, 1979.
6. J. Rissanen, "Modeling by shortest data description," Automation, vol. 14, pp. 465-471, 1979.
7. M. Grote and T. Huckle, "Parallel preconditioning with sparse approximate inverses," SIAM J. Sci. Comput., vol. 18, no. 3, pp. 838-853, 1997.
8. F. Girosi, "An equivalence between sparse approximation and support vector machines," Neural Comput., vol. 10, no. 6, pp. 1455-1480, 1998.
9. S. S. Chen, D. L. Donoho, and M. A. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Sci. Comput., vol. 20, no. 1, pp. 33-61, 1999.
10. I. Daubechies, M. Defrise, and C. D. Mol, "An iterative thresholding algorithm for linear inverse problems with a sparsity constraint," Commun. Pure Appl. Math., vol. 57, pp. 1413-1457, 2004.
11. B. D. Rao and K. Kreutz-Delgado, "An affine scaling methodology for best basis selection," IEEE Trans. Signal Process., vol. 47, no. 1, pp. 187-200, Jan. 1999.
12. G. Davis, S. Mallat, and M. Avellaneda, "Greedy adaptive approximation," J. Constr. Approx., vol. 13, pp. 57-98, 1997.
13. V. Temlyakov, "Nonlinear methods of approximation," Foundations Comput. Math., vol. 3, no. 1, pp. 33-107, Jul. 2002.
14. J. A. Tropp, "Greed is good: Algorithmic results for sparse approximation," IEEE Trans. Inf. Theory, vol. 50, no. 10, pp. 2231-2242, Oct. 2004.
15. J. F. Claerbout and F. Muir, "Robust modeling of erratic data," Geophysics, vol. 38, no. 5, pp. 826-844, Oct. 1973.
16. B. K. Natarajan, "Sparse approximate solutions to linear systems," SIAM J. Comput., vol. 24, pp. 227-234, 1995.
17. S. Boyd and L. Vanderberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press. 2004.
18. D. L. Donoho, M. Elad, and V. N. Temlyakov, "Stable recovery of sparse overcomplete representations in the presence of noise," IEEE Trans. Inf. Theory, vol. 52, no. 1, pp. 6-18, Jan. 2006.
19. D. L. Donoho and X. Huo, "Uncertainty principles and ideal atomic decomposition," IEEE Trans. Inf. Theory, vol. 47, no. 7, pp. 2845-2862, Nov. 2001.
20. M. Elad and A. M. Bruckstein, "A generalized uncertainty principle and sparse representation in pairs of bases," IEEE Trans. Inf. Theory, vol. 48, no. 9, pp. 2558-2567, Sep. 2002.
21. 1 minimization," Proc. Nat. Acad. Sci., vol. 100, pp. 2197-2202, Mar. 2003.
22. R. Gribonval and M. Nielsen, "Sparse representations in unions of bases," IEEE Trans. Inf. Theory, vol. 49, no. 12, pp. 3320-3325, Dec. 2003.
23. J.-J. Fuchs, "On sparse representations in arbitrary redundant bases," IEEE Trans. Inf. Theory, vol. 50, no. 6, pp. 1341-1344, Jun. 2004.
24. C. Dossal and S. Mallat, "Sparse spike deconvolution with minimum scale," in Proc. Signal Processing with Adaptive Sparse Structured Representations 2005. Rennes, France, Nov. 2005, pp. 123-126.
25. J. A. Tropp, "Just Relax: Convex Programming Methods for Subset Selection and Sparse Approximation," The Univ. Texas at Austin, ICES Rep. 04-04, 2004.
26. J.-J. Fuchs, "Recovery of exact sparse representations in the presence of noise," in Proc. 2004 IEEE Int. Conf. Acoustics, Speech, and Signal Processing, Montreal, QC, Canada, May 2004.
27. _, "Recovery of exact sparse representations in the presence of bounded noise," IEEE Trans. Inf. Theory, vol. 51, no. 10, pp. 3601-3608, Oct. 2005.
28. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
29. A. C. Gilbert and J. A. Tropp, "Applications of sparse approximation in communications," in Proc. 2005 IEEE Int. Symp. Information Theory, Adelaide, Australia, Sep. 2005, pp. 1000-1004.
30. R. A. Horn and C. R. Johnson, Matrix Analysis. Cambridge, U.K.: Cambridge Univ. Press, 1985.
31. E. Kreyszig, Introductory Functional Analysis with Applications. New York: Wiley, 1989.
32. G. H. Golub and C. F. Van Loan, Matrix Computations, 3rd ed. Baltimore, MD: Johns Hopkins Univ. Press, 1996.
33. J. A. Tropp, "Topics in Sparse Approximation," Ph.D. dissertation, Computational and Applied Mathematics, The Univ. Texas at Austin, Aug. 2004.
34. R. T. Rockafellar, Convex Analysis. Princeton, NJ: Princeton Univ. Press, 1970.
35. F. Santosa and W. W. Symes, "Linear inversion of band-limited reflection seismograms," SIAM J. Sci. Statist. Comput., vol. 7, no. 4, pp. 1307-1330, 1986.
36. J.-J. Fuchs, "Extension of the Pisarenko method to sparse linear arrays," IEEE Trans. Signal Process., vol. 45, no. 10, pp. 2413-2421, Oct. 1997.
37. _, "On the application of the global matched filter to DOA estimation with uniform circular arrays," IEEE Trans. Signal Process., vol. 49, no. 4, pp. 702-709, Apr. 2001.
38. B. A. Olshausen and D. J. Field, "Emergence of simple-cell receptive field properties by learning a sparse code for natural images," Nature, vol. 381, pp. 607-609, 1996.
39. D. L. Donoho and I. M. Johnstone, "Minimax Estimation via Wavelet Shrinkage," Stanford Univ., Statistics Dept. Tech. Rep., 1992.
40. S. Mallat, A Wavelet Tour of Signal Processing, 2nd ed. London, U.K.: Academic, 1999.
41. 1 minimization," IEEE Trans. Inf. Theory, vol. 51, no. 4, pp. 1568-1570, Apr. 2005.
42. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Roy. Statist. Soc., vol. 58, 1996.
43. R. Gribonval, R. F. i. Venturas, and P. Vandergheynst, "A Simple Test to Check the Optimality of a Sparse Signal Approximation," Université de Rennes I, Tech. Rep. 1661, Nov. 2004.
44. Y. C. Pati, R. Rezaiifar, and P. S. Krishnaprasad, "Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition," in Proc. 27th Annu. Asilomar Conf. Signals, Systems and Computers, Pacific Grove, CA, Nov. 1993, pp. 40-44.
45. J. A. Tropp, A. C. Gilbert, and M. J. Strauss, "Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit," EURASIP J. Signal Process., to be published.
46. E. Candès, J. Romberg, and T. Tao, "Exact Signal Reconstruction From Highly Incomplete Frequency Information," manuscript submitted for publication, June 2004.
47. E. J. Candès and J. Romberg, "Quantitative Robust Uncertainty Principles and Optimally Sparse Decompositions," manuscript submitted for publication, Nov. 2004.
48. E. J. Candès and T. Tao, "Near Optimal Signal Recovery from Random Projections: Universal Encoding Strategies?," manuscript submitted for publication, Nov. 2004.
49. 1-Norm Solution is Also the Sparsest Solution," unpublished manuscript, Sep. 2004.
50. 1-Norm Solution Approximates the Sparsest Near-Solution," unpublished manuscript, Sep. 2004.
51. D. L. Donoho and M. Elad, On the Stability of Basis Pursuit in the Presence of Noise, unpublished manuscript, Nov. 2004.
52. J.-J. Fuchs, "Some further results on the recovery algorithms," in Proc. Signal Processing with Adaptive Sparse Structured Representations 2005. Rennes, France, Nov. 2005, pp. 87-90.
53. 1 data-fidelity term," in Proc. Signal Processing with Adaptive Sparse Structured Representations 2005. Rennes, France, Nov. 2005, pp. 135-138.
54. H. H. Bauschke and J. M. Borewein, "Legendre functions and the method of random Bregman projections," J. Convex Anal., vol. 4, no. 1, pp. 27-67, 1997.
55. A. Banerjee, S. Merugu, I. S. Dhillon, and J. Ghosh, "Clustering with Bregman divergences," in Proc. 4th SIAM Int. Conf. Data Mining. Philadelphia, PA: SIAM, Apr. 2004.
56. H. Zuo and T. Hastie, "Regularization and Variable Selection via the Elastic Net," unpublished manuscript, Sep. 2004.
57. D. Leviatan and V. N. Temlyakov, "Simultaneous Approximation by Greedy Algorithms," Univ. South Carolina at Columbia, IMI Rep. 2003:02, 2003.
58. S. Cotter, B. D. Rao, K. Engan, and K. Kreutz-Delgado, "Sparse solutions of linear inverse problems with multiple measurement vectors," IEEE Trans. Signal Process., vol. 53, no. 7, pp. 2477-2488, Jul. 2005.
59. 1 Minimization and Greedy Algorithms," manuscript submitted for publication, Oct. 2004.
60. J. A. Tropp, "Algorithms for simultaneous sparse approximation. Part II: Convex relaxation," EURASIP J. Signal Process., to be published.
61. S. Levy and P. K. Fullagar, "Reconstruction of a sparse spike train from a portion of its spectrum and application to high-resolution deconvolution," Geophysics, vol. 46, no. 9, pp. 1235-1243, 1981.
62. D. W. Oldenburg, T. Scheuer, and S. Levy, "Recovery of the acoustic impedance from reflection seismograms," Geophysics, vol. 48, pp. 1318-1337, 1983.
63. C. H. Papadimitriou and K. Steiglitz, Combinatorial Optimization: Algorithms and Complexity. New York: Dover, 1998. Corrected republication.
64. M. X. Goemans and D. P. Williamson, "Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming," J. Assoc. Comput. Mach., vol. 42, pp. 1115-1145, 1995.
65. A. Feuer and A. Nemirovsky, "On sparse representation in pairs of bases," IEEE Trans. Inf. Theory, vol. 49, no. 6, pp. 1579-1581, Jun. 2003.
66. K. Ball, "Convex geometry and functional analysis," in Handbook of Banach Space Geometry, W. B. Johnson and J. Lindenstrauss, Eds. Amsterdam, The Netherlands: Elsevier, 2002, pp. 161-193.
67. _, "An elementary introduction to modern convex geometry," in Flavors of Geometry, ser. MSRI Publications. Cambridge, U.K.: Cambridge Univ. Press, 1997, vol. 31, pp. 1-58.