Stable recovery of sparse overcomplete representations in the presence of noise

IEEE Transactions on Information Theory

Volumn 52, Issue 1, 2006, Pages 6-18

  Donoho, David L ^a   Elad, Michael ^b   Temlyakov, Vladimir N ^c  

^a STANFORD UNIVERSITY   (United States)

^b TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

^c UNIVERSITY OF SOUTH CAROLINA   (United States)

Author keywords

Basis pursuit; Greedy approximation; Incoherent dictionary; Kruskal rank; Matching pursuit; Overcomplete representation; Sparse representation; Stability; Stepwise regression; Superresolution

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; INFORMATION THEORY; SIGNAL PROCESSING; SPURIOUS SIGNAL NOISE;

BASIS PURSUIT; GREEDY APPROXIMATION; INCOHERENT DICTIONARY; KRUSKAL RANK; MATCHING PURSUIT; OVERCOMPLETE REPRESENTATION; SPARSE REPRESENTATION; STEPWISE REGRESSION; SUPERRESOLUTION;

SIGNAL THEORY;

EID: 33144483155     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2005.860430     Document Type: Article

References (43)

1. A. P. Berg and W. B. Mikhael, "A survey of mixed transform techniques for speech and image coding", in Proc. 1999 IEEE Int. Symp. Circuits and Systems, vol. 4, Orlando, FL, Jun. 1999, pp. 106-109.
2. 2 edges," Commun. Pure Appl. Math., vol. 57, pp. 219-266, Feb. 2004.
3. S. S. Chen, "Basis Pursuit," Ph.D. dissertation, Stanford Univ., Stanford, CA, Nov. 1995.
4. S. S. Chen,D.L. Donoho, and M A Saunders, "Atomic decomposition by basis pursuit," SIAM Rev., Vol 43, no 1, pp 129-159, 2001.
5. R. Coifman, Y. Meyer and V Wickerhauser, "Adapted wave form analysis, wavelet-packets and applications" in Pro, 2nd Int Colf. Industrial and Applied Mathematics (ICIAM 91), Philadelphia, PA, 1992, pp. 41-50
6. S. F. Cotter and B. D. Rao, "Sparse channel estimation via matching pursuit with application to equalitation," IEEE Trans. Commun., vol. 50, no 3, pp. 374-377, Mar. 2002.
7. C. Daniel and F. S. Wood, Fitting Equations to Data Analysis of Multifactor Data, 2nd ed. New York: Wiley, 1980.
8. V. E. DeBrunner, L.X. Chen, and H.J. Li, "Lapped multiple basis algorithms for still image compression without blocking effect," IEEE Trans. Image Process., vol. 6, pp. 1316-1322, Sep. 1997.
9. R. A. DeVore and V.N. Temlyakov, "Some remarks on greedy algorithms," Adv. Comput. Math., vol. 12, pp. 213-227, 1996.
10. D. L. Donoho, "Superresolution via sparsity constrains," SIAM J. Math. Anal., Vol 23, pp. 1309-1331, 1992.
11. 1 minimization," in Proc. Nat. Acad. Sci., vol. 100, 2003, pp. 2197-2202.
12. D. L. Donoho and M. Elad, and V. Temlyakov, "Stable Recovery of Sparse Overcomplete Representations in the Presence of Noise," Dep. Statistics, Stanford Univ., http://www-stat.stanford-edu/~donoho/Reports/ 2004/StableSparse.pdf, Tech. Rep., 2004.
13. D. L. Donoho and X. Huo, "Uncertainty principles and ideal atomic decomposition," IEEE Trans. Inf. Theory, Vol 47, no 7, pp. 2845-2862, Nov. 2001.
14. N bases," IEEE Trans. Inf. Theory, dvol. 48, no. 9, pp. 2558-2567, Sep. 2002.
15. J. J. Fuchs, "On sparse representations in arbitrary redundant bases," IEEE Trans. Inf. Theory, Vol. 50, no. 6, pp. 1341-1344 Jun. 2004.
16. J. J. Fuchs, "Recovery of exact sparse representations in the presence of bounded noise," IEEE Trans. Inf. Theory, submitted for publication.
17. A. C. Gilbert, S. Muthukrishnan, and M. J. Strauss, "Approximation of functions over redundant dictionaries using coherence," in Proc. 14th Annu. ACM-SIAM Symp. Discrete Algorithms, Baltimore, MD, 2003, pp. 243-252.
18. I.F. Gorodnitsky and B. D. Rao, "Sparse signal reconstruction from limited data using FOCUSS: A re-weighted norm minimization algorithm," IEEE Trans. Signal Process., vol. 45, no. 3, pp. 600-616, Mar. 1997.
19. R. Gribonval and M. Nielsen, "sparse representations in unions of bases," IEEE Trans. Inf. Theory, Vol. 49, no. 12, pp. 3320-3325, Dec. 2003.
20. T. Hastie, R. Tibshirani, and J. H. Friedman, Elements of Statistical Learning. New York: Springer-Verlag, 2001.
21. R. A. Horn and C. R. Johnson, Matrix Analysis. New York: Cambridge Univ. Press, 1985.
22. X. Duo, Sparse Image Representation via Combined Transforms. Stanford, CA: Stanford Univ., Nov., 1999.
23. 1 norms," J. Approx.Theory, Vol. 3, pp. 123-127, 1970.
24. J. B. Kruskal, "Three-way arrays: Rank and uniqueness of trilinear decompositions, with application to arithmetic complexityu and statistics," Linear Alg. Its Applic., vol. 18, no. 2, pp. 95-138, 1977.
25. T. G. Kolda, "Orthogonal tensor decompositions," SIAM J. Matrix Anal. Its Applic., vol. 23, no. 1, pp. 243-255, 2001.
26. X. Liu and N. D. Sidiropoulos, "Cramer-Rao lower bounds, for low-rank decomposition of multidimensional arrays," IEEE Trans. Signal Process., vol. 49, no 9, pp. 2074-2086, Sep. 2001.
27. S. Mallat and Z. Zhang, "Matching pursuit in a time-frequency dictionary," IEEE Trans. Signal Process., vol. 41, no. 12, pp. 3397-3415, Dec. 1993.
28. B. A. Olshausen and D. J. Field, "Sparse coding with an overcomplete basis set: A strategy employed by V1?," Visions Res., vol. 37, pp. 311-325, 1997.
29. Y. C. Pati, R. Rezaiifar, and P. S. Krishnaprasa, "Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition," in Proc. 27th Annu. Asilomar Conf. Signals, Systems, and Computers, Pacific Grove, CA, Nov. 1993, pp. 40-44.
30. S. Qian and D. Chen, "Signal representation using adaptive normalized Gaussian functions," Signal Process., vol. 36, pp. 1-11, 1994.
31. J.-L. Starck, M. Elad, and D. L. Donoho, "Image decomposition: Separation of texture from piecewise smooth content," presented at the SPIE Meeting, San-Diego, CA, Aug. 2003.
32. J.-L. Starck, E. Candès, and D. L. Donoho, "The curvelet transform for image denoising," IEEE Trans. Image Process., vol. 11, no. 6, pp. 131-141, Jun. 2002.
33. T. Strohmer and R. Heath Jr, "Grassmannian frames with applications to coding and communications," Appl. Comp. Harm. Anal., vol. 14, pp. 257-275, May 2003.
34. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Roy. Statist. Soc. B., vol. 58, no. 1, pp. 267-288, 1996.
35. V. N. Temlyacov, "Greedy algorithms and m-term approximation," J. Approx. Theory, vol. 98, pp. 117-145, 1999.
36. V. N. Temlyacov, "Weak greedy algorithms," Adv. Comput. Math., vol. 5, pp. 173-187, 2000.
37. V. N. Temlyacov, "Greedy algorithms with regard to multivariate systems with special structure," Constr. Approx., vol. 16, pp. 399-425, 2000.
38. V. N. Temlyacov, "Nonlinear methods of approximation," Found. Comput. Math., vol. 3, pp. 33-107, 2003.
39. J. A. Tropp, "Greed is good: Algorithmic results for sparse approximation," IEEE Trans. Inf. Theory, vol. 50, no. 10, pp. 2231-2242, Oct. 2004.
40. J. A. Tropp, "Just relax: Convex programming methods for subset selection and sparse approximation," IEEE Trans. Inf. Theory, submitted for publication.
41. R. Venkataramani and Y. Bresler, "Optimal sub-nyquist nonuniform sampling and reconstruction of multiband signals," IEEE Trans. Signal Process., vol. 49, no. 10, pp. 2301-2313, Oct. 2001.
42. M. V. Wickerhauser, Adapted Wavelet Analysis From Theory to Software. Wellesley, MA: A K Peters Ltd., 1994.
43. B. Wohlberg, "Noise sensitivity of sparse signal representations: Reconstruction error bounds for the inverse problem," IEEE Trans. Signal Process vol. 5 1, no. 12, pp. 3053-3060, Dec. 2003.