Sparse solution of underdetermined systems of linear equations by stagewise orthogonal matching pursuit

IEEE Transactions on Information Theory

Volumn 58, Issue 2, 2012, Pages 1094-1121

  Donoho, David L ^a   Tsaig, Yaakov ^a   Drori, Iddo ^a   Starck, Jean Luc ^b  

^a STANFORD UNIVERSITY   (United States)

^b UNIVERSITÉ PARIS DIDEROT   (France)

Author keywords

1Minimization; Compressed sensing; Decoding error correcting codes; False alarm rate; False discovery rate; Iterative thresholding; Mutual access interference; Phase transition; Sparse overcomplete representation; Stepwise regression

Indexed keywords

COMPRESSED SENSING; FALSE ALARM RATE; FALSE DISCOVERY RATE; ITERATIVE THRESHOLDING; OVERCOMPLETE REPRESENTATIONS; STEPWISE REGRESSION;

DECODING; INFORMATION THEORY; LINEAR ALGEBRA; PHASE TRANSITIONS; SIGNAL RECONSTRUCTION;

LINEAR EQUATIONS;

EID: 84856938223     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2011.2173241     Document Type: Article

References (61)

1. F. Abramovich, Y. Benjamini, D. L. Donoho, and I. M. Johnstone, "Adapting to unknown sparsity by controlling the false discovery rate," Ann. Statist., vol. 34, no. 2, pp. 584-653, 2006.
2. Y. Benjamini and Y. Hochberg, "Controlling the false discovery rate: A practical and powerful approach to multiple testing," J. R. Statist. Soc., Series B, vol. 57, pp. 289-300, 1995.
3. E. R. Berlekamp, R. J. McEliece, and H. C. A. van Tilborg, "On the inherent intractability of certain coding problems," IEEE Trans. Inf. Theory, vol. AC-24, pp. 384-386, 1978.
4. J. Boutros and G. Caire, "Iterative multiuser joint decoding: Unified framework and asymptotic analysis," IEEE Trans. Inf. Theory, vol. 48, no. 7, pp. 1772-1793, Jul. 2002.
5. J. Buckheit and D. L. Donoho, "WaveLab and reproducible research," in Wavelets and Statistics, A. Antoniadis, Ed. New York: Springer, 1995.
6. G. Caire, R.Müller, and T. Tanaka, "Iterative multiuser joint decoding: Optimal power allocation and low-complexity implementation," IEEE Trans. Inf. Theory, vol. 50, no. 9, pp. 1950-1973, Sep. 2004.
7. E. J. Candès, J. Romberg, and T. Tao, "Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information," IEEE Trans. Inf. Theory, vol. 52, no. 2, pp. 489-509, Feb. 2006.
8. E. J. Candès and T. Tao, "Near optimal signal recovery from random projections: Universal encoding strategies?," IEEE Trans. Inf. Theory, vol. 52, no. 12, pp. 5406-5425, Dec. 2006.
9. E. J. Candès and T. Tao, "Decoding via linear programming," IEEE Trans. Inf. Theory, vol. 51, no. 12, pp. 4203-4215, Dec. 2005.
10. N. Chayat and S. Shamai, "Convergence properties of iterative soft onion peeling," in Proc. 1999 IEEE Inf. Theory Workshop, Kruger National Park, South Africa, 1999.
11. S. Chen, D. Donoho, and M. A. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Sci Comput., vol. 20, no. 1, pp. 33-61, 1999.
12. E. Chong, J. Zhang, and D. Tse, "Output MAI distribution of linear MMSE multiuser receivers in DS-CDMA systems," IEEE Trans. Inf. Theory, vol. 47, no. 3, pp. 1128-1144, Mar. 2001.
13. S. Y. Chung, T. J. Richardson, and R. L. Urbanke, "Analysis of Sum- Product decoding of low-density parity-check codes using a Gaussian approximation," IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 657-670, Feb. 2001.
14. R. R. Coifman and M. V.Wickerhauser, "Wavelets and adapted waveform analysis: A toolkit for signal processing and numerical analysis," in Different Perspectives on Wavelets, I. Daubechies, Ed. Providence, RI: Amer. Math. Soc., 1993, vol. 47, Proceedings of Symposia in Applied Mathematics.
15. I. Daubechies, M. Defrise, and C. De Mol, "An iterative thresholding algorithm for linear inverse problems with a sparsity constraint," Commun. Pure Appl. Math., vol. 57, pp. 1413-1457, 2004.
16. D. L. Donoho, "For most large underdetermined systems of linear equations, the minimal -norm solution is also the sparsest solution," Commun. Pure Appl. Math., vol. 59, no. 6, pp. 797-829, Jun. 2006.
17. D. L. Donoho, "Compressed sensing," IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289-1306, Apr. 2004.
18. D. L. Donoho, Neighborly polytopes and sparse solution of underdetermined linear equations Dept. of Statistics, Stanford Univ., Stanford, CA, Tech. Rep. 2005-4, 2005 [Online]. Available: http://statistics.stanford. edu/~ckirby/techreports/GEN/2005/2005-04.pdf
19. D. L. Donoho, "High-dimensional centrosymmetric polytopes with neighborliness proportional to dimension," Discr. Comput. Geom., vol. 35, no. 4, pp. 617-652, 2006.
20. D. L. Donoho and M. Elad, "Optimally sparse representation from overcomplete dictionaries via norm minimization," Proc. Natl. Acad. Sci. USA, vol. 100, no. 5, pp. 2197-2002, 2003.
21. D. L. Donoho and X. Huo, "Uncertainty principles and ideal atomic decomposition," IEEE Trans. Inf. Theory, vol. 47, no. 7, pp. 2845-62, Jul. 2001.
22. D. L. Donoho and X. Huo, "BeamLab and reproducible research," Int. J. Wavelets,Multiresolut. Inf. Process., vol. 2, no. 4, pp. 391-414, 2004.
23. D. L. Donoho and J. Jin, "Asymptotic minimaxity of false discovery rate thresholding for exponential data," Ann. Statist., vol. 34, no. 6, pp. 2980-3018, 2006.
24. D. L. Donoho and I. M. Johnstone, "Minimax risk over -balls for -error," Probab. Theory Related Fields, vol. 99, pp. 277-303, 1994.
25. D. L. Donoho and I.M. Johnstone, "Ideal spatial adaptation via wavelet shrinkage," Biometrika, vol. 81, pp. 425-455, 1994.
26. D. L. Donoho, I.M. Johnstone, A. S. Stern, and J. C. Hoch, "Maximum entropy and the nearly black object (with discussion)," J. R. Statist. Soc., Series B, vol. 54, no. 1, pp. 41-81, 1992.
27. D. L. Donoho and I. M. Johnstone, "Adapting to unknown smoothness via wavelet shrinkage," J. Amer. Statist. Assoc., vol. 90, pp. 1200-1224, 1995.
28. D. L. Donoho and B. F. Logan, "Signal recovery and the large Sieve," SIAM J. Appl. Math., vol. 52, pp. 577-591, 1992.
29. M. F. Duarte, M. B. Wakin, and R. G. Baraniuk, "Fast reconstruction of piecewise smooth signals from random projections," in Proc. SPARS Workshop, Nov. 2005.
30. H. El Gamal and A. R. Hammons Jr., "Analyzing the turbo decoder using the Gaussian approximation," IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 671-686, Feb. 2001.
31. R. Freeman and E. Kupce, "New methods for fast multidimensional NMR," J. Biomolec. NMR, vol. 27, pp. 101-113, 2003.
32. J.-J. Fuchs, "On sparse representations in arbitrary redundant bases," IEEE Trans. Inf. Theory, vol. 50, no. 6, pp. 1341-44, Jun. 2002.
33. A. C. Gilbert, S. Guha, P. Indyk, S. Muthukrishnan, and M. Strauss, "Near-optimal sparse Fourier representations via sampling," in Proc 34th ACM Symp. Theory Computing, 2002, pp. 152-161.
34. G. Golub and C. van Loan, Matrix Computations, 3rd ed. London, U.K.: The Johns Hopkins Univ. Press, 1996.
35. R. Gribonval and M. Nielsen, "Sparse representations in unions of bases," IEEE Trans. Inf. Theory, vol. 49, no. 12, pp. 1320-1325, Dec. 2003.
36. D. Guo, S. Verdú, and L. Rasmussen, "Asymptotic normality of linear multiuser receiver outputs," IEEE Trans. Inf. Theory, vol. 48, no. 12, pp. 3080-3095, Dec. 2002.
37. I. M. Johnstone and B.W. Silverman, "Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences," Ann. Statist., vol. 32, pp. 1594-1649, 2004.
38. Y. Hochberg and A. Tamhane, Multiple Comparison Procedures. New York: Wiley, 1987.
39. E. Lehmann and J. Romano, Testing Statistical Hypotheses, 3rd ed. New York: Springer, 2003.
40. M. Lustig, J. H. Lee, D. L. Donoho, and J. M. Pauly, "Faster imaging with randomly perturbed spirals and reconstruction," in Proc. ISMRM 13th Annu. Meeting, 2004.
41. M. Lustig, J. M. Santos, D. L. Donoho, and J. M. Pauly, "Rapid MR angiography with randomly under-sampled 3DFT trajectories and nonlinear reconstruction," in Proc. SCMR 9th Annu. Sci. Meeting, 2006.
42. M.W. Maciejewski, A. S. Stern, G. F. King, and J. C. Hoch, "Nonuniform sampling in biomolecular NMR," in Handbook of Modern Magnetic Resonance, G. A. Webb, Ed. New York: Springer, 2006.
43. S. Mallat and Z. Zhang, "Matching pursuit with time-frequency dictionaries," IEEE Trans. Signal Process., vol. 41, no. 12, pp. 3397-3415, Dec. 1993.
44. P. McCullagh, TensorMethods in Statistics. London, U.K.: Chapman and Hall, 1987.
45. A. Paulraj, R. Nabar, and D. Gore, Introduction to Space-TimeWireless Communications. Cambridge, U.K.: Cambridge Univ. Press, 2003.
46. H. V. Poor and S. Verdú, "Probability of error in MMSE multiuser detection," IEEE Trans. Inf. Theory, vol. 43, no. 3, pp. 858-871, May 1997.
47. D. Rovnyak, D. P. Frueh,M. Sastry, Z. J. Sun, A. S. Stern, J. C. Hoch, and G. Wagner, "Accelerated acquisition of high resolution triple-resonance spectra using non-uniform sampling and maximum entropy reconstruction," J. Magn. Resonance, vol. 170, no. 1, pp. 15-21, 2004.
48. M. Rudelson and R. Vershynin, "Geometric approach to error-correcting codes and reconstruction of signals," Int. Math. Res. Notices, vol. 2005, pp. 4019-4041, 2005.
49. M. A. Saunders, PDCO: Primal-Dual Interior Method for Convex Objectives 2002 [Online]. Available: http://www.stanford.edu/group/ SOL/software/pdco.html
50. J.-L. Starck, M. Elad, and D. L. Donoho, "Redundant multiscale transforms and their application for morphological component analysis," Adv. Imaging Electron Phys., vol. 132, 2004.
51. J.-L. Starck, M. Elad, and D. L. Donoho, "Image decomposition via the combination of sparse representation and a variational approach," IEEE Trans. Image Process., vol. 14, no. 10, pp. 1570-1582, Oct. 2005.
52. V. N. Temlyakov, "Greedy algorithms and -term approximation," J. Approx. Theory, vol. 98, pp. 117-145, 1999.
53. V. N. Temlyakov, "Nonlinear methods of approximation," Found. Comput. Math., vol. 3, pp. 33-107, 2003.
54. J. A. Tropp and A. Gilbert, "Signal recovery from partial information by orthogonal matching pursuit. Manuscript," IEEE Trans. Inf. Theory, vol. 53, no. 12, pp. 4655-4666, Dec. 2007.
55. J. A. Tropp, "Greed is good: Algorithmic results for sparse approximation," IEEE Trans Inf. Theory, vol. 50, no. 11, pp. 2231-42, Nov. 2003.
56. J. A. Tropp, "Just relax: Convex programming methods for subset selection and sparse approximation," IEEE Trans. Inf. Theory, vol. 52, no. 3, pp. 1030-1051, Mar. 2006.
57. Y. Tsaig and D. L. Donoho, "Breakdown of equivalence between the minimal -norm solution and the sparsest solution," Signal Process., vol. 86, no. 3, pp. 533-548, Mar. 2006.
58. Y. Tsaig and D. L. Donoho, "Extensions of compressed sensing," Signal Process., vol. 86, no. 3, pp. 549-571, Mar. 2006.
59. D. N. C. Tse and S. Verdú, "Optimum asymptotic multiuser efficiency of randomly spread CDMA," IEEE Trans Inf. Theory, vol. 46, no. 7, pp. 2718-2722, Nov. 2000.
60. S. Verdú, Multiuser Detection. Cambridge, U.K.: Cambridge Univ. Press, 1998.
61. S. Verdú and S. Shamai, "Spectral efficiency of CDMA with random spreading," IEEE Trans. Inf. Theory, vol. 45, no. 2, pp. 622-640, Mar. 1999.