Information-theoretic limits on sparse signal recovery: Dense versus sparse measurement matrices

IEEE Transactions on Information Theory

Volumn 56, Issue 6, 2010, Pages 2967-2979

  Wang, Wei ^a   Wainwrigh, Martin J ^a   Ramchandran, Kannan ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

1 relaxation; Compressed sensing; Fano's method; High dimensional statistical inference; Information theoretic bounds; Sparse approximation; Sparse random matrices; Sparsity recovery; Subset selection; Support recovery

Indexed keywords

COMPRESSED SENSING; FANO'S METHOD; HIGH-DIMENSIONAL STATISTICAL INFERENCE; INFORMATION-THEORETIC BOUNDS; SPARSE APPROXIMATION; SPARSE RANDOM MATRICES; SPARSITY RECOVERY; SUBSET SELECTION;

ASYMPTOTIC ANALYSIS; INFORMATION THEORY; MATRIX ALGEBRA; SIGNAL RECONSTRUCTION; STATISTICAL METHODS;

RECOVERY;

EID: 77956932923     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2010.2046199     Document Type: Article

References (35)

1. S. Aeron, M. Zhao, and V. Saligrama, "Information-theoretic bounds to sensing capacity of sensor networks under fixed snr," presented at the Information Theory Workshop, Sep. 2007.
2. S. Aeron, M. Zhao, and V. Saligrama, Fundamental Limits on Sensing Capacity for Sensor Networks and Compressed Sensing, 2008, Tech. Rep. arXiv:0804.3439v1 [cs.IT].
3. M. Akcakaya and V. Tarokh, Shannon Theoretic Limits on Noisy Compressive Sampling 2007, Tech. Rep. arXiv:0711.0366v1 [cs.IT].
4. R. Berinde, A. C. Gilbert, P. Indyk, H. Karloff, and M. J. Strauss, "Combining geometry and combinatorics: A unified approach to sparse signal recovery," presented at the Allerton Conf. Communication, Control and Computing, Monticello, IL, Sep. 2008.
5. L. Birgé, "An alternative point of view on Lepski's method," in State of the Art in Probability and Statistics, ser. IMS Lecture Notes. Beachwood, OH: Institute of Mathematical Statistics, 2001, pp. 113-133.
6. E. Candes, J. Romberg, and T. Tao, "Stable signal recovery from incomplete and inaccurate measurements," Commun. Pure Appl. Math., vol.59, no.8, pp. 1207-1223, Aug. 2006.
7. E. Candes and T. Tao, "Decoding by linear programming," IEEE Trans. Inf. Theory, vol.51, no.12, pp. 4203-4215, Dec. 2005.
8. S. Chen, D. L. Donoho, and M. A. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Sci. Comput., vol.20, no.1, pp. 33-61, 1998.
9. G. Cormode and S. Muthukrishnan, Towards an Algorithmic Theory of Compressed Sensing, Rutgers Univ., 2005, Tech. Rep.
10. T. Cover and J. Thomas, Elements of Information Theory. New York: Wiley, 1991.
11. D. Donoho, "Compressed sensing," IEEE Trans. Inf. Theory, vol.52, no.4, pp. 1289-1306, Apr. 2006.
12. D. Donoho, M. Elad, and V. M. Temlyakov, "Stable recovery of sparse overcomplete representations in the presence of noise," IEEE Trans. Inf. Theory, vol.52, no.1, pp. 6-18, Jan. 2006.
13. A. K. Fletcher, S. Rangan, and V. K. Goyal, Necessary and Sufficient Conditions on Sparsity Pattern Recovery, 2008, Tech. Rep. arXiv:0804. 1839v1 [cs.IT].
14. A. K. Fletcher, S. Rangan, V. K. Goyal, and K. Ramchandran, "Denoising by sparse approximation: Error bounds based on rate-distortion theory," J. Appl. Signal Process., vol.10, pp. 1-19, 2006.
15. A. Gilbert, M. Strauss, J. Tropp, and R. Vershynin, "Algorithmic linear dimension reduction in the ℓ -norm for sparse vectors," presented at the Allerton Conf. Communication, Control and Computing, Monticello, IL, Sep. 2006.
16. R. Z. Has'minskii, "A lower bound on the risks of nonparametric estimates of densities in the uniform metric," Theory Prob. Appl., vol.23, pp. 794-798, 1978.
17. I. A. Ibragimov and R. Z. Has'minskii, Statistical Estimation: Asymptotic Theory. New York: Springer-Verlag, 1981.
18. N. Meinshausen and P. Buhlmann, "High-dimensional graphs and variable selection with the lasso," Ann. Statist., vol.34, no.3, pp. 1436-1462, 2006.
19. A. J. Miller, Subset Selection in Regression. New York: Chapman- Hall, 1990.
20. B. K. Natarajan, "Sparse approximate solutions to linear systems," SIAM J. Comput., vol.24, no.2, pp. 227-234, 1995.
21. D. Omidiran and M. J. Wainwright, High-Dimensional Subset Recovery in Noise: Sparsified Measurements Without Loss of Statistical Efficiency, Dept. Statistics, Univ. California, Berkeley, 2008, Tech. Rep. 753.
22. G. Reeves and M. Gastpar, "Sampling bounds for sparse support recovery in the presence of noise," presented at the Int. Symp. Information Theory, Toronto, Canada, 2008.
23. J. Riordan, Combinatorial Identities. New York: Wiley, 1968, Wiley Series in Probability and Mathematical Statistics.
24. S. Sarvotham, D. Baron, and R. G. Baraniuk, "Measurements versus bits: Compressed sensing meets information theory," presented at the Allerton Conf. Control, Communication and Computing, Sep. 2006.
25. S. Sarvotham, D. Baron, and R. G. Baraniuk, "Sudocodes: Fast measurement and reconstruction of sparse signals," presented at the Int. Symp. Information Theory, Seattle, WA, Jul. 2006.
26. R. J. Serfling, Approximation Theorems of Mathematical Statistics, ser. Wiley Series in Probability and Statistics. New York: Wiley, 1980.
27. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Roy. Statist. Soc. Ser. B, vol.58, no.1, pp. 267-288, 1996.
28. J. Tropp, "Just relax: Convex programming methods for identifying sparse signals in noise," IEEE Trans. Inf. Theory, vol.52, no.3, pp. 1030-1051, Mar. 2006.
29. M. J.Wainwright, "Information-theoretic limits on sparsity recovery in the high-dimensional and noisy setting," IEEE Trans. Inf. Theory, vol.55, no.12, pp. 5728-5741, Dec. 2009.
30. M. J. Wainwright, "Sharp thresholds for high-dimensional and noisy sparsity recovery using ℓ -constrained quadratic programming (Lasso)," IEEE Trans. Inf. Theory, vol.55, no.5, pp. 2183-2202, May 2009.
31. W. Wang, M. Garofalakis, and K. Ramchandran, "Distributed sparse random projections for refinable approximation," presented at the Int. Conf. Information Processing in Sensor Networks, Nashville, TN, Apr. 2007.
32. W. Wang, M. J. Wainwright, and K. Ramchandran, Information-Theoretic Limits on Sparse Support Recovery: Dense Versus Sparse Measurements, Dept. Statistics, Univ. California, Berkeley, 2008, Tech. Rep. 754.
33. W. Xu and B. Hassibi, "Efficient compressed sensing with deterministic guarantees using expander graphs," presented at the Information Theory Workshop (ITW), Sep. 2007.
34. Y. Yang and A. Barron, "Information-theoretic determination of minimax rates of convergence," Ann. Statist., vol.27, no.5, pp. 1564-1599, 1999.
35. B. Yu, "Assouad, Fano and Le Cam," in Festschrift for Lucien Le Cam. Berlin, Germany: Springer-Verlag, 1997, pp. 423-435.