The noise-sensitivity phase transition in compressed sensing

IEEE Transactions on Information Theory

Volumn 57, Issue 10, 2011, Pages 6920-6941

  Donoho, David L ^a   Maleki, Arian ^b   Montanari, Andrea ^a,b  

^a STANFORD UNIVERSITY   (United States)

^b Stanford University   (United States)

Author keywords

Approximate message passing; basis pursuit; LASSO; minimax risk of soft thresholding; minimax risk over nearly black objects

Indexed keywords

APPROXIMATE MESSAGE PASSING; BASIS PURSUITS; COMPRESSED SENSING; COMPUTATIONAL EXPERIMENT; GAUSSIAN WHITE NOISE; GAUSSIANS; LASSO; MAXIMIN; MEAN SQUARED ERROR; MEASUREMENT MATRIX; MINIMAX; MINIMAX RISK; NOISE SENSITIVITY; PHASE SPACES; PHASE TRANSITION CURVES; SOFT-THRESHOLDING ALGORITHM; SPARSE SIGNALS; SYSTEM OF LINEAR EQUATIONS; WORST-CASE NOISE;

GAME THEORY; MESSAGE PASSING; PHASE SPACE METHODS; WHITE NOISE;

PHASE TRANSITIONS;

EID: 80053933128     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2011.2165823     Document Type: Article

References (38)

1. J. D. Blanchard, C. Cartis, and J. Tanner, "Compressed sensing: How sharp is the restricted isometry property," SIAM Rev., vol. 53, pp. 105-125, 2011.
2. M. Bayati and A. Montanari, "The LASSO risk for Gaussian matrices," arXiv:1008.2581, 2010.
3. M. Bayati and A. Montanari, "The dynamics of message passing on dense graphs, with applications to compressed sensing," IEEE Trans. Inf. Theory, vol. 57, no. 2, pp. 764-785, Feb. 2011.
4. A. Beck and M. Teboulle, "A fast iterative shrinkage-thresholding algorithm for linear inverse problems," SIAM J. Imaging Sciences, vol. 2, pp. 183-202, 2009.
5. I. Daubechie, M. Defrise, and C. De Mol, "An iterative thresholding algorithm for linear inverse problems with a sparsity constraint," Commun. Pure and Appl. Mathemat., vol. 57, no. 11, pp. 1413-1457, Nov. 2004.
6. S. S. Chen and D. L. Donoho, "Examples of basis pursuit," in Proc. Wavelet Applications in Signal and Image Processing III, San Diego, CA, 1995.
7. S. S. Chen, D. L. Donoho, and M. A. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Scientif. Comput., vol. 20, pp. 33-61, 1998.
8. E. Candes, J. K. Romberg, and T. Tao, "Stable signal recovery from incomplete and inaccurate measurements," Commun. Pure and Appl. Math., vol. 59, pp. 1207-1223, 2006.
9. E. J. Candes and T. Tao, "Decoding by linear programming," IEEE Trans. Inf. Theory, vol. 51, no. 12, pp. 4203-4215, Dec. 2005.
10. E. J. Candes and T. Tao, "The Dantzig selector: Statistical estimation when p is much larger than n," Ann. Statist., vol. 35, pp. 2313-2351, 2007.
11. D. L. Donoho and I. M. Johnstone, "Minimax risk over balls," Prob. Theory and Rel. Fields, vol. 99, pp. 277-303, 1994.
12. D. L. Donoho, I. M. Johnstone, J. C. Hoch, and A. S. Stern, "Maximum entropy and the nearly black object," J. Roy. Statist. Soc., Ser. B (Methodological), vol. 54, no. 1, pp. 41-81, 1992.
13. pballs: Minimax mean square error," submitted, 2011.
14. D. L. Donoho, A. Maleki, and A. Montanari, "Message passing algorithms for compressed sensing," Proc. National Academy of Sciences, vol. 106, no. 45, pp. 18914-18915, 2009.
15. D. L. Donoho, A. Maleki, and A. Montanari, "Online supplement to message passing algorithms for compressed sensing," in Proc. National Academy of Sciences, 2009, vol. 106, no. 45, pp. 18914-18915.
16. D. L. Donoho, A. Maleki, and A. Montanari, "Message passing algorithms for compressed sensing: I. motivation and construction," in Proc. IEEE Information Theory Workshop, Cairo, Egypt, Jan. 2010.
17. D. L. Donoho, A. Maleki, and A. Montanari, "Theoretical prediction of lasso operating characteristics," 2010, unpublished manuscript.
18. D. L. Donoho, "High-dimensional centrally symmetric polytopes with neighborliness proportional to dimension," Discrete and Comput. Geom., vol. 35, no. 4, pp. 617-652, 2006.
19. D. L. Donoho and J. Tanner, "Neighborliness of randomly-projected simplices in high dimensions," in Proc. National Academy of Sciences, 2005, vol. 102, no. 27, pp. 9452-9457.
20. D. L. Donoho and J. Tanner, "Observed universality of phase transitions in high-dimensional geometry, with implications for modern data analysis and signal processing," Phil. Trans. Roy. Soc. A, vol. 367, pp. 4273-4293, 2009.
21. D. Guo, D. Baron, and S. Shamai, "A single-letter characterization of optimal noisy compressed sensing," in Proc. 47th Annu. Allerton Conf. , Monticello, IL, Sep. 2009.
22. Y. Kabashima, "A CDMA multiuser detection algorithm on the basis of belief propagation," J. Phys. A, vol. 36, pp. 11111-11121, 2003.
23. F. R. Kschischang, B. J. Frey, and H.-A. Loeliger, "Factor graphs and the sum-product algorithm," IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 498-519, 2001.
24. Y. Kabashima, T.Wadayama, and T. Tanaka, "A typical reconstruction limit for compressed sensing based on lp-norm minimization," J. Stat. Mech., p. L09003, 2009.
25. M. Mézard and A. Montanari, Information, Physics and Computation. Oxford, U.K.: Oxford University Press, 2009.
26. C. Méasson, A. Montanari, T. Richardson, and R. Urbanke, "The generalized area theorem and some of its consequences," IEEE Trans. Inf. Theory, vol. 55, no. 11, pp. 4793-4821, Nov. 2009.
27. C. Méasson, A. Montanari, and R. Urbanke, "Maxwell construction: The hidden bridge between iterative and maximum a posteriori decoding," IEEE Trans. Inf. Theory, vol. 54, no. 12, pp. 5277-5307, Dec. 2004.
28. A. Montanari, B. Prabhakar, and D. Tse, "Belief propagation based multi-user detection," in Proc. IEEE Information Theory Workshop, Punta del Este, Uruguay, Mar. 2006.
29. A. Montanari and D. Tse, "Analysis of belief propagation for non-linear problems: The example of CDMA (or: How to prove Tanaka's formula)," in Proc. IEEE Information Theory Workshop, Punta del Este, Uruguay, Mar. 2006.
30. J. P. Neirotti and D. Saad, "Improved message passing for inference in densely connected systems," Europhys. Lett., vol. 71, pp. 866-872, 2005.
31. M. Opper and O. Winther, , M. Opper and D. Saad, Eds., "From naive mean field theory to the TAP equations," in Advanced Mean Field Methods: Theory and Practice. Cambridge, MA: MIT Press, 2001, pp. 7-20.
32. S. Rangan, A. K. Fletcher, and V. K. Goyal, Asymptotic Analysis of Map Estimation Via the Replica Method and Applications to Compressed Sensing 2009, arXiv:0906.3234.
33. T. J. Richardson and R. Urbanke, Modern Coding Theory. Cambridge, U.K.: Cambridge University Press, 2008 [Online]. Available: http://lthcwww.epfl.ch/ mct/index.php, Available online at
34. M. Talagrand, Spin Glasses: A Challenge for Mathematicians. Berlin, Germany: Springer-Verlag, 2003.
35. D. J. Thouless, P. W. Anderson, and R. G. Palmer, "Solution of 'Solvable model of a spin glass'," Phil. Mag., vol. 35, pp. 593-601, 1977.
36. R. Tibshirani, "Regression shrinkage and selection with the LASSO," J. Roy. Statist. Soc. B, vol. 58, pp. 267-288, 1996.
37. W. Xu and B. Hassibi, "On sharp performance bounds for robust sparse signal recovery," in Proc. IEEE Int. Symp. Inform. Theory, Seoul, Korea, Jul. 2009, pp. 493-497.
38. J. S. Yedidia, W. T. Freeman, and Y.Weiss, "Constructing free energy approximations and generalized belief propagation algorithms," IEEE Trans. Inf. Theory, vol. 51, no. 7, pp. 2282-2313, Jul. 2005.