The LASSO risk for Gaussian matrices

IEEE Transactions on Information Theory

Volumn 58, Issue 4, 2012, Pages 1997-2017

  Bayati, Mohsen ^a   Montanari, Andrea ^b  

^a STANFORD UNIVERSITY   (United States)

^b Stanford University   (United States)

Author keywords

Compressed sensing; graphical models; message passing algorithms; random matrix theory; state evolution; statistical learning

Indexed keywords

COMPRESSED SENSING; GRAPHICAL MODEL; MESSAGE PASSING ALGORITHMS; RANDOM MATRIX THEORY; STATE EVOLUTIONS; STATISTICAL LEARNING;

ALGORITHMS; GRAPHIC METHODS; IMAGE PROCESSING; SPEECH RECOGNITION;

MATRIX ALGEBRA;

EID: 84858956033     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2011.2174612     Document Type: Article

References (28)

1. G. Andrew and J. Gau, "Scalable training of -regularized log-linear models," in Proc. 24th Int. Conf. Mach. Learn., 2007, pp. 33-40.
2. M. Bayati, J. Bento, and A. Montanari, "The LASSO risk: Asymptotic results and real world examples," in Advances in Neural Information Processing Systems, 2010, pp. 145-153.
3. M. Bayati and A. Montanari, "The dynamics of message passing on dense graphs, with applications to compressed sensing," IEEE Trans. Inf. Theory, vol. 57, no. 2, pp. 764-785, Feb. 2011.
4. Z. Bai and J. Silverstein, Spectral Analysis of Large Dimensional Random Matrices. New York: Springer-Verlag, 2005.
5. A. Beck and M. Teboulle, "A fast iterative shrinkage-thresholding algorithm for linear inverse problems," SIAM J. Imag. Sci., vol. 2, pp. 183-202, 2009.
6. Z. D. Bai and Y. Q. Yin, "Limit of the smallest eigenvalue of a large dimensional sample covariance matrix," Ann. Probabil., vol. 21, pp. 1275-1294, 1993.
7. S. Chen and D. L. Donoho, "Examples of basis pursuit," in Proc. SPIE, San Diego, 1995, vol. 2569, pp. 564-574.
8. E. Candes, J. K. Romberg, and T. Tao, "Stable signal recovery from incomplete and inaccurate measurements," Commun. Pure Appl. Math., vol. 59, pp. 1207-1223, 2006. (Pubitemid 43988295)
9. E. Candes and T. Tao, "The Dantzig selector: Statistical estimation when is much larger than ," Ann. Statist., vol. 35, pp. 2313-2351, 2007.
10. D. L. Donoho and I. M. Johnstone, "Minimax risk over balls," Prob. Theor. Rel. Fields, vol. 99, pp. 277-303, 1994.
11. D. L. Donoho and I. M. Johnstone, "Minimax estimation via wavelet shrinkage," Ann. Statist., vol. 26, pp. 879-921, 1998.
12. D. L. Donoho, A. Maleki, and A. Montanari, "Message passing algorithms for compressed sensing," in Proc. Nat. Acad. Sci., 2009, vol. 106, pp. 18914-18919.
13. D. L. Donoho, A. Maleki, and A. Montanari, The Noise Sensitivity Phase Transition in Compressed Sensing, pp. 6920-6941, Oct. 2011.
14. M. Grant and S. Boyd, CVX: MATLAB Software for Disciplined Convex Programming, Version 1.21 2010 [Online]. Available: Http://cvxr.com/cvx
15. D. Guo, D. Baron, and S. Shamai, "A single-letter characterization of optimal noisy compressed sensing," presented at the 47th Annu. Allerton Conf. Communication, Control and Computing Monticello, IL, Sep. 2009.
16. I. Johnstone, "High dimensional statistical inference and random matrices," presented at the Int. Congr. Mathematicians Madrid, Spain, 2006.
17. B. Kashin, "Diameters of some finite-dimensional sets and classes of smooth functions," Math. USSR Izv., vol. 11, pp. 317-333, 1977.
18. S. Korada and A. Montanari, "Applications of Lindeberg Principle in Communications and Statistical Learning," IEEE Inf. Theory pp. 2440-2550, Apr. 2011.
19. Y. Kabashima, T.Wadayama, and T. Tanaka, "A typical reconstruction limit for compressed sensing based on Lp-norm minimization," J. Stat. Mech., pp. L09003-L09003, 2009.
20. M. Ledoux, The Concentration of Measure Phenomenon. Berlin, Germany: American Mathematical Society, 2001.
21. A. E. Litvak, A. Pajor, M. Rudelson, and N. Tomczak-Jaegermann, "Smallest singular value of random matrices and geometry of random polytopes," Adv. Math., vol. 195, pp. 491-523, 2005. (Pubitemid 40664844)
22. M. Mézard and A. Montanari, Information, Physics and Computation. Oxford, U.K.: Oxford Univ. Press, 2009.
23. S. Rangan, A. K. Fletcher, and V. K. Goyal, "Asymptotic analysis of map estimation via the replica method and applications to compressed sensing," in Advances in Neural Information Processing Systems, 2009, pp. 1545-1553.
24. E. Telatar, "Capacity of multi-antenna Gaussian channels," Eur. Trans. Telecommun., vol. 10, pp. 585-595, 1999. (Pubitemid 129568930)
25. R. Tibshirani, "Regression shrinkage and selection with the lasso," J. Roy. Statist. Soc B, vol. 58, pp. 267-288, 1996.
26. P. Zhao and B. Yu, "On model selection consistency of Lasso," J. Mach. Learn. Res., vol. 7, pp. 2541-2563, 2006. (Pubitemid 44866738)
27. M. V. Schrebina and B. Tirozzi, "On the rigorous solution of Gardners problem," Commun. Math. Phys, vol. 234, no. 3, pp. 383-422, 2003.
28. M. Talagrand, Mean Field Models for Spin Glasses: Volume I: Basic Examples. New York: Springer-Verlag, 2010.