The dynamics of message passing on dense graphs, with applications to compressed sensing

IEEE Transactions on Information Theory

Volumn 57, Issue 2, 2011, Pages 764-785

  Bayati, Mohsen ^a   Montanari, Andrea ^b  

^a Stanford University   (United States)

^b STANFORD UNIVERSITY   (United States)

Author keywords

Compressed sensing; density evolution; message passing algorithms; random matrix theory; state evolution

Indexed keywords

COMPRESSED SENSING; DENSITY EVOLUTION; MESSAGE PASSING ALGORITHMS; RANDOM MATRIX THEORY; STATE EVOLUTIONS;

MATRIX ALGEBRA; MESSAGE PASSING; SIGNAL RECONSTRUCTION; SPIN GLASS;

EVOLUTIONARY ALGORITHMS;

EID: 79251496987     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2010.2094817     Document Type: Article

References (34)

1. D. Aldous, "The¢ (2) limit in the random assignment problem," Random Struct. Algorith., vol. 18, pp. 381-418, 2001.
2. D. Aldous and J. M. Steele, "The objective method: Probabilistic combinatorial optimization and local weak convergence," in Probability On Discrete Structures, H. Kesten, Ed. New York: Springer-Verlag, 2003, pp. 1-72.
3. M. Bayati and A. Montanari, The LASSO risk for gaussian matrices, 2010, preprint. [Online]. Available: http://arxiv.org/abs/1008.2581
4. E. Bolthausen, On the high-temperature phase of the Sherrington- Kirkpatrick model, Seminar presented at EURANDOM. Eindhoven, The Netherlands, September 2009 [Online]. Available: http://www.eurandom. nl/events/workshops/ 2009/OD/Order/Bolthausen.pdf
5. d many questions and a few answers," Electron. Commun. Probab., vol. 1, pp. 71-82, 1996.
6. I. Daubechies, M. Defrise, and C. De Mol, "An iterative thresholding algorithm for linear inverse problems with a sparsity constraint," Commun. Pure and Appl. Math., vol. 75, pp. 1412-1457, 2004.
7. D. L. Donoho and I. M. Johnstone, "Ideal spatial adaptation viawavelet shrinkage," Biometrika, vol. 81, pp. 425-455, 1994.
8. D. L. Donoho and I. M. Johnstone, "Minimax risk over balls," Prob. Theory Rel. Fields, vol. 99, pp. 277-303, 1994.
9. D. L. Donoho and A. Maleki, "Optimally tuned iterative thresholding algorithm for compressed sensing," IEEE J. Sel. Areas Signal Process. 2009 [Online]. Available: http://arxiv.org/abs/0909.0777, to be published
10. D. L. Donoho, A. Maleki, and A. Montanari, "Message passing algorithms for compressed sensing," Proc. Nat. Acad. Sci., vol. 106, pp. 18914-18919, 2009.
11. D. L. Donoho, A. Maleki, and A. Montanari, "Message passing algorithms for compressed sensing: I. Motivation and construction," in Proc. IEEE Inf. Theory Workshop, Cairo, Egypt, 2010.
12. D. L. Donoho, A. Maleki, and A. Montanari, "On the construction of approximate message passing algorithms", to be published.
13. D. L. Donoho, A. Maleki, and A. Montanari, The Noise-Sensitivity Phase Transition in Compressed Sensing [Online]. Available: http:// arxiv.org/abs/1004. 1218
14. 1 -norm near-solution approximates the sparsest near-solution," Commun. Pure and Appl. Math., vol. 59, pp. 907-934, 2006.
15. D. L. Donoho and J. Tanner, "Neighborliness of randomly-projected simplices in high dimensions," Proc. Nat. Acad. Sci., vol. 102, no. 27, pp. 9452-9457, 2005.
16. D. L. Donoho and J. Tanner, "Counting faces of randomly projected polytopes when the projection radically lowers dimension," J. Amer. Math. Soc., vol. 22, pp. 1-53, 2009.
17. D. Guo, D. Baron, and S. Shamai, "A single-letter characterization of optimal noisy compressed sensing," in Proc. 47th Ann. Allerton Conf., Monticello, IL, Sep. 2009.
18. D. Guo and S. Verdu, "Randomly spread CDMA: Asymptotics via statistical physics," IEEE Trans. Inf. Theory, vol. 51, pp. 1982-2010, 2005.
19. T. C. Hu and R. L. Taylor, "Strong law for arrays and for the bootstrap mean and variance," Int. J. Math. and Math. Sci., vol. 20, pp. 375-383, 1997.
20. Y. Kabashima, "A CDMA multiuser detection algorithm on the basis of belief propagation," J. Phys. A, vol. 36, pp. 11111-11121, 2003.
21. Y. Kabashima, T.Wadayama, and T. Tanaka, "A typical reconstruction limit for compressed sensing based on -norm minimization," J. Stat. Mech., 2009.
22. M. Mézard and A. Montanari, Information, Physics and Computation. Oxford, U.K.: Oxford Univ. Press, 2009.
23. V. A. Marcenko and L. A. Pastur, "Distribution of eignevalues for some sets of random matrices," Math. USSR-Sbornik, vol. 1, 1967.
24. A. Montanari and D. Tse, "Analysis of belief propagation for non-linear problems: The example of CDMA (or: how to prove Tanaka's formula)," in Proc. IEEE Inf. TheoryWorkshop, Punta de l'Este, Uruguay, 2006.
25. J. P. Neirotti and D. Saad, "Improved message passing for inference in densely connected systems," Europhys. Lett., vol. 71, pp. 866-872, 2005.
26. S. Rangan, A. K. Fletcher, and V. K. Goyal, "Asymptotic analysis of MAP estimation via the replica method and applications to compressed sensing," in Proc. Advances in Neural Information Processing Systems (NIPS), 2009, vol. 22, pp. 1545-1553.
27. T. J. Richardson and R. Urbanke, Modern Coding Theory. Cambridge, U.K.: Cambridge Univ. Press, 2008.
28. J. Salez and D. Shah, "Belief propagation: Optimal algorithm for random assignment problem," Math. Operat. Res., vol. 32, pp. 468-480, 2009.
29. C. Stein, "A bound for the error in the normal approximation to the distribution of a sum of dependent random variables," in Proc. 6th Berkeley Symp. Math. Statist. Probab., 1972.
30. M. Talagrand, Mean Field Models for Spin Glasses. Berlin, Germany: Springer, 2010, Vol. I: Basic Examples.
31. T. Tanaka, "A statistical-mechanics approach to large-system analysis of CDMA multiuser detectors," IEEE Trans. Inf. Theory, vol. 48, pp. 2888-2910, 2002.
32. T. Tanaka, "Statistical physics of information processing in mobile communications," Progr. Theory Phys. Suppl., vol. 157, pp. 176-183, 2005.
33. D. N. C. Tse and S. V. Hanly, "Linear multiuser receivers: Effective interference, effective bandwidth and user capacity," IEEE Trans. Inf. Theory, vol. 45, pp. 641-657, 1999.
34. S. Verdú and S. Shamai, "Spectral efficiency of CDMA with random spreading," IEEE Trans. Inf. Theory, vol. 45, pp. 622-640, 1999.