Graphical models concepts in compressed sensing

Compressed Sensing: Theory and Applications

Volumn , Issue , 2009, Pages 394-438

  Montanari, Andrea ^a  

^a STANFORD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPRESSED SENSING; GRAPHIC METHODS; REGRESSION ANALYSIS; RISK ASSESSMENT; SIGNAL RECONSTRUCTION;

GRAPHICAL MODEL; HIGH-DIMENSIONAL; MESSAGE PASSING ALGORITHM; PENALIZED LEAST-SQUARES; REGRESSION PROBLEM;

MESSAGE PASSING;

EID: 84938523795     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1017/CBO9780511794308.010     Document Type: Chapter

References (85)

1. G. Andrew and G. Jianfeng, Scalable training of l1 -regularized log-linear models. Proc 24th Inte Conf Mach Learning, 2007, pp. 33-40.
2. R. Affentranger and R. Schneider, Random projections of regular simplices. Discr Comput Geom, 7:219-226, 1992.
3. D. Aldous and J. M. Steele. The objective method: probabilistic combinatorial optimization and local weak convergence. Probability on Discrete Structures H. Kesten, ed., Springer Verlag, pp. 1-72, 2003.
4. M. Bayati, J. Bento, and A. Montanari. Universality in sparse reconstruction: A comparison between theories and empirical results, in preparation, 2011.
5. R. G. Baraniuk, V. Cevher, M. F. Duarte, and C. Hegde. Model-based compressive sensing, IEEE Trans Inform Theory 56:1982-2001, 2010.
6. R. Berinde, A. C. Gilbert, P. Indyk, H. Karloff, and M. J. Strauss. Combining geometry and combinatorics: a unified approach to sparse signal recovery. 46th Ann Allerton Conf (Monticello, IL), September 2008.
7. P. Billingsley. Probability and Measure, Wiley, 1995.
8. M. Bayati and A. Montanari. Approximate message passing algorithms for generalized linear models, in preparation, 2010.
9. M. Bayati and A. Montanari. The dynamics of message passing on dense graphs, with applications to compressed sensing. IEEE Trans Inform Theory, accepted, http://arxiv.org/pdf/1001.3448, 2011.
10. S. S. Chen and D. L. Donoho. Examples of basis pursuit. Proc Wavelet Appl Sig Image Proc III, San Diego, CA, 1995.
11. V. Cevher, Learning with compressible priors. Neur Inform Proc Syst, Vancouver, 2008.
12. V. Cevher, C. Hegde, M. F. Duarte, and R. G. Baraniuk, Sparse signal recovery using Markov random fields. Neur Inform Proc Syst, Vancouver, 2008.
13. V. Cevher, P. Indyk, L. Carin, and R. G. Baraniuk. Sparse signal recovery and acquisition with graphical models. IEEE Sig Process Mag 27:92-103, 2010.
14. G. Cormode and S. Muthukrishnan. Improved data streams summaries: The count-min sketch and its applications. LATIN, Buenos Aires, pp. 29-38, 2004.
15. E. J. Candes and Y. Plan. Near-ideal model selection by l1 minimization. Ann Statist, 37:21452177, 2009.
16. E. J. Candès and Y. Plan, Matrix completion with noise. Proc IEEE 98:925-936, 2010.
17. E. J. Candès and Y. Plan. A probabilistic and ripless theory of compressed sensing. arXiv:1011.3854, November 2010.
18. E. J. Candes and B. Recht. Exact matrix completion via convex optimization. Found Comput Math 9:717-772, 2009.
19. E. Candes, J. K. Romberg, and T. Tao. Stable signal recovery from incomplete and inaccurate measurements. Commun Pure Appl Math, 59:1207-1223, 2006.
20. V. Chandar, D. Shah, and G. W. Wornell, A simple message-passing algorithm for compressed sensing, Proc IEEE Int Symp Inform Theory (ISIT) (Austin), 2010.
21. E. Candes and T. Tao. The Dantzig selector: statistical estimation when p is much larger than n. Ann Stat, 35:2313-2351, 2007.
22. D. L. Donoho and I. M. Johnstone. Ideal spatial adaptation via wavelet shrinkage. Biometrika, 81:425-455, 1994.
23. D. L. Donoho and I. M. Johnstone. Minimax risk over lp balls. Prob Th Rel Fields, 99:277-303, 1994.
24. D. L. Donoho, I. M. Johnstone, J. C. Hoch, and A. S. Stern. Maximum entropy and the nearly black object. J Roy Statist Soc, Ser. B (Methodological), 54(1):41-81, 1992.
25. D. Donoho and A. Montanari. Approximate message passing for reconstruction of block- sparse signals, in preparation, 2010.
26. D. L. Donoho, A. Maleki, and A. Montanari. Message passing algorithms for compressed sensing. Proc Nat Acad Sci 106:18914-18919, 2009.
27. D. L. Donoho, A. Maleki, and A. Montanari. Message passing algorithms for compressed sensing: I. Motivation and construction. Proc IEEE Inform Theory Workshop, Cairo, 2010.
28. D. L. Donoho, A. Maleki, and A. Montanari. The noise sensitivity phase transition in compressed sensing, http://arxiv.org/abs/1004.1218, 2010.
29. D. Donoho. High-dimensional centrally symmetric polytopes with neighborliness proportional to dimension. Discr Comput Geom, 35:617-652, 2006.
30. D. L. Donoho and J. Tanner. Neighborliness of randomly-projected simplices in high dimensions. Proc Nat Acad Sci, 102(27):9452-9457, 2005.
31. D. L. Donoho and J. Tanner. Counting faces of randomly projected polytopes when the projection radically lowers dimension, J Am Math Soc, 22:1-53, 2009.
32. Y. C. Eldar, P. Kuppinger, and H. Bolcskei. Block-sparse signals: uncertainty relations and efficient recovery. IEEE Trans Sig Proc, 58:3042-3054, 2010.
33. M. A. T. Figueiredo and R. D. Nowak. An EM algorithm for wavelet-based image restoration. IEEE Trans Image Proc, 12:906-916, 2003.
34. M. Grant and S. Boyd. CVX: Matlab software for disciplined convex programming, version 1.21, http://cvxr.com/cvx, May 2010.
35. D. Guo, D. Baron, and S. Shamai. A single-letter characterization of optimalnoisy compressed sensing. 47th Ann Allerton Conf (Monticello, IL), September 2009.
36. D. Gamarnik and D. Katz. Correlation decay and deterministic FPTAS for counting list- colorings of a graph. 18th Ann ACM-SIAM Symp Discrete Algorithms, New Orleans, 1245-1254, 2007.
37. D. Gross. Recovering low-rank matrices from few coefficients in any basis. arXiv:0910.1879v2, 2009.
38. D. Guo and S. Verdu. Randomly spread CDMA: asymptotics via statistical physics. IEEE Trans Inform Theory, 51:1982-2010, 2005.
39. T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning, Springer- Verlag, 2003.
40. P. Indyk. Explicit constructions for compressed sensing of sparse signals, 19th Ann ACM- SIAM Symp Discrete Algorithm, San Francisco, 2008.
41. I. Johnstone, Function Estimation and Gaussian Sequence Models. Draft of a book, available at www-stat.stanford.edu/~imj/based.pdf, 2002.
42. M. Jordan (ed.). Learning in Graphical Models, MIT Press, 1998.
43. S. Ji, Y. Xue, and L. Carin. Bayesian compressive sensing. IEEE Trans Sig Proc, 56:2346-2356, 2008.
44. D. Koller and N. Friedman. Probabilistic Graphical Models, MIT Press, 2009.
45. R. H. Keshavan and A. Montanari. Fast algorithms for matrix completion, in preparation, 2010.
46. S. Korada and A. Montanari. Applications of Lindeberg Principle in communications and statistical learning. http://arxiv.org/abs/1004.0557, 2010.
47. R. H. Keshavan, A. Montanari, and S. Oh. Learning low rank matrices from O(n) entries. Proc Allerton Conf Commun, Control Comput, September 2008, arXiv:0812.2599.
48. R. H. Keshavan, A. Montanari, and S. Oh, Matrix completion from a few entries. IEEE Trans Inform Theory, 56:2980-2998, 2010.
49. R. H. Keshavan, A. Montanari, and S. Oh. Matrix completion from noisy entries. J Mach Learn Res 11:2057-2078, 2010.
50. Y. Kabashima, T. Wadayama, and T. Tanaka, A typical reconstruction limit for compressed sensing based on lp-norm minimization. J Stat Mech, L09003, 2009.
51. B.-H. Kim, A. Yedla, and H. D. Pfister. Imp: A message-passing algorithm for matrix completion. Proc Stand 6th Inte Symp Turbo Codes, September 2010, arXiv:1007.0481.
52. E.L. Lehmann and G. Casella. Theory of Point Estimation, Springer-Verlag, 1998.
53. M. Ledoux, The Concentration of Measure Phenomenon, American Mathematical Society, 2001.
54. Y. Lu, A. Montanari, and B. Prabhakar, Detailed network measurements using sparse graph counters: The Theory, 45th Ann Allerton Confe Monticello, IL, 2007.
55. Y. Lu, A. Montanari, and B. Prabhakar. Counter braids: asymptotic optimality of the message passing decoding algorithm. 46th Ann Allerton Confe, Monticello, IL, 2008.
56. Y. Lu, A. Montanari, B. Prabhakar, S. Dharmapurikar, and A. Kabbani. Counter braids: a novel counter architecture for per-flow measurement. SIGMETRICS 2010, 2008.
57. A. Maleki and D. L. Donoho, Optimally tuned iterative thresholding algorithm for compressed sensing. IEEE J Sel Topics Sig Process, 4:330-341, 2010.
58. M. Mezard and A. Montanari. Information, Physics and Computation. Oxford University Press, 2009.
59. A. Montanari. Estimating random variables from random sparse observations. Eur Trans Telecom, 19:385-403, 2008.
60. M. Mezard, G. Parisi, and M. A. Virasoro. Spin Glass Theory and Beyond. World Scientific, 1987.
61. C. Moallemi and B. Van Roy. Convergence of the min-sum algorithm for convex optimization. 45th Ann Allerton Confe (Monticello, IL), September 2007.
62. A. Montanari and D. Tse. Analysis of belief propagation for non-linear problems: the example of CDMA (or: how to prove Tanaka's formula). Proc IEEE Inform Theory Workshop (Punta de l’Este, Uruguay), 2006.
63. S. Negahban and M. J. Wainwright. Estimation of (near) low-rank matrices with noise and high-dimensional scaling. arXiv:0912.5100, 2009.
64. L. Onsager. Electric moments of molecules in liquids. J Am Chem Soc, 58:1486-1493, 1936.
65. J. Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, 1988.
66. S. Rangan. Generalized approximate message passing for estimation with random linear mixing. arXiv:1010.5141, 2010.
67. S. Rangan, A. K. Fletcher, and V. K. Goyal. Asymptotic Analysis of Map Estimation via the Replica Method and Applications to Compressed Sensing, 2009.
68. T. J. Richardson and R. Urbanke. Modern Coding Theory, Cambridge University Press, 2008.
69. S. Sarvotham, D. Baron, and R. Baraniuk. Bayesian compressive sensing via belief propagation. IEEE Trans Sig Proc, 58:269-280, 2010.
70. A. Singer and M. Cucuringu. Uniqueness of low-rank matrix completion by rigidity theory. arXiv:0902.3846, 2009.
71. P. Schniter. Turbo reconstruction of structured sparse signals. Proc Conf Inform Sci Syst, Princeton, NJ, 2010.
72. S. Som, L. C. Potter, and P. Schniter. Compressive imaging using approximate message passing and a Markov-Tree Prior. Proc Asilomar Conf Sig Syst Comput, 2010.
73. L. C. Potter, S. Som, and P. Schniter. On approximate message passing for reconstruction of non-uniformly sparse signals, Proc Nati Aereospace Electron Confe, Dayton, OH, 2010.
74. M. Stojnic, F. Pavaresh, and B. Hassibi. On the reconstruction of block-sparse signals with an optimal number of measurements. IEEE Trans Sig Proc, 57:3075-3085, 2009.
75. T. Tanaka. A statistical-mechanics approach to large-system analysis of CDMA multiuser detectors. IEEE Trans Inform Theory, 48:2888-2910, 2002.
76. D. J. Thouless, P. W. Anderson, and R. G. Palmer. Solution of “Solvable model of a spin glass.” Phil Mag 35:593-601, 1977.
77. R. Tibshirani. Regression shrinkage and selection with the lasso. J Roy Stat Soc, B 58:267-288, 1996.
78. J. A. Tropp, J. N. Laska, M. F. Duarte, J. K. Romberg, and R. G. Baraniuk. Beyond Nyquist: efficient sampling of sparse bandlimited signals. IEEE Trans Inform. Theory, 56:520-544, 2010.
79. T. Tanaka and J. Raymond. Optimal incorporation of sparsity information by weighted L1 optimization. Proc IEEE Int Symp Inform Theory (ISIT), Austin, 2010.
80. A. W. van der Vaart. Asymptotic Statistics. Cambridge University Press, 2000.
81. D. Weitz. Combinatorial criteria for uniqueness of Gibbs measures. Rand Struct Alg, 27:445-475, 2005.
82. D. Williams. Probability with Martingales. Cambridge University Press, 1991.
83. M. J. Wainwright and M. I. Jordan. Graphical models, exponential families, and variational inference. Found Trends Mach Learn, 1, 2008.
84. W. Xu and B. Hassibi. Efficient compressive sensing with deterministic guarantees using expander graphs. Proc IEEE Inform Theory Workshop, Tahoe City, CA, 2007.
85. P. Zhao and B. Yu. On model selection consistency of Lasso. J Mach Learn Res, 7:2541-2563, 2006.