Bayesian compressive sensing via belief propagation

IEEE Transactions on Signal Processing

Volumn 58, Issue 1, 2010, Pages 269-280

  Baron, Dror ^a   Sarvotham, Shriram ^b   Baraniuk, Richard G ^c  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

^b HALLIBURTON   (United States)

^c Rice University   (United States)

Author keywords

Bayesian inference; Belief propagation; Compressive sensing; Fast algorithms; Sparse matrices

Indexed keywords

ASYMPTOTICALLY OPTIMAL; BAYESIAN; BAYESIAN INFERENCE; BELIEF PROPAGATION; BELIEF PROPAGATION DECODING; DECODING ALGORITHM; ENCODING MATRIX; FAST ALGORITHMS; FAST COMPUTATION; GRAPHICAL MODEL; GREEDY ALGORITHMS; LINEAR PROJECTIONS; MIXTURE GAUSSIAN MODEL; NYQUIST; SIGNAL ACQUISITIONS; SIGNAL MODELS; SPARSE MATRICES; SPARSE SIGNALS; STATISTICAL CHARACTERIZATION; TWO-STATE;

ALGORITHMS; BACKPROPAGATION; BACKPROPAGATION ALGORITHMS; BAYESIAN NETWORKS; DECODING; ENCODING (SYMBOLS); GRAPHIC METHODS; LINEARIZATION; MATRIX ALGEBRA; SIGNAL PROCESSING; SIGNAL RECONSTRUCTION;

INFERENCE ENGINES;

EID: 72949124697     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2009.2027773     Document Type: Article

References (60)

1. S. Sarvotham, D. Baron, and R. G. Baraniuk, "Compressed sensing reconstruction via belief propagation," Rice Univ., Houston, TX, Tech. Rep. TREE0601, Jul. 2006.
2. R. A. DeVore, B. Jawerth, and B. J. Lucier, "Image compression through wavelet transform coding," IEEE Trans. Inf. Theory, vol.38, no.2, pp. 719-746, Mar. 1992.
3. I. F. Gorodnitsky and B. D. Rao, "Sparse signal reconstruction from limited data using FOCUSS: A re-weighted minimum norm algorithm," IEEE Trans. Signal Process., vol.45, no.3, pp. 600-616, Mar. 1997.
4. E. Candès, J. Romberg, and T. Tao, "Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information," IEEE Trans. Inf. Theory, vol.52, no.2, pp. 489-509, Feb. 2006.
5. D. Donoho, "Compressed sensing," IEEE Trans. Inf. Theory, vol.52, no.4, pp. 1289-1306, Apr. 2006.
6. R. G. Baraniuk, "A lecture on compressive sensing," IEEE Signal Process. Mag., vol.24, no.4, pp. 118-121, 2007.
7. E. Candès, M. Rudelson, T. Tao, and R. Vershynin, "Error correction via linear programming," Found. Comput. Math., pp. 295-308, 2005.
8. S. Chen, D. Donoho, and M. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Sci. Comput., vol.20, no.1, pp. 33-61, 1998.
9. D. Donoho and J. Tanner, "Neighborliness of randomly projected simplices in high dimensions," Proc. Nat. Acad. Sci., vol.102, no.27, pp. 9452-9457, 2005.
10. D. Donoho, "High-dimensional centrally symmetric polytopes with neighborliness proportional to dimension," Discrete Comput. Geomet., vol.35, no.4, pp. 617-652, Mar. 2006.
11. P. M. Vaidya, "An algorithm for linear programming which requires O(((m+n)n2+(m+n)1.5n)L) arithmetic operations," in Proc. 19th ACM Symp. Theory Computing (STOC), New York, 1987, pp. 29-38.
12. J. A. Tropp and A. C. Gilbert, "Signal recovery from random measurements via orthogonal matching pursuit," IEEE Trans. Inf. Theory, vol.53, no.12, pp. 4655-4666, Dec. 2007.
13. A. Cohen, W. Dahmen, and R. A. DeVore, "Near optimal approximation of arbitrary vectors from highly incomplete measurements," Institut für Geometrie und Praktische Mathematik, Aachen, Germany, 2007, Preprint.
14. M. F. Duarte, M. B. Wakin, and R. G. Baraniuk, "Fast reconstruction of piecewise smooth signals from random projections," presented at the SPARS, Rennes, France, Nov. 2005.
15. D. L. Donoho, Y. Tsaig, I. Drori, and J.-C. Starck, "Sparse solution of underdetermined linear equations by stagewise orthogonal matching pursuit," Stanford Statistics Dept., Stanford Univ., Stanford, CA, TR-2006-2012, Mar. 2006, Preprint.
16. D. Needell and J. A. Tropp, "CoSaMP: Iterative signal recovery from incomplete and inaccurate samples," Appl. Comput. Harmon. Anal., vol.26, no.3, pp. 301-321, 2008.
17. T. Blumensath and M. E. Davies, "Iterative hard thresholding for compressed sensing," Appl. Comput. Harmon. Anal., 2008, to be published.
18. W. Dai and O. Milenkovic, "Subspace pursuit for compressive sensing: Closing the gap between performance and complexity," IEEE Trans. Inf. Theory, vol.55, no.5, pp. 2230-2249, May 2009.
19. 1-minimization: Methodology and convergence," SIAM J. Optimiz., vol.19, 2008.
20. M. Figueiredo, R. Nowak, and S. J. Wright, "Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems," IEEE J. Sel. Topics Signal Process., no.4, pp. 586-597, Dec. 2007.
21. E. van den Berg and M. P. Friedlander, "Probing the Pareto frontier for basis pursuit solutions," Comput. Sci. Dept., Univ. British Columbia, Vancouver, BC, Canada, Tech. Rep. TR-2008-2101, Jan. 2008.
22. G. Cormode and S. Muthukrishnan, "Towards an algorithmic theory of compressed sensing," Center for Discrete Math. and Comp. Sci. (DIMACS), Tech. Rep. TR 2005-2025, 2005.
23. G. Cormode and S. Muthukrishnan, "Combinatorial algorithms for compressed sensing," Lecture Notes in Computer Science, vol.4056, pp. 280-294, 2006.
24. 1 norm for sparse vectors," Apr. 2006, Preprint.
25. S. Ji, Y. Xue, and L. Carin, "Bayesian compressive sensing," IEEE Trans. Signal Process., vol.56, no.6, pp. 2346-2356, Jun. 2008.
26. M. W. Seeger and H. Nickisch, "Compressed sensing and Bayesian experimental design," in Proc. 25th Int. Conf. Machine Learning (ICML), 2008, pp. 912-919.
27. P. Schniter, L. C. Potter, and J. Ziniel, "Fast Bayesian matching pursuit: Model uncertainty and parameter estimation for sparse linear models," Mar. 2009.
28. T. Hastie, R. Tibshirani, and J. H. Friedman, The Elements of Statistical Learning. New York: Springer, Aug. 2001.
29. G. Guo and C.-C. Wang, "Multiuser detection of sparsely spread CDMA," IEEE J. Sel. Areas Commun., vol.26, no.3, pp. 421-431, 2008.
30. J. Pearl, Probablistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Mateo, CA: Morgan-Kaufmann, 1988.
31. F. V. Jensen, An Introduction to Bayesian Networks. New York: Springer-Verlag, 1996.
32. B. J. Frey, Graphical Models for Machine Learning and Digital Communication. Cambridge, MA: MIT Press, 1998.
33. J. S. Yedidia, W. T. Freeman, and Y. Weiss, "Understanding belief propagation and its generalizations," Mitsubishi Electric Res. Labs., Tech. Rep. TR2001-022, Jan. 2002.
34. D. J. C. MacKay, Information Theory, Inference and Learning Algorithms. Cambridge, U.K.: Cambridge Univ. Press, 2002.
35. R. G. Cowell, A. P. Dawid, S. L. Lauritzen, and D. J. Spiegelhalter, Probabilistic Networks and Expert Systems. New York: Springer-Verlag, 2003.
36. N. Sommer, M. Feder, and O. Shalvi, "Low-density lattice codes," IEEE Trans. Inf. Theory, vol.54, no.4, pp. 1561-1585, 2008.
37. R. G. Gallager, "Low-density parity-check codes," IEEE Trans. Inf. Theory, vol.8, no.1, pp. 21-28, Jan. 1962.
38. T. J. Richardson, M. A. Shokrollahi, and R. L. Urbanke, "Design of capacity-approaching irregular low-density parity-check codes," IEEE Trans. Inf. Theory, vol.47, pp. 619-637, Feb. 2001.
39. S. Sarvotham, D. Baron, and R. G. Baraniuk, "Sudocodes-Fast measurement and reconstruction of sparse signals," presented at the Int. Symp. Inf. Theory (ISIT), Seattle, WA, Jul. 2006.
40. R. Berinde and P. Indyk, "Sparse recovery using sparse random matrices," MIT, Cambridge, MA, Tech. Rep. MIT-CSAIL-TR-2008-3001, 2008.
41. J.-C. Pesquet, H. Krim, and E. Hamman, "Bayesian approach to best basis selection," in Proc. IEEE 1996 Int. Conf. Acoustics, Speech, Signal Process. (ICASSP), 1996, pp. 2634-2637.
42. H. Chipman, E. Kolaczyk, and R. McCulloch, "Adaptive Bayesian wavelet shrinkage," J. Amer. Statist. Assoc., vol.92, pp. 1413-1421, 1997.
43. M. S. Crouse, R. D. Nowak, and R. G. Baraniuk, "Wavelet-based signal processing using hidden Markov models," IEEE Trans. Signal Process., vol.46, no.4, pp. 886-902, Apr. 1998.
44. E. I. George and R. E. McCulloch, "Variable selection via Gibbs sampling," J. Amer. Statist. Assoc., vol.88, pp. 881-889, 1993.
45. J. Geweke, "Variable selection and model comparison in regression," in Bayesian Statist. 5, 1996, pp. 609-620.
46. L. He and L. Carin, "Exploiting structure in wavelet-based Bayesian compressive sensing," IEEE Trans. Signal Process., vol.57, no.9, pp. 3488-3497, Sep. 2008.
47. H. W. Sorenson and D. L. Alspach, "Recursive Bayesian estimation using Gaussian sums," Automatica, vol.7, pp. 465-479, 1971.
48. J. A. Tropp, "Greed is good: Algorithmic results for sparse approximation," IEEE Trans. Inf. Theory, vol.50, no.10, pp. 2231-2242, Oct. 2004.
49. W. Wang, M. Garofalakis, and K. Ramchandran, "Distributed sparse random projections for refinable approximation," in Proc. Inf. Process. Sensor Networks (IPSN), 2007, pp. 331-339.
50. G. Cooper, "The computational complexity of probabilistic inference using Bayesian belief networks," Artif. Intell., vol.42, pp. 393-405, 1990.
51. F. R. Kschischang, B. J. Frey, and H.-A. Loeliger, "Factor graphs and the sum-product algorithm," IEEE Trans. Inf. Theory, vol.47, no.2, pp. 498-519, Feb. 2001.
52. E. Sudderth, A. Ihler,W. Freeman, and A. S.Willsky, "Nonparametric belief propagation," MIT, Cambridge, MA, MIT LIDS Tech. Rep. 2551, Oct. 2002.
53. B. J. Frey and D. J. C. MacKay, "A revolution: Belief propagation in graphs with cycles," in Adv. Neural Inf. Process. Syst., M. Jordan, M. S. Kearns, and S. A. Solla, Eds., 1998, vol. 10.
54. A. Ihler, J. Fisher, and A. S.Willsky, "Loopy belief propagation: Convergence and effects of message errors," J. Mach. Learn. Res., vol.6, pp. 905-936, May 2005.
55. M. Pretti, "A message-passing algorithm with damping," J. Stat. Mech., Nov. 2005.
56. D. J. C. MacKay, "Good error-correcting codes based on very sparse matrices," IEEE Trans. Inf. Theory, vol.45, no.11, pp. 399-431, Mar. 2005.
57. D. W. Scott and W. F. Szewczyk, "From kernels to mixtures," Technometrics, vol.43, pp. 323-335, Aug. 2001.
58. R. G. Baraniuk, V. Cevher,M. F. Duarte, and C. Hegde, "Model-based compressive sensing," 2008 [Online]. Available: http://arxiv.org/abs/ 0808.3572, Preprint
59. J. W. Byers, M. Luby, and M. Mitzenmacher, "A digital fountain approach to asynchronous reliable multicast," IEEE J. Sel. Areas Commun., vol.20, no.8, pp. 1528-1540, Oct. 2002.
60. D. Guo, D. Baron, and S. Shamai, "A single-letter characterization of optimal noisy compressed sensing," in Proc. 47th Allerton Conf. Commun., Control, and Comput., Sep. 2009.