A coding theory approach to noisy compressive sensing using low density frames

IEEE Transactions on Signal Processing

Volumn 59, Issue 11, 2011, Pages 5369-5379

  Akçakaya, Mehmet ^a   Park, Jinsoo ^b   Tarokh, Vahid ^b  

^a HARVARD MEDICAL SCHOOL   (United States)

^b HARVARD UNIVERSITY   (United States)

Author keywords

Belief propagation; coding theory; compressive sensing; EM algorithm; Gaussian scale mixtures; low density frames; sum product algorithm

Indexed keywords

BELIEF PROPAGATION; CODING THEORY; COMPRESSIVE SENSING; EM ALGORITHMS; GAUSSIAN SCALE MIXTURES; LOW DENSITY; SUM-PRODUCT ALGORITHM;

ALGORITHMS; DECODING; GAUSSIAN DISTRIBUTION; GAUSSIAN NOISE (ELECTRONIC); INFORMATION THEORY; SIGNAL RECONSTRUCTION;

COMPUTER PROGRAMMING;

EID: 80054073830     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2011.2163402     Document Type: Article

References (56)

1. M. Akçakaya and V. Tarokh, "Aframe construction and a universal distortion bound for sparse representations," IEEE Trans. Signal Process., vol. 56, no. 6, pp. 2443-2550, Jun. 2008.
2. M. Akçakaya and V. Tarokh, "On sparsity, redundancy and quality of frame representations," presented at the IEEE Int. Symp. Inf. Theory (ISIT), Nice, France, Jun. 2007.
3. M. Akçakaya and V. Tarokh, "Shannon theoretic limits on noisy compressive sampling," IEEE Trans. Inf. Theory, vol. 56, no. 1, pp. 492-504, Jan. 2010.
4. D. Andrews and C. Mallows, "Scale mixtures of normal distributions," J. R. Stat. Soc., vol. 36, pp. 99-102, 1974.
5. B. Babadi, N. Kalouptsidis, and V. Tarokh, "SPARLS: A low complexity recursive L1-regularized least squares algorithm," IEEE Trans. Signal Process., vol. 58, pp. 4013-4025, Aug. 2010.
6. R. Baraniuk, V. Cevher,M. Duarte, and C. Hegde, "Model-based compressive sensing," IEEE Trans. Inf. Theory, vol. 56, pp. 1982-2001, Apr. 2010.
7. D. Baron, S. Sarvotham, and R. Baraniuk, "Bayesian compressive sensing via belief propagation," IEEE Trans. Signal Process., vol. 58, pp. 269-280, Jan. 2010.
8. R. Berinde, A. C. Gilbert, P. Indyk, H. Karloff, and M. J. Strauss, "Combining geometry and combinatorics: A unified approach to sparse signal recovery," presented at the Allerton Conf. Commun., Control, Comput., Monticello, IL, Sep. 2008.
9. R. Berinde, P. Indyk, and M. Ruzic, "Practical near-optimal sparse recovery in the ell-1 norm," presented at the Allerton Conf. Comm., Control, Comput., Monticello, IL, Sep. 2008.
10. R. Berinde and P. Indyk, "Sequential sparse matching pursuit," presented at the Allerton Conf. Commun., Control, Comput., Monticello, IL, Sep. 2009.
11. D. Bickson, A. T. Ihler, and D. Dolev, "A low density lattice decoder via non-parametric belief propagation," presented at the Allerton Conf. Commun., Control, Comput., Monticello, IL, Sep. 2009.
12. J. M. Bioucas-Dias, "Bayesian wavelet-based image deconvolution: A GEM algorithm exploiting a class of heavy-tailed priors," IEEE Trans. Image Process., vol. 15, pp. 937-951, Apr. 2006.
13. C. M. Bishop, Pattern Recognition and Machine Learning, 1st ed. New York: Springer, 2006.
14. E. J. Candès, J. Romberg, and T. Tao, "Stable signal recovery for incomplete and inaccurate measurements," Commun. Pure Appl. Math., vol. 59, pp. 1207-1223, Aug. 2006.
15. E. J. Candès and T. Tao, "Decoding by linear programming," IEEE Trans. Inf. Theory, vol. 51, pp. 4203-4215, Dec. 2005.
16. E. J. Candès, M. Wakin, and S. Boyd, "Enhancing sparsity by reweighted l1 minimization," J. Fourier Anal. Appl., vol. 14, pp. 877-905.
17. R. Chartrand and W. Yin, "Iteratively reweighted algorithms for compressive sensing," presented at the IEEE Int. Conf. Acoust., Speech, Signal Process. (IEEE ICASSP), Las Vegas, NV, Apr. 2008.
18. T. Cormen, C. Leiserson, L. Rivest, and C. Stein, Introduction to Algorithms, 2nd ed. Cambridge, MA: MIT Press, 2001.
19. W. Dai and O. Milenkovic, "Subspace pursuit for compressive sensing: Closing the gap between performance and complexity," IEEE Trans. Inf. Theory, vol. 55, pp. 2230-2249, May 2009.
20. D. L. Donoho, "Compressed sensing," IEEE Trans. Inf. Theory, vol. 52, pp. 1289-1306, Apr. 2006.
21. D. L. Donoho, A. Maleki, and A. Montanari, "Message passing algorithms for compressed sensing," Proc. Natl Acad. Sci., vol. 106, pp. 18914-18919, Nov. 2009.
22. M. Figueiredo and R. Nowak, "An EM algorithm for wavelet-based image restoration," IEEE Trans. Image Process., vol. 12, pp. 906-916, Aug. 2003.
23. M. A. T. Figueiredo and R. Nowak, "Wavelet-based image estimation: An empirical bayes approach using Jeffreys' noninformative prior," IEEE Trans. Image Process., vol. 10, pp. 1322-1331, Sep. 2001.
24. M. A. T. Figueiredo, R. D. Nowak, and S. J. Wright, "Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems," IEEE J. Sel. Topics Signal Process., vol. 1, pp. 586-598, Dec. 2007.
25. A. Gilbert and P. Indyk, "Sparse recovery using sparsematrices, " Proc. IEEE, vol. 98, pp. 937-947, Jun. 2010.
26. X. -Y. Hu, E. Eleftheriou, and D. M. Arnold, "Regular and irregular progressive edge-growth tanner graphs," IEEE Trans. Inf. Theory, vol. 51, pp. 386-398, Jan. 2005.
27. P. Indyk and M. Ruzic, "Near-optimal sparse recovery in the norm," presented at the IEEE Symp. Found. Comput. Sci., Philadelphia, PA, Oct. 2008.
28. S. Ji, Y. Xue, and L. Carin, "Bayesian compressive sensing," IEEE Trans. Signal Process., vol. 56, pp. 2346-2356, Jun. 2008.
29. F. R. Kschischang, B. J. Frey, and H. -A. Loeliger, "Factor graphs and the sum-product algorithm," IEEE Trans. Inf. Theory, vol. 47, pp. 498-519, Feb. 2001.
30. D. J. C. MacKay, "Good error correcting codes based on very sparse matrices," IEEE Trans. Inf. Theory, vol. 45, pp. 399-431, Mar. 1999.
31. D. J. C. MacKay, Information Theory, Inference and Learning Algorithms, 3rd ed. Cambridge, U. K: Cambridge Univ. Press, 2002.
32. Y. Mao and A. H. Banihashemi, "Decoding low-density parity-check codes with probabilistic scheduling," IEEE Commun. Lett., vol. 5, pp. 414-416, Oct. 2001.
33. A. Montanari, Graphical Models Concepts in Compressed Sensing, preprint [Online]. Available: http://arxiv. org/abs/1011. 4328, preprint.
34. T. K. Moon, "The EM algorithm in signal processing," IEEE Signal Process. Mag., vol. 13, pp. 47-60, Nov. 1996.
35. D. Needell and J. A. Tropp, "CoSaMP: Iterative signal recovery from incomplete and inaccurate samples," Appl. Comp. Harmon. Anal., vol. 26, pp. 301-321, 2008.
36. C. C. Paige and M. A. Saunders, "LSQR: Sparse linear equations and least squares problems," ACM Trans. Math. Softw. (TOMS), vol. 8, pp. 195-209, Jun. 1982.
37. J. Portilla, V. Strela, M. J. Wainwright, and E. P. Simoncelli, "Image denoising using scale mixtures of Gaussians in the wavelet domain," IEEE Trans. Image Process., vol. 12, pp. 1338-1351, Nov. 2003.
38. R. Rubinstein, M. Zibulevsky, and M. Elad, "Double sparsity: Learning sparse dictionaries for sparse signal approximation," IEEE Trans. Signal Process., vol. 58, pp. 1553-1564, Mar. 2010.
39. T. J. Richardson and R. L. Urbanke, "The capacity of low-density parity-check codes under message passing decoding," IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 599-618, Feb. 2001.
40. C. Robert, The Bayesian Choice: A Decision Theoretic Motivation, 1st ed. New York, NY: Springer-Verlag, 1994.
41. S. Sarvotham, D. Baron, and R. Baraniuk, "Sudocodes-fast measurement and reconstruction of sparse signals," presented at the IEEE Int. Symp. Inf. Theory (ISIT), Seattle, WA, Jul. 2006.
42. S. Sarvotham, D. Baron, and R. Baraniuk, "Compressed sensing reconstruction via belief propagation," Electrical and Comput. Eng. Dept.,. Rice Univ., Houston, TX, 2006.
43. A. Shokrollahi, "Raptor codes," IEEE Trans. Inf. Theory, vol. 52, pp. 2551-2567, Jun. 2006.
44. M. Sipser and D. A. Spielman, "Expander codes," IEEE Trans. Inf. Theory, vol. 42, pp. 1710-1722, Nov. 1996.
45. R. M. Tanner, "A recursive approach to low complexity codes," IEEE Trans. Inf. Theory, vol. 27, pp. 533-547, Sep. 1981.
46. M. E. Tipping, "Sparse bayesian learning and the relevance vector machine," J. Mach. Learn. Res., vol. 1, pp. 211-244, 2001.
47. M. E. Tipping, "Bayesian inference: An introduction to principles and practice in machine learning," in Advanced Lectures on Machine Learning. New York: Springer, 2004, pp. 41-62.
48. E. van den Berg and M. P. Friedlander, "Probing the pareto frontier for basis pursuit solutions," SIAM J. Sci. Comput., vol. 31, no. 2, pp. 890-912, 2008.
49. M. J. Wainwright, "Sharp thresholds for noisy and high-dimensional recovery of sparsity using -constrained quadratic programming," IEEE Trans. Inf. Theory, vol. 55, pp. 2183-2202, May 2009.
50. Y. Weiss and W. T. Freeman, "Correctness of belief propagation in Gaussian graphical models of arbitrary topology," Neural Comput., vol. 13, no. 10, pp. 2173-2200, Oct. 2001.
51. D. Wipf and S. Nagarajan, "Iterative reweighted and methods for finding sparse solutions," IEEE J. Sel. Topics Signal Process., vol. 4, pp. 317-329, Apr. 2010.
52. H. Xiao andA. H. Banihashemi, "Graph-basedmessage-passing schedules for decoding LDPC codes," IEEE Trans. Commun., vol. 52, pp. 2098-2105, Dec. 2004.
53. W. Xu and B. Hassibi, "Efficient compressive sensing with deterministic guarantees using expander graphs," presented at the IEEE Inf. Theory Workshop, Lake Tahoe, CA, Sep. 2007.
54. F. Zhang and H. D. Pfister, "On the iterative decoding of high rate LDPC codes with applications in compressed sensing," presented at the Allerton Conf. Commun., Control, Comput., Monticello, IL, Sep. 2008.
55. F. Zhang and H. D. Pfister, "Verification decoding of high-rate LDPC codes with applications in compressed sensing," [Online]. Available: http://arxiv. org/abs/0903. 2232, preprint.
56. LDF Website [Online]. Available: http://people. fas. harvard. edu/akcakaya/ suprem. html