Exploiting statistical dependencies in sparse representations for signal recovery

IEEE Transactions on Signal Processing

Volumn 60, Issue 5, 2012, Pages 2286-2303

  Peleg, Tomer ^a   Eldar, Yonina C ^a,b   Elad, Michael ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

^b STANFORD UNIVERSITY   (United States)

Author keywords

Bayesian estimation; Boltzmann machine (BM); decomposable model; denoising; greedy pursuit; image patches; maximum a posteriori (MAP); message passing; MRF; pseudo likelihood; sequential subspace optimization (SESOP); signal synthesis; sparse representations; unitary dictionary

Indexed keywords

BAYESIAN ESTIMATIONS; BOLTZMANN MACHINES; DE-NOISING; GREEDY PURSUIT; IMAGE PATCHES; MAXIMUM A POSTERIORI; MRF; PSEUDO-LIKELIHOOD; SIGNAL SYNTHESIS; SPARSE REPRESENTATION; SUBSPACE OPTIMIZATION;

ALGORITHMS; BAYESIAN NETWORKS; IMAGE PROCESSING; MESSAGE PASSING; SIGNAL RECONSTRUCTION;

GRAPH THEORY;

EID: 84860009365     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2012.2188520     Document Type: Article

References (42)

1. A. M. Bruckstein, D. L. Donoho, and M. Elad, "From sparse solutions of systems of equations to sparse modeling of signals and images," SIAM Rev., vol. 51, no. 1, pp. 34-81, Feb. 2009.
2. J. A. Tropp and S. J. Wright, "Computational methods for sparse solution of linear inverse problems," Proc. IEEE, vol. 98, no. 6, pp. 948-958, Jun. 2010.
3. C. La and M. N. Do, "Tree-based orthogonal matching pursuit algorithm for signal reconstruction," in Proc. ICIP, Atlanta, GA, Oct. 2006.
4. Y.M. Lu and M. N. Do, "Sampling signals from a union of subspaces," IEEE Signal Process. Mag., vol. 25, no. 2, pp. 41-47, 2008.
5. Y. C. Eldar and M. Mishali, "Robust recovery of signals from a structured union of subspaces," IEEE Trans. Inf. Theory, vol. 55, no. 11, pp. 5302-5316, Nov. 2009.
6. Y. C. Eldar, P. Kuppinger, and H. Bolcskei, "Block-sparse signals: Uncertainty relations and efficient recovery," IEEE Trans. Signal Process., vol. 58, no. 6, pp. 3042-3054, Jun. 2010.
7. R. G. Baraniuk, V. Cevher,M. F. Duarte, and C. Hedge, "Model-based compressive sensing," IEEE Trans. Inf. Theory, vol. 56, no. 4, pp. 1982-2001, Apr. 2010.
8. M. Duarte and Y. C. Eldar, "Structured compressed sensing: From theory to applications," IEEE Trans. Signal Process., vol. 59, no. 9, pp. 4053-4085, Sep. 2011.
9. S. G. Mallat, A Wavelet Tour of Signal Processing. NewYork:Academic Press, 1998.
10. M. Mishali and Y. C. Eldar, "Reduce and boost: Recovering arbitrary sets of jointly sparse vectors," IEEE Trans. Signal Process., vol. 56, no. 10, pp. 4692-4702, Oct. 2008.
11. M. Mishali and Y. C. Eldar, "Blind multi-band signal reconstruction: Compressed sensing for analog signals," IEEE Trans. Signal Process., vol. 57, no. 3, pp. 993-1009, Mar. 2009.
12. M. S. Crouse, R. D. Nowak, and R. G. Baraniuk, "Wavelet-based statistical signal processing using hidden Markov models," IEEE Trans. Signal Process., vol. 46, no. 4, pp. 886-902, Apr. 1998. (Pubitemid 128743828)
13. M. W. Seeger, "Bayesian inference and optimal design for the sparse linear model," J. Mach. Learn. Res., vol. 9, pp. 759-813, Apr. 2008.
14. P. Schniter, L. C. Potter, and J. Ziniel, "Fast Bayesian matching pursuit," in Proc. ITA, La Jolla, CA, Jan. 2008.
15. J. S. Turek, I. Yavneh, and M. Elad, "On MMSE and MAP denoising under sparse representation modeling over a unitary dictionary," IEEE Trans. Signal Process., vol. 59, no. 8, pp. 3526-3535, Aug. 2011.
16. M. E. Tipping, "Sparse Bayesian learning and the relevance vector machine," J. Mach. Learn. Res., vol. 1, no. 4, pp. 211-244, 2001. (Pubitemid 33687203)
17. R. M. Neal, Probabilistic Inference Using Markov Chain Monte Carlo Methods CS Dept., Univ. Toronto, Toronto, ON, Canada, 2003, Tech. Rep..
18. S. Ji, Y. Xue, and L. Carin, "Bayesian compressive sensing," IEEE Trans. Signal Process., vol. 56, no. 6, pp. 2346-2356, Jun. 2008. (Pubitemid 351795888)
19. M. Elad and I. Yavneh, "A plurality of sparse representations is better than the sparsest one alone," IEEE Trans. Inf. Theory, vol. 55, no. 10, pp. 4701-4714, Oct. 2009.
20. T. Michaeli, D. Sigalov, and Y. C. Eldar, "Partially linear estimation with application to sparse signal recovery from measurement pairs," IEEE Trans. Signal Process., to be published.
21. M. F. Duarte,M. B.Wakin, and R. G. Baraniuk, "Wavelet-domain compressive signal reconstruction using a hidden Markov tree model," in Proc. ICASSP, Las Vegas, NV, Apr. 2008.
22. L. He and L. Carin, "Exploiting structure in wavelet-based Bayesian compressive sensing," IEEE Trans. Signal Process., vol. 57, no. 9, pp. 3488-3497, Sep. 2009.
23. P. J. Wolfe, S. J. Godsill, and W. J. Ng, "Bayesian variable selection and regularization for time-frequency surface estimation," J. R. Statist. Soc. B, vol. 66, no. 3, pp. 575-589, June 2004. (Pubitemid 39015902)
24. P. J. Garrigues and B. A. Olshausen, "Learning horizontal connections in a sparse coding model of natural images," in Proc. Adv. Neural Inform. Process. Syst. 20, J. C. Platt, D. Koller, Y. Singer, and S. Roweis, Eds., 2008, pp. 505-512.
25. V. Cevher, M. F. Duarte, C. Hedge, and R. G. Baraniuk, "Sparse signal recovery using Markov random fields," in Proc. Adv. Neural Inform. Process. Syst. 21, D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, Eds., 2009, pp. 257-264.
26. M. I. Jordan, "Graphical models," Statist. Sci., vol. 19, no. 1, pp. 140-155, 2004. (Pubitemid 39056883)
27. A. Wiesel, Y. C. Eldar, and A. O. Hero, III, "Covariance estimation in decomposable Gaussian graphical models," IEEE Trans. Signal Process., vol. 58, no. 3, pp. 1482-1492, Mar. 2010.
28. S. L. Lauritzen, Graphical Models. London, U.K.: Oxford Univ. Press, 1996.
29. Handbook of Combinatorial Optimization, D. Z. Du and P. M. Pardalos, Eds. New York: Kluwer Academic Publishers, 1998, vol. 3, pp. 1-19.
30. S. Verdu, Multiuser Detection. Cambridge, U.K.: Cambridge Univ. Press, 1998.
31. V. Kolmogorov and R. Zabih, "What energy functions can be minimized via graph cuts?," IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 2, pp. 147-159, Feb. 2004.
32. D. Baron, S. Sarvotham, and R. G. Baraniuk, "Bayesian compressive sensing via belief propagation," IEEE Trans. Signal Process., vol. 58, no. 1, pp. 269-280, Jan. 2010.
33. D. L. Donoho, A. Maleki, and A. Montanari, "Message passing algorithms for compressed sensing: I. Motivation and construction," in Proc. Inf. Theory Workshop, Cairo, Egypt, Jan. 2010.
34. P. Schniter, "Turbo reconstruction of structured sparse signals," in Proc. Conf. Inform. Sci. Syst., Princeton, NJ, Mar. 2010.
35. M.Aharon, M. Elad, and A.M. Bruckstein, "K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation," IEEE Trans. Signal Process., vol. 54, no. 11, pp. 4311-4322, Nov. 2006. (Pubitemid 44637761)
36. J. Besag, "Statistical analysis of non-lattice data," The Statistican, vol. 24, pp. 179-195, 1975.
37. A. Hyvarinen, "Consistency of pseudolikelihood estimation of fully visible Boltzmann machines," Neural Comput., vol. 18, no. 10, pp. 2283-2292, Oct. 2006. (Pubitemid 44511765)
38. S. P. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
39. G. Narkiss and M. Zibulevsky, Sequential Subspace Optimization Method for Large-Scale Unconstrained Optimization Electr. Eng. Dept., Technion-Israel Inst. of Technology, Haifa, Israel, 2005, Tech. Rep..
40. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, "Non-local sparse models for image restoration," in Proc. ICCV, Tokyo, 2009.
41. M. Elad and M. Aharon, "Image denoising via sparse and redundant representations over learned dictionaries," IEEE Trans. Image Process., vol. 15, no. 12, pp. 3736-3745, Dec. 2006. (Pubitemid 44811686)
42. T. Faktor, Y. C. Eldar, and M. Elad, "Denoising of image patches via sparse representations with learned statistical dependencies," in Proc. ICASSP, Prague, Czech Republic, May 2011, pp. 5820-5823.