Reconstruction of signals drawn from a gaussian mixture via noisy compressive measurements

IEEE Transactions on Signal Processing

Volumn 62, Issue 9, 2014, Pages 2265-2277

  Renna, Francesco ^a   Calderbank, Robert ^b   Carin, Lawrence ^b   Rodrigues, Miguel R D ^c  

^a INSTITUTO DE TELECOMUNICAÇÕES   (Portugal)

^b Duke University   (United States)

^c UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

classification; Compressive sensing; Gaussian mixtures; kernel design; MMSE; MMSE decay; MMSE power offset; phase transition; reconstruction

Indexed keywords

CLASSIFICATION (OF INFORMATION); IMAGE RECONSTRUCTION; PHASE TRANSITIONS; SIGNAL PROCESSING;

COMPRESSIVE SENSING; GAUSSIAN MIXTURES; KERNEL DESIGN; MMSE; MMSE DECAY; POWER OFFSET;

ELECTRONICS ENGINEERING;

EID: 84907977633     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2014.2309560     Document Type: Article

References (58)

1. D. Healy and D. Brady, "Compression at the physical interface," IEEE Signal Process. Mag., vol. 25, no. 2, pp. 67-71, 2008.
2. M. Lustig, D. Donoho, J. Santos, and J. Pauly, "Compressed sensing MRI," IEEE Signal Process. Mag., vol. 25, no. 2, pp. 72-82, 2008.
3. M. Duarte, M. Davenport, D. Takhar, J. Laska, T. Sun, K. Kelly, and R. Baraniuk, "Single-pixel imaging via compressive sampling," IEEE Signal Process. Mag., vol. 25, no. 2, pp. 83-91, 2008.
4. Analog Devices Corp., A/D converters, 2009 [Online]. Available: http://www.analog.com/en/analog-to-digital-converters/adconverters/products/ index.html
5. Texas Instruments Corp., Data converters, 2009 [Online]. Available: http://focus.ti.com/analog/docs/dataconvertershome.tsp
6. E. Candès, J. K. Romberg, and T. Tao, "Stable signal recovery for incomplete and inaccurate measurements," Comm. Pure Appl. Math., vol. 59, no. 8, pp. 1207-1223, 2006.
7. E. Candès and T. Tao, "Near-optimal signal recovery fromrandom projections: Universal encoding strategies?," IEEE Trans. Inf. Theory, vol. 52, no. 12, pp. 5406-5425, 2006.
8. D. Donoho, "Compressed sensing," IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289-1306, 2006.
9. E. Candès and T. Tao, "Decoding by linear programming," IEEE Trans. Inf. Theory, vol. 51, no. 12, pp. 4203-4215, 2005.
10. R. G. Baraniuk, M. Davenport, R. DeVore, and M. Wakin, "A simple proof of the restricted isometry property for random matrices," Constr. Approx., vol. 28, pp. 253-263, Dec. 2008.
11. A. Ashok, P. Baheti, and M. Neifeld, "Compressive imaging system design using task-specific information," Appl. Opt., vol. 47, no. 25, pp. 4457-4471, 2008.
12. P. Baheti and M. Neifeld, "Recognition using information-optimal adaptive feature-specific imaging," J. Opt. Soc. Amer. A, vol. 26, no. 4, pp. 1055-1070, 2009.
13. R. Venkataramani and Y. Bresler, "Further results on spectrum blind sampling of 2d signals," in Proc. IEEE Int. Conf. Image Process. (ICIP), 1998, vol. 2, pp. 752-756.
14. D. Baron, M. B. Wakin, M. F. Duarte, S. Sarvotham, and R. G. Baraniuk, "Distributed compressed sensing," 2005 [Online]. Available: http://dsp.rice.edu/cs/
15. E. Candès, J. Romberg, and T. Tao, "Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information," IEEE Trans. Inf. Theory, vol. 52, no. 2, pp. 489-509, 2006.
16. S.Mallat and Z. Zhang, "Matching pursuitswith time-frequency dictionaries," IEEE Trans. Signal Process., vol. 41, no. 12, pp. 3397-3415, 1993.
17. S. S. Chen,D. L.Donoho, andM.A. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Sci. Comput., vol. 20, no. 1, pp. 33-61, 1998.
18. J. A. Tropp and S. J. Wright, "Computational methods for sparse solution of linear inverse problems," Proc. IEEE, vol. 98, no. 6, pp. 948-958, 2010.
19. T. Blumensath andM.Davies, "Sampling theorems for signals from the union of finite-dimensional linear subspaces," IEEE Trans. Inf. Theory, vol. 55, no. 4, pp. 1872-1882, 2009.
20. M. Stojnic, F. Parvaresh, and B. Hassibi, "On the reconstruction of block-sparse signals with an optimal number of measurements," IEEE Trans. Signal Process., vol. 57, no. 8, pp. 3075-3085, 2009.
21. Y. C. Eldar and M. Mishali, "Robust recovery of signals from a structured union of subspaces," IEEE Trans. Inf. Theory, vol. 55, no. 11, pp. 5302-5316, 2009.
22. Y. Eldar, P. Kuppinger, and H. Bölcskei, "Block-sparse signals: Uncertainty relations and efficient recovery," IEEE Trans. Signal Process., vol. 58, no. 6, pp. 3042-3054, 2010.
23. R. G. Baraniuk, V. Cevher,M. F. Duarte, and C. Hedge, "Model-based compressive sensing," IEEE Trans. Inf. Theory, vol. 56, no. 4, pp. 1982-2001, 2010.
24. R.G. Baraniuk andM. B.Wakin, "Randomprojections of smooth manifolds," Found. Comput. Math., vol. 9, no. 1, pp. 51-77, 2009.
25. M. Chen, J. Silva, J. Paisley, D. Dunson, and L. Carin, "Compressive sensing on manifolds using a nonparametric mixture of factor analyzers: Algorithm and performance bounds," IEEE Trans. Signal Process., vol. 58, no. 12, pp. 6140-6155, 2010.
26. M. Crouse, R. Nowak, and R. Baraniuk, "Wavelet-based statistical signal processing using hidden markov models," IEEE Trans. Signal Process., vol. 46, no. 4, pp. 886-902, 1998.
27. D. Needell and J. A. Tropp, "CoSaMP: Iterative signal recovery from incomplete and inaccurate samples," Appl. Computat. Harmon. Anal., vol. 26, no. 3, pp. 301-321, 2009.
28. G. Yu and G. Sapiro, "Statistical compressed sensing of Gaussian mixture models," IEEE Trans. Signal Process., vol. 59, no. 12, pp. 5842-5858, 2011.
29. G. Yu, G. Sapiro, and S.Mallat, "Solving inverse problems with piecewise linear estimators: From Gaussian mixture models to structured sparsity," IEEE Trans. Image Process., vol. 21, no. 5, pp. 2481-2499, 2012.
30. J. Duarte-Carvajalino, G. Yu, L. Carin, and G. Sapiro, "Task-driven adaptive statistical compressive sensing of Gaussian mixture models," IEEE Trans. Signal Process., vol. 61, no. 3, pp. 585-600, 2013.
31. S. Ji, Y. Xue, and L. Carin, "Bayesian compressive sensing," IEEE Trans. Signal Process., vol. 56, no. 6, pp. 2346-2356, 2008.
32. H. W. Sorenson and D. L. Alspach, "Recursive Bayesian estimation using Gaussian sums," Automatica, vol. 7, no. 4, pp. 465-479, 1971.
33. W. R. Carson, M. Chen, M. R. D. Rodrigues, R. Calderbank, and L. Carin, "Communications-inspired projection design with applications to compressive sensing," SIAM J. Imag. Sci., vol. 5, no. 4, pp. 1185-1212, 2012.
34. M. Chen,W. Carson,M. R. D. Rodrigues, R. Calderbank, and L. Carin, "Communications inspired linear discriminant analysis," in Proc. Int. Conf. Machine Learn. (ICML), Jun.-Jul. 2012, pp. 919-926.
35. E. Candès and T. Tao, "The Dantzig selector: Statistical estimation when is much larger than ," Ann. Statist., vol. 35, no. 6, pp. 2313-2351, 2007.
36. D. Donoho and J. Tanner, "Observed universality of phase transitions in high-dimensional geometry, with implications for modern data analysis and signal processing," Philos. Trans. Roy. Soc. A, vol. 367, no. 1906, pp. 4273-4293, 2009.
37. D. Donoho, A.Maleki, and A.Montanari, "The noise-sensitivity phase transition in compressed sensing," IEEE Trans. Inf. Theory, vol. 57, no. 10, pp. 6920-6941, 2011.
38. Y. Wu and S. Verdú, "Optimal phase transitions in compressed sensing," IEEE Trans. Inf. Theory, vol. 58, no. 10, pp. 6241-6263, 2012.
39. G. Reeves and M. Gastpar, "The sampling rate-distortion tradeoff for sparsity pattern recovery in compressed sensing," IEEE Trans. Inf. Theory, vol. 58, no. 5, pp. 3065-3092, 2012.
40. A. Tulino, G. Caire, S. Shamai, and S. Verdú, "Support recovery with sparsely sampled free random matrices," in Proc. IEEE Int. Symp. Inf. Theory (ISIT), Jul.-Aug. 2011, pp. 2239-2243.
41. D. Baron, S. Sarvotham, and R. Baraniuk, "Bayesian compressive sensing via belief propagation," IEEE Trans. Signal Process., vol. 58, no. 1, pp. 269-280, 2010.
42. D. Guo and C.-C. Wang, "Asymptotic mean-square optimality of belief propagation for sparse linear systems," in Proc. IEEE Inf. Theory Workshop, Oct. 2006, pp. 194-198.
43. D. L. Donoho, A. Maleki, and A. Montanari, "Message-passing algorithms for compressed sensing," Proc. Nat. Acad. Sci., vol. 106, no. 45, pp. 18 914-18 919, Nov. 2009.
44. M. Bayati and A. Montanari, "The dynamics of message passing on dense graphs, with applications to compressed sensing," IEEE Trans. Inf. Theory, vol. 57, no. 2, pp. 764-785, 2011.
45. S. Rangan, "Generalized approximate message passing for estimation with random linear mixing," in Proc. IEEE Int. Symp. Inf. Theory (ISIT), Jul.-Aug. 2011, pp. 2168-2172.
46. D. Guo, D. Baron, and S. Shamai, "A single-letter characterization of optimal noisy compressed sensing," in Proc. 47th Annu. Allerton Conf. Commun., Control, Comput., Sep. 2009, pp. 52-59.
47. G. Reeves and M. Gastpar, "Compressed sensing phase transitions: Rigorous bounds versus replica predictions," in Proc. Conf. Inf. Sci. Syst. (CISS), Mar. 2012, pp. 1-6.
48. S. Rangan, A. Fletcher, and V. Goyal, "Asymptotic analysis of MAP estimation via the replica method and applications to compressed sensing," IEEE Trans. Inf. Theory, vol. 58, no. 3, pp. 1902-1923, 2012.
49. T. Kailath, A. H. Sayed, and B. Hassibi, Linear Estimation. Upper Saddle River, NJ, USA: Prentice-Hall, 2000.
50. Y. Wu and S. Verdú, "MMSE dimension," IEEE Trans. Inf. Theory, vol. 57, no. 8, pp. 4857-4879, 2011.
51. H. Reboredo, F. Renna, R. Calderbank, andM. R. D. Rodrigues, "Compressive classification," in Proc. IEEE Int. Symp. Inf. Theory (ISIT), Jul. 2013, pp. 674-678.
52. J. Duarte-Carvajalino, G. Yu, L. Carin, and G. Sapiro, "Adapted statistical compressive sensing: Learning to sense Gaussian mixture models," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Mar. 2012, pp. 3653-3656.
53. D. P. Palomar, J. M. Cioffi, and M. A. Lagunas, "Joint Tx-Rx beamforming design for multicarrier MIMO channels: A unified framework for convex optimization," IEEE Trans. Signal Process., vol. 51, no. 9, pp. 2381-2401, 2003.
54. T. M. Cover and J.A. Thomas, Elements of Information Theory. New York, NY, USA: Wiley, 1991.
55. J. Duarte-Carvajalino, G. Sapiro, G. Yu, and L. Carin, "Online adaptive statistical compressed sensing of Gaussian mixturemodels," arXiv e-print, 2011 [Online]. Available: http://arxiv.org/abs/1112.5895
56. M. S. Srivastava and C. G. Khatri, An Introduction to Multivariate Statistics. Amsterdam, The Netherlands: North-Holland, 1979.
57. J. T. Flåm, S. Chatterjee, K. Kansanen, and T. Ekman, "Minimum mean square error estimation under Gaussian mixture statistics," arXiv e-prints, 2011 [Online]. Available: http://xxx.tau.ac.il/abs/1108.3410
58. R. Horn and C. Johnson, Matrix Analysis. Cambridge, U.K.: Cambridge Univ. Press, 1985.