Compressible distributions for high-dimensional statistics

IEEE Transactions on Information Theory

Volumn 58, Issue 8, 2012, Pages 5016-5034

  Gribonval, Rémi ^a   Cevher, Volkan ^b   Davies, Mike E ^c  

^a INRIA   (France)

^b EPFL   (Switzerland)

^c UNIVERSITY OF EDINBURGH   (United Kingdom)

Author keywords

Basis pursuit; compressed sensing; compressible distribution; high dimensional statistics; instance optimality; Lasso; linear inverse problems; maximum a posteriori (MAP) estimator; order statistics; sparsity; statistical regression

Indexed keywords

BASIS PURSUITS; COMPRESSIBLE DISTRIBUTION; COMPRESSIVE SENSING; HIGH-DIMENSIONAL; INSTANCE OPTIMALITY; LASSO; LINEAR INVERSE PROBLEMS; MAXIMUM A POSTERIORI; ORDER STATISTICS; SPARSITY; STATISTICAL REGRESSION;

ESTIMATION; IMAGE RECONSTRUCTION; INVERSE PROBLEMS; LAPLACE TRANSFORMS; PROBABILITY DISTRIBUTIONS;

INCOMPRESSIBLE FLOW;

EID: 84863937465     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2012.2197174     Document Type: Article

References (33)

1. A.Amini,M.Unser, and F. Marvasti, "Compressibility of deterministic and random infinite sequences," IEEE Trans. Signal Process., vol. 59, no. 11, pp. 5193-5201, Nov. 2011.
2. R. G. Baraniuk, V. Cevher, and M. B. Wakin, "Low-dimensional models for dimensionality reduction and signal recovery: A geometric perspective," Proc. IEEE, vol. 98, no. 6, pp. 959-971, Jun. 2010.
3. A. Barvinok, Math 710: Measure Concentration, Lecture Notes, 2005.
4. T. Blumensath and M. E. Davies, "Sampling theorems for signals from the union of finite-dimensional linear subspaces," IEEE Trans. Inf. Theory, vol. 55, no. 4, pp. 1872-1882, Apr. 2009.
5. F. T. Bruss and J. B. Robertson, "'Wald's lemma' for sums of order statistics of i.i.d. random variables," Adv. Appl. Probability, vol. 23, no. 3, pp. 612-623, Sep. 1991.
6. E. J. Candès, J. Romberg, and T. Tao, "Stable signal recovery from incomplete and inaccuratemeasurements," Commun. Pure Appl.Math, vol. 59, pp. 1207-1223, 2006. (Pubitemid 43988295)
7. E. J. Candès and T. Tao, "Near-optimal signal recovery from random projections: Universal encoding strategies?," IEEE Trans. Inf. Theory, vol. 52, no. 12, pp. 5406-5425, Dec. 2006. (Pubitemid 46445381)
8. E. Candès, "The restricted isometry property and its implications for compressed sensing," C. R. Acad. Sci., Paris, Ser. I, vol. 346, pp. 589-592, 2008.
9. V. Cevher, "Learning with compressible priors," in Proc. Neural Information Processing Systems, Vancouver, B.C., Canada, Dec. 7-12, 2008.
10. G. Chang, B. Yu, and M. Vetterli, "Adaptive wavelet thresholding for image denoising and compression," IEEE Trans. Image Proc., vol. 9, no. 9, pp. 1532-1546, Sep. 2000.
11. S. Chen, D. L. Donoho, and M. A. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Sci. Comput., vol. 20, no. 1, pp. 33-61, Jan. 1999. (Pubitemid 128689501)
12. H. Choi and R. G. Baraniuk, "Wavelet statistical models and Besov spaces," Proc. SPIE, vol. 3813, pp. 489-501, Jul. 1999. (Pubitemid 30554046)
13. A. Cohen, W. Dahmen, and R. A. DeVore, "Compressed sensing and best k-term approximation," J. Amer. Math. Soc., vol. 22, pp. 211-231, 2009.
14. M. E. Davies and R. Gribonval, "On Lp minimisation, instance optimality, and restricted isometry constants for sparse approximation," in Proc. Sampling Theory and Appl., Marseille, France, May 2009.
15. R. DeVore, G. Petrova, and P. Wojtaszczyk, "Instance-optimality in probability with an l1-minimization decoder," Appl. Comput. Harmonic Anal., vol. 27, no. 3, pp. 275-288, 2009.
16. D. Donoho and J. Tanner, "Counting faces of randomly-projected polytopes when the projection radically lowers dimension," J. Amer. Math. Soc., vol. 22, no. 1, pp. 1-53, Jan. 2009.
17. D. L. Donoho, "Compressed sensing," IEEE Trans. Inform. Theory, vol. 52, no. 4, pp. 1289-1306, Apr. 2006.
18. D. L. Donoho, I. Johnstone, A. Maleki, and A. Montanari, "Compressed sensing over -balls: Minimax mean square error," in Proc. IEEE Int. Symp. Inform. Theory, 2011, pp. 129-133.
19. R. M. Gray, Probability, Random Processes, and Ergodic Properties. New York: Springer, 2009.
20. R.Gribonval, "Should penalized least squares regression be interpreted as maximum a posteriori estimation?," IEEE Trans. Signal Process., vol. 59, no. 5, pp. 2405-2410, May 2011.
21. R. Gribonval, R. M. V. F. i Ventura, and P. Vandergheynst, "A simple test to check the optimality of sparse signal approximations," EURASIP Signal Process., vol. 86, no. 3, pp. 496-510, Mar. 2006. (Pubitemid 43106568)
22. R. Gribonval and M. Nielsen, "On the strong uniqueness of highly sparse expansions from redundant dictionaries," in Proc. Int. Conf. Independent Comp. Anal., Granada, Spain, Sep. 2004.
23. R. Gribonval and M. Nielsen, "Highly sparse representations from dictionaries are unique and independent of the sparseness measure," Appl. Comput. Harm. Anal., vol. 22, no. 3, pp. 335-355, May 2007. (Pubitemid 46528317)
24. D. Martin, C. Fowlkes, D. Tal, and J.Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics," in Proc. 8th Int. Conf. Comput. Vision, Jul. 2001, vol. 2, pp. 416-423. (Pubitemid 32795086)
25. R. J. Muirhead, Aspects of Multivariate Statistical Theory. Hoboken, NJ: Wiley, 2008.
26. M. Nikolova, "Model distortions in Bayesian MAP reconstruction, " Inverse Problems Imaging, vol. 1, no. 2, pp. 399-422, 2007.
27. J. Portilla, V. Strela, M. J. Wainwright, and E. P. Simoncelli, "Image denoising using scale mixtures of gaussians in the wavelet domain," IEEE Trans. Image Process., vol. 12, no. 11, pp. 1338-1351, Nov. 2003.
28. D. L. Ruderman and W. Bialek, "Statistics of natural images: Scaling in the woods," Phys. Rev. Lett., vol. 73, no. 6, pp. 814-817, 1994. (Pubitemid 24976330)
29. M. Seeger and H. Nickisch, "Compressed sensing and Bayesian experimental design," in Proc. Int. Conf.Mach. Learning, 2008, vol. 25.
30. E. P. Simoncelli and B. A. Olshausen, "Natural image statistics and neural representation," Annu. Rev. Neurosci., vol. 24, no. 1, pp. 1193-1216, 2001. (Pubitemid 32695258)
31. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Roy. Stat. Soc., Series B, vol. 58, pp. 267-288, 1994.
32. D. Wipf and S. Nagarajan, "A new view of automatic relevance determination," in Advances in Neural Information Processing Systems 20 (NIPS), J. C. Platt, D. Koller, Y. Singer, and S. Roweis, Eds. Cambridge, MA: MIT Press, 2008.
33. W.Xu and B.Hassibi, "Compressive sensing over the Grassmannmanifold: A unified geometric framework," IEEE Trans. Inf. Theory, to be published.