Low-dimensional models for dimensionality reduction and signal recovery: A geometric perspective

Proceedings of the IEEE

Volumn 98, Issue 6, 2010, Pages 959-971

  Baraniuk, Richard G ^a   Cevher, Volkan ^a,b   Wakin, Michael B ^c  

^a Rice University   (United States)

^b EPFL   (Switzerland)

^c Department of Geology and Geological Engineering   (United States)

Author keywords

Compression; Compressive sensing; Dimensionality reduction; Manifold; Point cloud; Sparsity; Stable embedding

Indexed keywords

COMPACTION; GEOMETRY;

COMPRESSIVE SENSING; DIMENSIONALITY REDUCTION; MANIFOLD; POINT CLOUD; SPARSITY; STABLE EMBEDDING;

SIGNAL RECONSTRUCTION;

EID: 77952743002     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/JPROC.2009.2038076     Document Type: Review

References (74)

1. G. E. Hinton, P. Dayan, and M. Revow, "Modeling the manifolds of images of handwritten digits," IEEE Trans. Neural Netw., vol.8, no.1, pp. 65-74, Jan. 1997. (Pubitemid 127767781)
2. M. W. Hirsch, Differential Topology, vol.33, Graduate Texts in Mathematics. New York: Springer-Verlag, 1976.
3. J. Nash, "The imbedding problem for Riemannian manifolds," Ann. Math., vol.63, pp. 20-63, Jan. 1956.
4. B. Kashin, "The widths of certain finite-dimensional sets and classes of smooth functions," Izv. Akad. Nauk SSSR Ser. Mat, vol.41, no.2, pp. 334-351, 1977.
5. A. Garnaev and E. Gluskin, "On the widths of the Euclidean ball," Doklady, Akad. Nauk SSSR, vol.277, pp. 1048-1052, 1984.
6. S. Dasgupta and A. Gupta, "An elementary proof of the Johnson-Lindenstrauss lemma," Berkeley, CA, Tech. Rep. TR-99-1006, 1999.
7. D. Achlioptas, "Database-friendly random projections," in Proc. Symp. Principles Database Syst., 2001, pp. 274-281, ACM Press. (Pubitemid 32865990)
8. R. Baraniuk, M. Davenport, R. DeVore, and M. Wakin, "A simple proof of the restricted isometry property for random matrices," Constr. Approx., vol.28, pp. 253-263, Dec. 2008.
9. M. Davenport, M. Duarte, M. Wakin, J. Laska, D. Takhar, K. Kelly, and R. Baraniuk, "The smashed filter for compressive classification and target recognition," in Proc. Comput. Imag. V SPIE Electronic Imag., Jan. 2007. (Pubitemid 47320352)
10. D. Baron, M. B. Wakin, M. Duarte, S. Sarvotham, and R. G. Baraniuk, "Distributed compressed sensing," Rice Univ., Houston, TX, Tech. Rep. TREE-0612, Nov. 2006.
11. E. J. Cand̀es and T. Tao, "Decoding by linear programming," IEEE Trans. Inf. Theory, vol.51, no.12, pp. 4203-4215, Dec. 2005. (Pubitemid 41800353)
12. D. L. Donoho, "Compressed sensing," IEEE Trans. Inf. Theory, vol.52, no.9, pp. 1289-1306, Sep. 2006.
13. R. G. Baraniuk, "Compressive sensing," IEEE Signal Process. Mag., vol.24, pp. 118-120, 124, Jul. 2007.
14. E. J. Cand̀es, "The restricted isometry property and its implications for compressed sensing," Compte Rendus de l'Academie des Sciences, ser. I, vol.346, pp. 589-592, May 2008.
15. D. Needell and R. Vershynin, "Uniform uncertainty principle and signal recovery via regularized orthogonal matching pursuit," Found. Comput. Math., vol.9, pp. 317-334, Jun. 2009.
16. D. Needell and J. Tropp, "CoSaMP: Iterative signal recovery from incomplete and inaccurate samples," Appl. Comput. Harmon. Anal., vol.26, pp. 301-321, May 2009.
17. T. Blumensath and M. E. Davies, "Iterative hard thresholding for compressed sensing," Appl. Comput. Harmon. Anal., vol.27, pp. 265-274, Nov. 2009.
18. A. Cohen, W. Dahmen, and R. DeVore, "Compressed sensing and best k-term approximation," J. Amer. Math. Soc., vol.22, pp. 211-231, 2009.
19. 1-minimization decoder," Appl. Comput. Harmon. Anal., vol.27, pp. 275-288, Nov. 2009.
20. R. A. DeVore, "Lecture notes on compressed sensing," Rice Univ., Spring 2006, ELEC 631 course notes. [Online]. Available: http://cnx.org/content/col10458/latest/
21. E. Candès and T. Tao, "Near optimal signal recovery from random projections: Universal encoding strategies?" IEEE Trans. Inf. Theory, vol.52, no.12, pp. 5406-5425, Dec. 2006. (Pubitemid 46445381)
22. D. P. Wipf and B. D. Rao, "Sparse Bayesian learning for basis selection," IEEE Trans. Signal Process., vol.52, no.8, pp. 2153-2164, Aug. 2004.
23. S. Ji, Y. Xue, and L. Carin, "Bayesian compressive sensing," IEEE Trans. Signal Process., vol.56, no.6, pp. 2346-2356, Jun. 2008. (Pubitemid 351795888)
24. P. Schniter, L. C. Potter, and J. Ziniel, "Fast bayesian matching pursuit," in Proc. Inf. Theory Appl. Workshop, 2008, pp. 326-333.
25. M. Tipping, "Sparse Bayesian learning and the relevance vector machine," J. Mach. Learn. Res., vol.1, pp. 211-244, 2001. (Pubitemid 33687203)
26. S. S. Chen, D. L. Donoho, and M. A. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Sci. Comput., vol.20, p. 33, 1998.
27. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. R. Stat. Soc., pp. 267-288, 1996.
28. R. Saab, R. Chartrand, and O. Yilmaz, "Stable sparse approximations via nonconvex optimization," in Proc. ICASSP, 2008, pp. 3885-3888.
29. V. Cevher, "Learning with compressible priors," in Neural Inf. Process. Syst. (NIPS), Vancouver, BC, Canada, Dec. 7-12, 2008.
30. T. Blumensath and M. E. Davies, "Sampling theorems for signals from the union of finite-dimensional linear subspaces," IEEE Trans. Inf. Theory, to be published.
31. Y. M. Lu and M. N. Do, "Sampling signals from a union of subspaces," IEEE Signal Process. Mag., vol.25, pp. 41-47, Mar. 2008.
32. V. Cevher, P. Indyk, C. Hegde, and R. G. Baraniuk, "Recovery of clustered sparse signals from compressive measurements," in Proc. Sampl. Theory Appl. (SAMPTA), Marseilles, France, May 2009.
33. R. G. Baraniuk, V. Cevher, M. F. Duarte, and C. Hegde, "Model-based compressive sensing," 2008, preprint. [Online]. Available: http://dsp.rice.edu/cs
34. J. Tropp, A. C. Gilbert, and M. J. Strauss, "Algorithms for simultaneous sparse approximation, Part I: Greedy pursuit," Signal Process., vol.86, pp. 572-588, Apr. 2006. (Pubitemid 43106572)
35. Y. C. Eldar and M. Mishali, "Robust recovery of signals from a structured union of subspaces," IEEE Trans. Inf. Theory, vol.55, no.11, pp. 5302-5316, Nov. 2009.
36. M. Stojnic, F. Parvaresh, and B. Hassibi, "On the reconstruction of block-sparse signals with an optimal number of measurements," IEEE Trans. Signal Process., vol.57, no.8, pp. 3075-3085, Aug. 2009.
37. Y. Kim, J. Kim, and Y. Kim, "Blockwise sparse regression," Stat. Sinica, vol.16, no.2, p. 375, 2006. (Pubitemid 44085508)
38. L. Meier, S. van de Geer, and P. Buhlmann, "The group lasso for logistic regression," J. R. Stat. Soc. B (Stat. Methodol.), vol.70, no.1, pp. 53-71, 2008.
39. F. Parvaresh, H. Vikalo, S. Misra, and B. Hassibi, "Recovering sparse signals using sparse measurement matrices in compressed DNA microarrays," IEEE J. Sel. Topics Signal Process., vol.2, no.3, pp. 275-285, 2008.
40. I. F. Gorodnitsky, J. S. George, and B. D. Rao, "Neuromagnetic source imaging with FOCUSS: A recursive weighted minimum norm algorithm," Electroenceph. Clin. Neurophys., vol.95, no.4, pp. 231-251, 1995.
41. M. S. Crouse, R. D. Nowak, and R. G. Baraniuk, "Wavelet-based statistical signal processing using hidden Markov models," IEEE Trans. Signal Process., vol.46, no.4, pp. 886-902, Apr. 1998. (Pubitemid 128743828)
42. J. K. Romberg, H. Choi, and R. G. Baraniuk, "Bayesian tree-structured image modeling using wavelet-domain hidden Markov models," IEEE Trans. Image Process., vol.10, pp. 1056-1068, Jul. 2001. (Pubitemid 32650882)
43. J. Portilla, V. Strela, M. J. Wainwright, and E. P. Simoncelli, "Image denoising using a scale mixture of Gaussians in the wavelet domain," IEEE Trans. Image Process., vol.12, pp. 1338-1351, Nov. 2003.
44. J. Shapiro, "Embedded image coding using zerotrees of wavelet coefficients," IEEE Trans. Signal Process., vol.41, no.12, pp. 3445-3462, Dec. 1993.
45. A. Cohen, W. Dahmen, I. Daubechies, and R. A. DeVore, "Tree approximation and optimal encoding," Appl. Comput. Harmon. Anal., vol.11, pp. 192-226, Sep. 2001. (Pubitemid 33675489)
46. C. Hegde, M. F. Duarte, and V. Cevher, "Compressive sensing recovery of spike trains using a structured sparsity model," in Proc. Signal Process. Adapt. Sparse Struct. Represent. (SPARS), Saint-Malo, France, Apr. 13-16, 2009.
47. Y. Ma, A. Yang, H. Derksen, and R. Fossum, "Estimation of subspace arrangements with applications in modeling and segmenting mixed data," SIAM Rev., vol.50, no.3, pp. 413-458, 2008.
48. L. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol.77, no.2, pp. 257-286, Feb. 1989. (Pubitemid 20604103)
49. R. Chellappa and A. Jain, Markov Random Fields: Theory and Application. New York: Academic, 1993.
50. L. He and L. Carin, "Exploiting structure in wavelet-based Bayesian compressive sensing," IEEE Trans. Signal Process., vol.57, no.9, pp. 3488-3497, Sep. 2009.
51. V. Cevher, M. F. Duarte, C. Hegde, and R. G. Baraniuk, "Sparse signal recovery using markov random fields," in Proc. Neural Inf. Process. Syst. (NIPS), Vancouver, BC, Canada, Dec. 8-11, 2008.
52. J. Tropp, "Algorithms for simultaneous sparse approximation, Part II: Convex relaxation," Signal Process., vol.86, pp. 589-602, Mar. 2006. (Pubitemid 43106573)
53. P. Nagesh and B. Li, "Compressive imaging of color images," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Taipei, Taiwan, Apr. 2009.
54. H. Jung, K. Sung, K. Nayak, E. Kim, and J. Ye, "k-t FOCUSS: A general compressed sensing framework for high resolution dynamic MRI," Magn. Res. Med., vol.61, no.1, 2009.
55. J. Y. Park and M. B. Wakin, "A multiscale framework for compressive sensing of video," in Proc. Picture Coding Symp. (PCS), Chicago, IL, May 2009.
56. R. Marcia and R. Willet, "Compressive coded aperture video reconstruction," in Proc. Eur. Signal Process. Conf. (EUSIPCO), 2008.
57. C. Qiu, W. Lu, and N. Vaswani, "Real-time dynamic MR image reconstruction using Kalman filtered compressed sensing," in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), Taipei, Taiwan, Apr. 2009.
58. D. Angelosante and G. B. Giannakis, "RLS-weighted LASSO for adaptive estimation of sparse signals," in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), Taipei, Taiwan, Apr. 2009.
59. S. Ji, D. Dunson, and L. Carin, "Multi-task compressive sensing," IEEE Trans. Signal Process., vol.57, no.1, pp. 92-106, Jan. 2009.
60. K. L. Clarkson, "Tighter bounds for random projections of manifolds," in Proc. Annu. Symp. Comput. Geometry, College Park, MD, 2008, pp. 39-48.
61. M. Fornasier and H. Rauhut, "Recovery algorithms for vector valued data with joint sparsity constraints," SIAM J. Numer. Anal., vol.46, no.2, pp. 577-613, 2008.
62. R. Gribonval, H. Rauhut, K. Schnass, and P. Vandergheynst, "Atoms of all channels, unite! Average case analysis of multi-channel sparse recovery using greedy algorithms," J. Fourier Anal. Applicat., vol.14, no.5, pp. 655-687, 2008.
63. P. Indyk and R. Motwani, "Approximate nearest neighbors: Towards removing the curse of dimensionality," in Proc. Symp. Theory Comput., 1998, pp. 604-613.
64. R. G. Baraniuk and M. B. Wakin, "Random projections of smooth manifolds," Found. Comput. Math., vol.9, pp. 51-77, Feb. 2009.
65. P. Indyk and A. Naor, "Nearest neighbor preserving embeddings," ACM Trans. Algorithms, 2006.
66. P. Agarwal, S. Har-Peled, and H. Yu, "Embeddings of surfaces, curves, and moving points in euclidean space," in Proc. 23rd Annu. ACM Symp. Comput. Geometry, New York, 2007, pp. 381-389. (Pubitemid 47582085)
67. S. Dasgupta and Y. Freund, "Random projection trees and low dimensional manifolds," in Proc. 40th Annu. ACM Symp. Theory Comput., New York, 2008, pp. 537-546.
68. M. B. Wakin, "Manifold-based signal recovery and parameter estimation from compressive measurements," 2008, preprint. [Online]. Available: http://arxiv.org/abs/1002.1247
69. M. B. Wakin, D. L. Donoho, H. Choi, and R. G. Baraniuk, "The multiscale structure of non-differentiable image manifolds," in Proc. Wavelets XI SPIE Opt. Photon., San Diego, CA, Aug. 2005. (Pubitemid 43105324)
70. C. Hegde, M. Wakin, and R. Baraniuk, "Random projections for manifold learning," in Proc. Neural Inf. Process. Syst. (NIPS), Dec. 2007.
71. M. B. Wakin, "A manifold lifting algorithm for multi-view compressive imaging," in Proc. Picture Coding Symp. (PCS 2009), Chicago, IL, May 2009.
72. M. A. Davenport, C. Hegde, M. F. Duarte, and R. G. Baraniuk, "A theoretical analysis of joint manifolds," ECE Dept., Rice Univ., Tech. Rep. TREE 0901, Jan. 2009, arXiv:0901.0760.
73. H. David and H. Nagaraja, Order Statistics. Wiley-Interscience, 2004.
74. H. Alzer, "On some inequalities for the incomplete gamma function," Math. Comput., vol.66, no.218, pp. 771-778, 1997. (Pubitemid 127058736)