Random projections of smooth manifolds

Foundations of Computational Mathematics

Volumn 9, Issue 1, 2009, Pages 51-77

  Baraniuk, Richard G ^a   Wakin, Michael B ^b  

^a Rice University   (United States)

^b UNIVERSITY OF MICHIGAN   (United States)

Author keywords

Compressed sensing; Dimensionality reduction; Johnson Lindenstrauss lemma; Manifold learning; Manifolds; Random projections; Sparsity

Indexed keywords

DIMENSIONALITY REDUCTION; JOHNSON-LINDENSTRAUSS LEMMATA; MANIFOLD LEARNING; RANDOM PROJECTIONS; SPARSITY;

AUTOMOBILE ENGINE MANIFOLDS; COMPRESSED SENSING; SIGNAL RECONSTRUCTION;

RECOVERY;

EID: 58849146227     PISSN: 16153375     EISSN: 16153383     Source Type: Journal    
DOI: 10.1007/s10208-007-9011-z     Document Type: Article

References (48)

1. D. Achlioptas, Database-friendly random projections, in Proc. Symp. on Principles of Database Systems (PODS '01), pp. 274-281, ACM Press, New York, 2001.
2. R. Baraniuk, M. Davenport, R. DeVore, and M. Wakin, A simple proof of the restricted isometry property for random matrices. Constr. Approx. (2008), to appear.
3. D. Baron, M. B. Wakin, M. F. Duarte, S. Sarvotham, and R. G. Baraniuk, Distributed compressed sensing, Preprint, 2005.
4. M. Belkin P. Niyogi 2003 Laplacian eigenmaps for dimensionality reduction and data representation Neural Comput. 15 6 1373 1396
5. M. Brand, Charting a manifold, in Advances in Neural Information Processing Systems (NIPS), Vol. 15, pp. 985-992, MIT Press, Cambridge, 2003.
6. D. S. Broomhead M. Kirby 2000 A new approach for dimensionality reduction: Theory and algorithms SIAM J. Appl. Math. 60 6 2114 2142
7. D. S. Broomhead M. J. Kirby 2001 The Whitney reduction network: A method for computing autoassociative graphs Neural Comput. 13 11 2595 2616
8. E. Candès J. Romberg T. Tao 2006 Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information IEEE Trans. Inf. Theory 52 2 489 509
9. E. Candès J. Romberg T. Tao 2006 Stable signal recovery from incomplete and inaccurate measurements Commun. Pure Appl. Math. 59 8 1207 1223
10. E. Candès T. Tao 2005 Decoding via linear programming IEEE Trans. Inf. Theory 51 12 4203 4215
11. E. Candès and T. Tao, The Dantzig selector: Statistical estimation when p is much larger than n, Ann. Stat. (2007), to appear. arXiv: math.ST/0506081.
12. E. Candès T. Tao 2006 Near optimal signal recovery from random projections: Universal encoding strategies? IEEE Trans. Inf. Theory 52 12 5406 5425
13. G. Carlsson, A. Zomorodian, A. Collins, and L. Guibas, Persistence bar codes for shapes, in Proc. Symp. on Geometry processing (SGP '04), pp. 124-135, ACM Press, New York, 2004.
14. R. R. Coifman M. Maggioni 2006 Diffusion wavelets Appl. Comput. Harmon. Anal. 21 1 53 94
15. J. A. Costa A. O. Hero 2004 Geodesic entropic graphs for dimension and entropy estimation in manifold learning IEEE Trans. Signal Process. 52 8 2210 2221
16. S. Dasgupta A. Gupta 2003 An elementary proof of a theorem of Johnson and Lindenstrauss Random Struct. Algorithms 22 1 60 65
17. D. Donoho, Neighborly polytopes and sparse solution of underdetermined linear equations, Technical Report 2005-04, Department of Statistics, Stanford University, 2005.
18. D. Donoho, Compressed sensing, IEEE Trans. Inf. Theory 52(4) (2006).
19. D. Donoho, For most large underdetermined systems of linear equations, the minimal L1-norm solution is also the sparsest solution, Commun. Pure Appl. Math. 59(6) (2006).
20. D. Donoho 2006 High-dimensional centrally symmetric polytopes with neighborliness proportional to dimension Discrete Comput. Geom. 35 4 617 652
21. D. Donoho J. Tanner 2005 Neighborliness of randomly-projected simplices in high dimensions Proc. Natl. Acad. Sci. USA 102 27 9452 9457
22. D. Donoho Y. Tsaig 2006 Extensions of compressed sensing Signal Process. 86 3 533 548
23. D. L. Donoho C. Grimes 2005 Image manifolds which are isometric to Euclidean space J. Math. Imaging Comput. Vis. 23 1 5 24
24. D. L. Donoho C. E. Grimes 2003 Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data Proc. Natl. Acad. Sci. USA 100 10 5591 5596
25. D. L. Donoho and J. Tanner, Counting faces of randomly-projected polytopes when then projection radically lowers dimension, Technical Report 2006-11, Department of Statistics, Stanford University, 2006. arXiv: math.MG/0607364.
26. C. Grimes, New Methods in Nonlinear Dimensionality Reduction, Ph.D. thesis, Department of Statistics, Stanford University, 2003.
27. J. Haupt R. Nowak 2006 Signal reconstruction from noisy random projections IEEE Trans. Inf. Theory 52 9 4036 4048
28. G. E. Hinton P. Dayan M. Revow 1997 Modeling the manifolds of images of handwritten digits IEEE Trans. Neural Netw. 8 1 65 74
29. M. W. Hirsch, Differential Topology, Graduate Texts in Mathematics, Vol. 33, Springer, New York, 1976.
30. P. Indyk and A. Naor, Nearest-neighbor-preserving embeddings, ACM Trans. Algorithms 3(3) (2007).
31. S. Kirolos, J. Laska, M. Wakin, M. Duarte, D. Baron, T. Ragheb, Y. Massoud, and R. Baraniuk, Analog-to-information conversion via random demodulation, Proc. IEEE Dallas Circuits and Systems Workshop (DCAS), Dallas, TX, October 2006.
32. G. G. Lorentz, M. von Golitschek, and Y. Makovoz, Constructive Approximation: Advanced Problems, Grundlehren der Mathematischen Wissenschaften, Vol. 304, Springer, Berlin, 1996.
33. S. Mallat, A Wavelet Tour of Signal Processing, Academic Press, San Diego, 1999.
34. P. Niyogi S. Smale S. Weinberger 2006 Finding the homology of submanifolds with confidence from random samples Discrete Comput. Geom. 10.1007/s00454-006-1250-7
35. A. Pinkus, n-Widths and Optimal Recovery, in Proceedings of Symposia in Applied Mathematics, Vol. 36, pp. 51-66, American Mathematical Society, Providence, 1986.
36. I. Ur Rahman I. Drori V. C. Stodden D. L. Donoho P. Schroeder 2005 Multiscale representations for manifold-valued data SIAM J. Multiscale Model. Simul. 4 4 1201 1232
37. S. T. Roweis L. K. Saul 2000 Nonlinear dimensionality reduction by locally linear embedding Science 290 5500 2323 2326
38. M. Rudelson R. Vershynin 2005 Geometric approach to error correcting codes and reconstruction of signals Int. Math. Res. Not. 64 4019 4041
39. D. Takhar, V. Bansal, M. Wakin, M. Duarte, D. Baron, K. F. Kelly, and R. G. Baraniuk, A compressed sensing camera: New theory and an implementation using digital micromirrors, Proc. Comp. Imaging IV at SPIE Electronic Imaging, San Jose, CA, January 2006.
40. D. S. Taubman and M. W. Marcellin, JPEG 2000: Image Compression Fundamentals, Standards and Practice, Kluwer Academic, Dordrecht, 2001.
41. J. B. Tenenbaum V. de Silva J. C. Langford 2000 A global geometric framework for nonlinear dimensionality reduction Science 290 5500 2319 2323
42. J. A. Tropp, M. B. Wakin, M. F. Duarte, D. Baron, and R. G. Baraniuk, Random filters for compressive sampling and reconstruction, in Proc. Int. Conf. Acoustics, Speech, Signal Processing (ICASSP), IEEE, New York, 2006.
43. M. Turk A. Pentland 1991 Eigenfaces for recognition J. Cogn. Neurosci. 3 1 71 83
44. M. B. Wakin, The Geometry of Low-Dimensional Signal Models, Ph.D. thesis, Department of Electrical and Computer Engineering, Rice University, Houston, TX, 2006.
45. M. B. Wakin and R. G. Baraniuk, Random projections of signal manifolds, in Proc. Int. Conf. Acoustics, Speech, Signal Processing (ICASSP), IEEE, New York, 2006.
46. M. B. Wakin, D. L. Donoho, H. Choi, and R. G. Baraniuk, The multiscale structure of non-differentiable image manifolds, in Proc. Wavelets XI at SPIE Optics and Photonics, San Diego, CA, August 2005.
47. K. Q. Weinberger L. K. Saul 2006 Unsupervised learning of image manifolds by semidefinite programming Int. J. Comput. Vis. 70 1 77 90 Special issue: Comput. Vis. Pattern Recognit. (CVPR 2004)
48. Z. Zhang H. Zha 2005 Principal manifolds and nonlinear dimension reduction via tangent space alignment SIAM J. Sci. Comput. 26 1 313 338