Learning low-dimensional signal models

IEEE Signal Processing Magazine

Volumn 28, Issue 2, 2011, Pages 39-51

  Carin, Lawrence ^a   Baraniuk, Richard ^b   Cevher, Volkan ^c   Dunson, David ^d   Jordan, Michael ^e   Sapiro, Guillermo ^f   Wakin, Michael ^c  

^a UNIVERSITY OF MARYLAND   (United States)

^b MIT Technology   (United States)

^c RICE UNIVERSITY   (United States)

^d DUKE UNIVERSITY   (United States)

^e UNIVERSITY OF CALIFORNIA   (United States)

^f UNIVERSITY OF MINNESOTA   (United States)

Author keywords

Analytical models; Bayesian methods; Data models; Manifolds; Monitoring; Training data

Indexed keywords

ANALOG TO DIGITAL CONVERSION; ANALYTICAL MODELS; AUTOMOBILE ENGINE MANIFOLDS; BAYESIAN NETWORKS; DATA STRUCTURES; MEDICAL IMAGING; MONITORING;

ANALOG TO DIGITAL CONVERTERS; BAYESIAN METHODS; COMMUNICATION BANDWIDTH; DATA DIMENSIONS; DIMENSIONAL SPACES; STORAGE RESOURCES; TRAINING DATA; WEATHER MONITORING;

DIGITAL STORAGE;

EID: 85032751900     PISSN: 10535888     EISSN: None     Source Type: Journal    
DOI: 10.1109/MSP.2010.939733     Document Type: Article

References (55)

1. E. Candè s, J. Romberg, and T. Tao, "Signal recovery from incomplete and inaccurate measurements," Commun. Pure Appl. Math., vol. 59, pp. 1207-1223, 2005.
2. D. L. Donoho, "Compressed sensing," IEEE Trans. Inform. Theory, vol. 52, pp. 1289-1306, 2006.
3. E. J. Candès and B. Recht, "Exact matrix completion via convex optimization," Found. Comput. Math., vol. 9, pp. 717-772, 2008.
4. B. Recht, M. Fazel, and P. Parrilo, "Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization," submitted for publication.
5. M. Chen, J. Silva, J. Paisley, C. Wang, D. Dunson, and L. Carin, "Compressive sensing on manifolds using a nonparametric mixture of factor analyzers: Algorithm and performance bounds," IEEE Trans. Signal Processing, 2010.
6. J. Paisley and L. Carin, "Nonparametric factor analysis with beta process priors," in Proc. Int. Conf. Machine Learning, 2009 .
7. R. Baraniuk, V. Cevher, and M. Wakin, "Low-dimensional models for dimensionality reduction and signal recovery: A geometric perspective," in Proc. IEEE, to be published.
8. R. G. Baraniuk, V. Cevher, M. F. Duarte, and C. Hegde (2008). Model-based compressive sensing. Preprint [Online]. Available: http://dsp.rice.edu/cs
9. E. Candès and T. Tao, "Decoding by linear programming," IEEE Trans. Inform. Theory, vol. 51, pp. 4203-4215, 2005.
10. R. Baraniuk, M. Davenport, R. DeVore, and M. Wakin, "A simple proof of the restricted isometry property for random matrices," Construct. Approx., vol. 28, pp. 253-263, 2008.
11. D. L. Donoho and C. Grimes, "Image manifolds which are isometric to Euclidean space," J. Math. Imag. Comput. Vision, vol. 23, no. 1, pp. 5-24, July 2005.
12. R. Baraniuk and M. Wakin, "Random projections of smooth manifolds," Foundat. Comput. Math., vol. 9, no. 1, pp. 51-77, 2009.
13. S. S. Chen, D. L. Donoho, and M. A. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Scient. Comput., vol. 20, p. 33, 1998.
14. Y. Pati, R. Rezaiifar, and P. Krishnaprasad, "Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition," in Proc. 27th Ann. Asilomar Conf. Signals, Systems, and Computers, 1993 .
15. D. Needell and J. Tropp, "CoSaMP: Iterative signal recovery from incomplete and inaccurate samples," Appl. Comput. Harmon. Anal., to be published.
16. D. P. Wipf and B. D. Rao, "Sparse Bayesian learning for basis selection," IEEE Trans. Signal Processing, vol. 52, no. 8, pp. 2153-2164, 2004.
17. S. Ji, Y. Xue, and L. Carin, "Bayesian compressive sensing," IEEE Trans. Signal Processing, vol. 56, no. 6, pp. 2346-2356, 2008.
18. P. Schniter, L. C. Potter, and J. Ziniel, "Fast Bayesian matching pursuit," in Proc. Information Theory and Applications Workshop, 2008, pp. 326-333.
19. M. B. Wakin, "Manifold-based signal recovery and parameter estimation from compressive measurements," Preprint, 2008.
20. J. Bernardo and A. Smith, Bayesian Theory. New York: Wiley, 2004.
21. T. L. Griffiths and Z. Ghahramani, "Infinite latent feature models and the Indian buffet process," in Proc. Advances in Neural Information Processing Systems, 2005.
22. Z. Ghahramani and M. Beal, "Variational inference for Bayesian mixtures of factor analysers," in Proc. Conf. Advances in Neural Information Processing Systems, NIPS, 2000 .
23. J. Berger, J. Ghosh, and N. Mukhopadhyay, "Approximation and consistency of Bayes factors as model dimension grows," J. Statist. Plan. Infer., vol. 112, pp. 241-258, 2003.
24. S. Press and K. Shigemasu, "A note on choosing the number of factors," Commun. Statist. Theory Methods, vol. 28, pp. 1653-1670, 1999.
25. S. Lee and X. Song, "Bayesian selection on the number of factors in a factor analysis model," Behaviormetrika, vol. 29, pp. 23-39, 2002.
26. T. Ferguson, "A Bayesian analysis of some nonparametric problems," Ann. Statistics, vol. 1, pp. 209-230, 1973.
27. M. Tipping and C. Bishop, "Mixture of principal component analyzers," Neural Comput., vol. 11, pp. 443-482, 1999.
28. A. Eftekhari and M. B. Wakin, "Performance bounds for compressive sensing with a mixture of factor analyzers," School of Mines, Tech. Rep. 2010.
29. L. He and L. Carin, "Exploiting structure in wavelet-based Bayesian compressive sensing," IEEE Trans. Signal Processing, vol. 57, pp. 3488-3497, 2009.
30. R. Salakhutdinov and A. Mnih, "Bayesian probabilistic matrix factorization with MCMC," in Proc. Advances in Neural Information Processing Systems, 2008.
31. R. Salakhutdinov and A. Mnih, "Probabilistic matrix factorization," in Proc. Advances in Neural Information Processing Systems, 2008.
32. Y. C. Eldar and M. Mishali, "Robust recovery of signals from a union of subspaces," IEEE Trans. Inform. Theory, vol. 12, pp. 1338-1351, 2008.
33. N. Hjort, C. Holmes, P. Muller, and S. Walker, Bayesian Nonparametrics. Cambridge, MA: Cambridge Univ. Press, 2010.
34. J. F. C. Kingman, "Completely random measures," Pacific J. Math., vol. 21, no. 1, pp. 59-78, 1967.
35. R. Thibaux and M. I. Jordan, "Hierarchical beta processes and the Indian buffet process," in Proc. Int. Workshop Artificial Intelligence and Statistics, 2007, vol. 11.
36. N. L. Hjort, "Nonparametric Bayes estimators based on beta processes in models for life history data," Ann. Stat., vol. 18, no. 3, pp. 1259-1294, 1990.
37. J. Paisley and L. Carin, "Nonparametric factor analysis with beta process priors," in Proc. Int. Conf. Machine Learning (ICML), 2009.
38. M. Zhou, H. Chen, J. Paisley, L. Ren, G. Sapiro, and L. Carin, "Nonparametric Bayesian dictionary learning for sparse image representations," in Proc. Conf. Advances in Neural Information Processing Systems, NIPS, 2009 .
39. E. Fox, E. Sudderth, M. I. Jordan, and A. Willsky, "Sharing features among dynamical systems with beta processes," in Proc. Advances in Neural Information Processing (NIPS) 22. Cambridge, MA: MIT Press, 2010.
40. M. D. Escobar, "Estimating normal means with a Dirichlet process prior," J. Amer. Stat. Assoc., vol. 89, pp. 268-277, 1994.
41. R. M. Neal, "Markov chain sampling methods for Dirichlet process mixture models," J. Comput. Graph. Stat., vol. 9, pp. 249-265, 2000.
42. H. Ishwaran and L. F. James, "Gibbs sampling methods for stick-breaking priros," J. Amer. Stat. Assoc. Theory Methods, vol. 96, no. 453, pp. 161-173, 2001.
43. S. G. Walker, "Sampling the Dirichlet mixture model with slices," Commun. Stat. Simulat. Comput., vol. 36, p. 45, 2007.
44. Y. W. Teh and M. I. Jordan, "Hierarchical Bayesian nonparametric models with applications," Bayesian Nonparametrics: Principles and Practice. Cambridge: U.K.: Cambridge Univ. Press, 2010.
45. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, "Supervised dictionary learning," in Proc. Conf. Advances in Neural Information Processing Systems, NIPS, 2008.
46. J. Mairal, F. Bach, J. Ponce, and G. Sapiro, "Online dictionary learning for sparse coding," in Proc. Int. Conf. Machine Learning (ICML), 2009.
47. E. Candès, J. Romberg, and T. Tao, "Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information," IEEE Trans. Inform. Theory, vol. 52, pp. 489-509, 2006.
48. E. J. Candès and M. B. Wakin, "An introduction to compressive sampling," IEEE Signal Processing Mag., vol. 25, pp. 21-30, 2008.
49. E. J. Candès, "The restricted isometry property and its implications for compressed sensing," Compte Rendus de l'Academie des Sciences, Paris, Serie I, vol. 346, pp. 589-592, 2008.
50. J. Mairal, M. Elad, and G. Sapiro, "Sparse representation for color image restoration," IEEE Trans. Image Processing, vol. 17, 2008.
51. N. Lawrence and R. Urtasun, "Non-linear matrix factorization with Gaussian processes," in Proc. Int. Conf. Machine Learning, 2009.
52. E. J. Candè s and T. Tao, "The power of convex relaxation: Near-optimal matrix completion," IEEE Trans. Inform. Theory, vol. 56, pp. 2053-2080, 2009.
53. E. J. Candè s and Y. Plan, "Matrix completion with noise," in Proc. IEEE, 2010.
54. S. Goldwater, T. Griffiths, and M. Johnson, "Interpolating between types and tokens by estimating power-law generators," in Proc. Conf. Advances in Neural Information Processing Systems, NIPS, 2006.
55. Y. Teh and D. Gorur, "Indian buffet processes with power-law behavior," in Proc. Conf. Advances in Neural Information Processing Systems, NIPS, 2009.