The sample complexity of dictionary learning

Journal of Machine Learning Research

Volumn 12, Issue , 2011, Pages 3259-3281

  Vainsencher, Daniel ^a   Mannor, Shie ^a   Bruckstein, Alfred M ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Dictionary learning; Generalization bound; Sparse representation

Indexed keywords

COEFFICIENT VECTOR; DE-NOISING; DICTIONARY LEARNING; GENERALIZATION BOUND; HIGH DIMENSIONS; L1 NORM; LINEAR COMBINATIONS; NUMBER OF SAMPLES; ORTHOGONALITY; SAMPLE COMPLEXITY; SIGNAL PROCESSING APPLICATIONS; SIGNAL SEPARATION; SPARSE REPRESENTATION; STATISTICAL LEARNING THEORY;

ARTIFICIAL INTELLIGENCE; SOFTWARE ENGINEERING;

SIGNAL PROCESSING;

EID: 84855384338     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (30)

1. Michal Aharon, Michael Elad, and Alfred M. Bruckstein. K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing, 54 (11):4311-4322, 2006. (Pubitemid 44637761)
2. Martin Anthony and Peter L. Bartlett. Neural Network Learning: Theoretical Foundations. Cambridge University Press, 1999.
3. Peter L. Bartlett, Tamás Linder, and Gábor Lugosi. The minimax distortion redundancy in empirical quantizer design. IEEE Transactions on Information Theory, 44(5):1802-1813, 1998. (Pubitemid 128738552)
4. Peter L. Bartlett, Olivier Bousquet, and Shahar Mendelson. Local Rademacher complexities. Ann. Statist., 33:1497-1537, 2005. (Pubitemid 41423979)
5. Gérard Biau, Luc Devroye, and Gábor Lugosi. On the performance of clustering in Hilbert spaces. IEEE Transactions on Information Theory, 54(2):781-790, 2008. (Pubitemid 351314665)
6. Gilles Blanchard, Olivier Bousquet, and Laurent Zwald. Statistical properties of kernel principal component analysis. Machine Learning, 66(2):259-294, 2007. (Pubitemid 46360614)
7. Alfred M. Bruckstein, David L. Donoho, and Michael Elad. From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Review, 51(1):34-81, 2009.
8. Emmanuel J. Candes and Terence Tao. Near-optimal signal recovery from random projections: Universal encoding strategies? IEEE Transactions on Information Theory, 52(12):5406-5425, 2006. (Pubitemid 46445381)
9. Scott S. Chen, David L. Donoho, andMichael A. Saunders. Atomic decomposition by basis pursuit. SIAM Review, 43(1):129-159, 2001. (Pubitemid 32406896)
10. Sheng Chen, Stephen A. Billings, and Wan Luo. Orthogonal least squares methods and their application to non-linear system identification. International Journal of Control, 50(5):1873-1896, 1989. (Pubitemid 20636409)
11. Felipe Cucker and Stephen Smale. On the mathematical foundations of learning. Bull. Amer. Math. Soc, 39(1):1-50, 2002.
12. Geoff Davis, Stèphane Mallat, and Marco Avellaneda. Adaptive greedy approximations. Constructive approximation, 13(1):57-98, 1997.
13. David L. Donoho and Michael Elad. Optimally sparse representation in general (nonorthogonal) dictionaries via 1 minimization. Proceedings of the National Academy of Sciences, 100(5): 2197-2202, 2003. (Pubitemid 36297476)
14. Richard J. Duffin and Albert C. Schaeer. A class of nonharmonic Fourier series. Trans. Amer. Math. Soc, 72:341-366, 1952.
15. Ralf Herbrich and Robert Williamson. Algorithmic luckiness. Journal of Machine Learning Research, 3:175-212, 2003.
16. Roger A. Horn and Charles R. Johnson. Matrix Analysis. Cambridge University Press, 1990.
17. Kenneth Kreutz-Delgado, Joseph F. Murray, Bhaskar D. Rao, Kjersti Engan, Te-Won Lee, and Terrance J. Sejnowski. Dictionary learning algorithms for sparse representation. Neural Computation, 15(2):349-396, 2003. (Pubitemid 37049825)
18. Honglak Lee, Alexis Battle, Rajat Raina, and Andrew Y. Ng. Efficient sparse coding algorithms. In Advances in Neural Information Processing Systems 19, pages 801-808. MIT Press, Cambridge, MA, 2007.
19. Paul Lévy. Problèmes concrets d'analyse fonctionnelle. Gauthier-Villars, Paris, 1951.
20. Michael S. Lewicki, Terrence J. Sejnowski, and Howard Hughes. Learning overcomplete representations. Neural Computation, 12:337-365, 1998.
21. Julien Mairal, Francis Bach, Jean Ponce, and Guillermo Sapiro. Online learning for matrix factorization and sparse coding. Journal of Machine Learning Research, 11:19-60, 2010.
22. Andreas Maurer and Massimiliano Pontil. K-dimensional coding schemes in hilbert spaces. IEEE Transactions on Information Theory, 56:5839-5846, November 2010.
23. Shahar Mendelson. A few notes on statistical learning theory. Advanced Lectures on Machine Learning, pages 1-40, 2003.
24. Bruno A. Olshausen and David J. Field. Sparse coding with an overcomplete basis set: A strategy employed by V1. Vision Research, 37:3311-3325, 1997. (Pubitemid 27493805)
25. Gabriel Peyré. Sparse modeling of textures. Journal of Mathematical Imaging and Vision, 34(1): 17-31, 2009.
26. Matan Protter and Michael Elad. Sparse and redundant representations and motion-estimation-free algorithm for video denoising. Wavelets XII. Proceedings of the SPIE, 6701:43, 2007.
27. John Shawe-Taylor and Nello Cristianini. Kernel Methods for Pattern Analysis. Cambridge University Press, 2004.
28. John Shawe-Taylor, Christopher K. I. Williams, Nello Cristianini, and Jaz Kandola. On the eigenspectrum of the Gram matrix and the generalization error of kernel-PCA. IEEE Transactions on Information Theory, 51(7):2510-2522, 2005. (Pubitemid 41136405)
29. Joel A. Tropp. Greed is good: Algorithmic results for sparse approximation. IEEE Transactions on Information Theory, 50:2231-2242, 2004.
30. JohnWright, Allen Y. Yang, Arvind Ganesh, S. Shankar Sastry, and YiMa. Robust face recognition via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 210-227, 2008.