On the identifiability of overcomplete dictionaries via the minimisation principle underlying K-SVD

Applied and Computational Harmonic Analysis

Volumn 37, Issue 3, 2014, Pages 464-491

  Schnass, Karin ^a  

^a UNIVERSITY OF SASSARI   (Italy)

Author keywords

Dictionary identification; Dictionary learning; Finite sample size; K SVD; Minimisation criterion; Sampling complexity; Sparse coding; Sparse component analysis; Sparse representation

Indexed keywords

ASYMPTOTIC ANALYSIS; SAMPLING;

DICTIONARY LEARNING; FINITE SAMPLE SIZES; MINIMISATION; SPARSE CODING; SPARSE COMPONENT ANALYSIS; SPARSE REPRESENTATION;

LEARNING ALGORITHMS;

EID: 84908556147     PISSN: 10635203     EISSN: 1096603X     Source Type: Journal    
DOI: 10.1016/j.acha.2014.01.005     Document Type: Article

References (43)

1. A. Agarwal, A. Anandkumar, P. Jain, P. Netrapalli, R. Tandon, Learning sparsely used overcomplete dictionaries via alternating minimization, arXiv:1310.7991, 2013.
2. A. Agarwal, A. Anandkumar, P. Netrapalli, Exact recovery of sparsely used overcomplete dictionaries, arXiv:1309.1952, 2013.
3. M. Aharon, M. Elad, A.M. Bruckstein, K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation, IEEE Trans. Signal Process. 54 (11) (November 2006) 4311-4322.
4. M. Aharon, M. Elad, A.M. Bruckstein, On the uniqueness of overcomplete dictionaries, and a practical way to retrieve them, Linear Algebra Appl. 416 (July 2006) 48-67.
5. S. Arora, R. Ge, A. Moitra, New algorithms for learning incoherent and overcomplete dictionaries, arXiv:1308.6273, 2013.
6. T. Blumensath, M.E. Davies, Iterative thresholding for sparse approximations, J. Fourier Anal. Appl. 14 (5-6) (2008) 629-654.
7. E. Candès, L. Demanet, D.L. Donoho, L. Ying, Fast discrete curvelet transforms, Multiscale Model. Simul. 5 (3) (2006) 861-899.
8. E. Candès, J. Romberg, T. Tao, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inform. Theory 52 (2) (2006) 489-509.
9. S.S. Chen, D.L. Donoho, M.A. Saunders, Atomic decomposition by basis pursuit, SIAM J. Sci. Comput. (1998).
10. O. Christensen, An Introduction to Frames and Riesz Bases, Birkhäuser, 2003.
11. I. Daubechies, Ten Lectures on Wavelets, CBMS-NSF Lecture Notes, SIAM, 1992.
12. I. Daubechies, R.A. DeVore, M. Fornasier, S. Güntürk, Iteratively reweighted least squares minimization for sparse recovery, Comm. Pure Appl. Math. 63 (1) (January 2010) 1-38.
13. G. Davis, S. Mallat, M. Avellaneda, Adaptive greedy approximations, in: Constr. Approx., vol. 13, Springer-Verlag New York Inc., 1997, pp. 57-98.
14. D.L. Donoho, Compressed sensing, IEEE Trans. Inform. Theory 52 (4) (2006) 1289-1306.
15. D.L. Donoho, M. Elad, V.N. Temlyakov, Stable recovery of sparse overcomplete representations in the presence of noise, IEEE Trans. Inform. Theory 52 (1) (January 2006) 6-18.
16. D.J. Field, B.A. Olshausen, Emergence of simple-cell receptive field properties by learning a sparse code for natural images, Nature 381 (1996) 607-609.
17. 1-minimization for dictionary learning, arXiv:1101.5672, 2011.
18. P. Georgiev, F.J. Theis, A. Cichocki, Sparse component analysis and blind source separation of underdetermined mixtures, IEEE Trans. Neural Netw. 16 (4) (2005) 992-996.
19. R. Gribonval, R. Jenatton, F. Bach, M. Kleinsteuber, M. Seibert, Sample complexity of dictionary learning and other matrix factorizations, arXiv:1312.3790, 2013.
20. R. Gribonval, H. Rauhut, K. Schnass, P. Vandergheynst, Atoms of all channels, unite! Average case analysis of multi-channel sparse recovery using greedy algorithms, J. Fourier Anal. Appl. 14 (5) (2008) 655-687.
21. 1-minimisation, IEEE Trans. Inform. Theory 56 (7) (July 2010) 3523-3539.
22. R. Jenatton, F. Bach, R. Gribonval, Local stability and robustness of sparse dictionary learning in the presence of noise, preprint, 2012.
23. K. Kreutz-Delgado, J.F. Murray, B.D. Rao, K. Engan, T. Lee, T.J. Sejnowski, Dictionary learning algorithms for sparse representation, Neural Comput. 15 (2) (2003) 349-396.
24. K. Kreutz-Delgado, B.D. Rao, FOCUSS-based dictionary learning algorithms, in: Proc. SPIE, vol. 4119, 2000.
25. M. Ledoux, M. Talagrand, Probability in Banach Spaces. Isoperimetry and Processes, Springer-Verlag, Berlin, Heidelberg, New York, 1991.
26. J. Mairal, F. Bach, J. Ponce, G. Sapiro, Online learning for matrix factorization and sparse coding, J. Mach. Learn. Res. 11 (2010) 19-60.
27. A. Maurer, M. Pontil, K-dimensional coding schemes in Hilbert spaces, IEEE Trans. Inform. Theory 56 (11) (2010) 5839-5846.
28. N.A. Mehta, A.G. Gray, On the sample complexity of predictive sparse coding, arXiv:1202.4050, 2012.
29. D. Needell, J.A. Tropp, CoSaMP: Iterative signal recovery from incomplete and inaccurate samples, Appl. Comput. Harmon. Anal. 26 (3) (2009) 301-321.
30. 1-exact sparse coding, in: M.E. Davies, C.J. James, S.A. Abdallah (Eds.), International Conference on Independent Component Analysis and Signal Separation, vol. 4666, Springer, 2007, pp. 406-413.
31. DSP Rice University, Compressive sensing resources, http://www.compressedsensing.com/.
32. R. Rubinstein, A. Bruckstein, M. Elad, Dictionaries for sparse representation modeling, Proc. IEEE 98 (6) (2010) 1045-1057.
33. K. Schnass, Dictionary identification results for K-SVD with sparsity parameter 1, in: SampTA13, 2013.
34. K. Schnass, On the identifiability of overcomplete dictionaries via the minimisation principle underlying K-SVD, arXiv:1301.3375, 2013.
35. K. Schnass, P. Vandergheynst, Average performance analysis for thresholding, IEEE Signal Process. Lett. 14 (11) (2007) 828-831.
36. K. Skretting, K. Engan, Recursive least squares dictionary learning algorithm, IEEE Trans. Signal Process. 58 (4) (April 2010) 2121-2130.
37. D. Spielman, H. Wang, J. Wright, Exact recovery of sparsely-used dictionaries, in: Conference on Learning Theory, 2012, arXiv:1206.5882.
38. J.A. Tropp, Greed is good: Algorithmic results for sparse approximation, IEEE Trans. Inform. Theory 50 (10) (October 2004) 2231-2242.
39. J.A. Tropp, On the conditioning of random subdictionaries, Appl. Comput. Harmon. Anal. 25 (2008) 1-24.
40. D. Vainsencher, S. Mannor, A.M. Bruckstein, The sample complexity of dictionary learning, J. Mach. Learn. Res. 12 (2011) 3259-3281.
41. R. Vershynin, Introduction to the non-asymptotic analysis of random matrices, in: Y. Eldar, G. Kutyniok (Eds.), Compressed Sensing, Theory and Applications, Cambridge University Press, 2012, Chapter 5.
42. M. Yaghoobi, T. Blumensath, M.E. Davies, Dictionary learning for sparse approximations with the majorization method, IEEE Trans. Signal Process. 57 (6) (June 2009) 2178-2191.
43. M. Zibulevsky, B.A. Pearlmutter, Blind source separation by sparse decomposition in a signal dictionary, Neural Comput. 13 (4) (2001) 863-882.