New algorithms for learning incoherent and overcomplete dictionaries

Journal of Machine Learning Research

Volumn 35, Issue , 2014, Pages 779-806

  Arora, Sanjeev ^a   Ge, Rong ^b   Moitra, Ankur ^c  

^a Princeton University   (United States)

^b MICROSOFT RESEARCH   (United States)

^c MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE; EDGE DETECTION; LEARNING SYSTEMS; POLYNOMIAL APPROXIMATION; RECOVERY; SIGNAL PROCESSING; STOCHASTIC SYSTEMS;

DICTIONARY LEARNING; INCOHERENT DICTIONARIES; OVER-COMPLETE DICTIONARIES; POLYNOMIAL-TIME ALGORITHMS; SAMPLE COMPLEXITY; SPARSE RECOVERY; SPARSE VECTORS; SUPER RESOLUTION;

LEARNING ALGORITHMS;

EID: 84939615428     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (53)

1. A. Agarwal, A. Anandkumar, P. Jain, P. Netrapalli, and R. Tandon. Learning sparsely used overcomplete dictionaries via alternating minimization. In arxiv:1310.7991, 2013a.
2. A. Agarwal, A. Anandkumar, and P. Netrapalli. Exact recovery of sparsely used overcomplete dictionaries. In arxiv:1309.1952, 2013b.
3. M. Aharon. Overcomplete dictionaries for sparse representation of signals. In PhD Thesis, 2006.
4. M. Aharon, M. Elad, and A. Bruckstein. K-svd: An algorithm for designing overcomplete dictionaries for sparse representation. In IEEE Trans. on Signal Processing, pages 4311-4322, 2006.
5. N. Alon and D. Kleitman. Piercing convex sets and the hadwigder debrunner (p, q)-problem. In Advances in Mathematics, pages 103-112, 1992.
6. A. Anandkumar, D. Foster, D. Hsu, S. Kakade, and Y. Liu. A spectral algorithm for latent dirichlet allocation. In NIPS, pages 926-934, 2012.
7. S. Arora, R. Ge, R. Kannan, and A. Moitra. Computing a nonnegative matrix factorization - provably. In STOC, pages 145-162, 2012a.
8. S. Arora, R. Ge, and A. Moitra. Learning topic models - going beyond svd. In FOCS, pages 1-10, 2012b.
9. S. Arora, R. Ge, S. Sachdeva, and G. Schoenebeck. Finding overlapping communities in social networks: Towards a rigorous approach. In EC, 2012c.
10. M. Balcan, C. Borgs, M. Braverman, J. Chayes, and S-H Teng. Finding endogenously formed communities. In SODA, 2013.
11. Boaz Barak, John Kelner, and David Steurer. Dictionary learning using sum-of-square hierarchy. unpublished manuscript, 2014.
12. M. Belkin and K. Sinha. Polynomial learning of distribution families. In FOCS, pages 103-112, 2010.
13. E. Candes and T. Tao. Decoding by linear programming. In IEEE Trans. on Information Theory, pages 4203-4215, 2005.
14. E. Candes, J. Romberg, and T. Tao. Stable signal recovery from incomplete and inaccurate measurements. In Communications of Pure and Applied Math, pages 1207-1223, 2006.
15. P. Comon. Independent component analysis: A new concept? In Signal Processing, pages 287-314, 1994.
16. G. Davis, S. Mallat, and M. Avellaneda. Greedy adaptive approximations. In J. of Constructive Approximation, pages 57-98, 1997.
17. D. Donoho and M. Elad. Optimally sparse representation in general (non-orthogonal) dictionaries via l1-minimization. In PNAS, pages 2197-2202, 2003.
18. D. Donoho and X. Huo. Uncertainty principles and ideal atomic decomposition. In IEEE Trans. on Information Theory, pages 2845-2862, 1999.
19. D. Donoho and P. Stark. Uncertainty principles and signal recovery. In SIAM J. on Appl. Math, pages 906-931, 1999.
20. M. Elad. Sparse and redundant representations. In Springer, 2010.
21. M. Elad and M. Aharon. Image denoising via sparse and redundant representations over learned dictionaries. In IEEE Trans. on Signal Processing, pages 3736-3745, 2006.
22. K. Engan, S. Aase, and J. Hakon-Husoy. Method of optimal directions for frame design. In ICASSP, pages 2443-2446, 1999.
23. A. Frieze, M. Jerrum, and R. Kannan. Learning linear transformations. In FOCS, pages 359-368, 1996.
24. Q. Geng, H. Wang, and J. Wright. On the local correctness of l1-minimization for dictionary learning. In arxiv:1101.5672, 2013.
25. A. Gilbert, S. Muthukrishnan, and M. Strauss. Approximation of functions over redundant dictionaries using coherence. In SODA, 2003.
26. G. Golub and C. Van Loan. Matrix computations. In The Johns Hopkins University Press, 1996.
27. I. J. Goodfellow, A. Courville, and Y. Bengio. Large-scale feature learning with spike-and-slab sparse coding. In ICML, pages 718-726, 2012.
28. N. Goyal, S. Vempala, and Y. Xiao. Fourier pca. In STOC, 2014.
29. R. Gribonval and M. Nielsen. Sparse representations in unions of bases. In IEEE Transactions on Information Theory, pages 3320-3325, 2003.
30. D. Gross. Recovering low-rank matrices from few coefficients in any basis. In arxiv:0910.1879, 2009.
31. D. Hanson and F. Wright. A bound on tail probabilities for quadratic forms in independent random variables. In Annals of Math. Stat., pages 1079-1083, 1971.
32. M. Hardt. On the provable convergence of alternating minimization for matrix completion. In arxiv:1312.0925, 2013.
33. R. Horn and C. Johnson. Matrix analysis. In Cambridge University Press, 1990.
34. P. Jain, P. Netrapalli, and S. Sanghavi. Low rank matrix completion using alternating minimization. In STOC, pages 665-674, 2013.
35. K. Kavukcuoglu, M. Ranzato, and Y. Le Cun. Fast inference in sparse coding algorithms with applications to object recognition. In NYU Tech Report, 2008.
36. K. Kreutz-Delgado, J. Murray, K. Engan B. Rao, T. Lee, and T. Sejnowski. Dictionary learning algorithms for sparse representation. In Neural Computation, 2003.
37. L. De Lathauwer, J. Castaing, and J. Cardoso. Fourth-order cumulant-based blind identification of underdetermined mixtures. In IEEE Trans. on Signal Processing, pages 2965-2973, 2007.
38. H. Lee, A. Battle, R. Raina, and A. Ng. Efficient sparse coding algorithms. In NIPS, 2006.
39. M. Lewicki and T. Sejnowski. Learning overcomplete representations. In Neural Computation, pages 337-365, 2000.
40. J. Mairal, M. Leordeanu, F. Bach, M. Herbert, and J. Ponce. Discriminative sparse image models for class-specific edge detection and image interpretation. In ECCV, 2008.
41. S. Mallat. A wavelet tour of signal processing. In Academic-Press, 1998.
42. J. Matousek. Lectures on discrete geometry. In Springer, 2002.
43. A. Moitra and G. Valiant. Setting the polynomial learnability of mixtures of gaussians. In FOCS, pages 93-102, 2010.
44. B. Olshausen and B. Field. Sparse coding with an overcomplete basis set: A strategy employed by v1? In Vision Research, pages 3331-3325, 1997.
45. M. Pontil, A. Argyriou, and T. Evgeniou. Multi-task feature learning. In NIPS, 2007.
46. M. Ranzato, Y. Boureau, and Y. Le Cun. Sparse feature learning for deep belief networks. In NIPS, 2007.
47. M. Rudelson. Random vectors in the isotropic position. In J. of Functional Analysis, pages 60-72, 1999.
48. D. Spielman, H. Wang, and J. Wright. Exact recovery of sparsely-used dictionaries. In Journal of Machine Learning Research, 2012.
49. J. Tropp. Greed is good: Algorithmic results for sparse approximation. In IEEE Transactions on Information Theory, pages 2231-2242, 2004.
50. J. Tropp, A. Gilbert, S. Muthukrishnan, and M. Strauss. Improved sparse approximation over quasiincoherent dictionaries. In IEEE International Conf. on Image Processing, 2003.
51. P. Wedin. Perturbation bounds in connection with singular value decompositions. In BIT, pages 99-111, 1972.
52. J. Yang, J. Wright, T. Huong, and Y. Ma. Image super-resolution as sparse representation of raw image patches. In CVPR, 2008.
53. X. Yuan and T. Zhang. Truncated power method for sparse eigenvalue problems. In Journal of Machine Learning Research, pages 899-925, 2013.