Dictionary learning for sparse representation: Complexity and algorithms

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2014, Pages 5247-5251

  Razaviyayn, Meisam ^a   Tseng, Hung Wei ^a   Luo, Zhi Quan ^a  

^a University of Minnesota   (United States)

Author keywords

computational complexity; Dictionary learning; K SVD; sparse representation

Indexed keywords

COMPUTATIONAL COMPLEXITY; SIGNAL PROCESSING;

DICTIONARY LEARNING; K-SVD; K-SVD ALGORITHMS; NP-HARD; SPARSE REPRESENTATION; STATIONARY POINTS;

LEARNING ALGORITHMS;

EID: 84905234331     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2014.6854604     Document Type: Conference Paper

References (32)

1. V. A. KotelŠnikov, "On the carrying capacity of the ether and wire in telecommunications," in Material for the First All-Union Conference on Questions of Communication, Izd. Red. Upr. Svyazi RKKA, Moscow, 1933.
2. H. Nyquist, "Certain topics in telegraph transmission theory," Transactions of the American Institute of Electrical Engineers, vol. 47, no. 2, pp. 617-644, 1928.
3. C. E. Shannon, "Communication in the presence of noise," Proceedings of the IRE, vol. 37, no. 1, pp. 10-21, 1949.
4. E. T. Whittaker, On the functions which are represented by the expansions of the interpolation-theory, Edinburgh University, 1915.
5. R. Prony, "Essai experimental et analytique sur les lois de la dilatabilite des fluides elastiques et sur celles de la force expansive de la vapeur de leau et de la vapeur de lalkool, r differentes temperatures," Journal Polytechnique ou Bulletin du Travail fait r Lecole Centrale des Travaux Publics, Paris, Premier Cahier, pp. 24-76, 1995.
6. D. L. Donoho, "Compressed sensing," IEEE Transactions on Information Theory, vol. 52, no. 4, pp. 1289-1306, 2006.
7. E. J. Candès, J. Romberg, and T. Tao, "Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information," IEEE Transactions on Information Theory, vol. 52, no. 2, pp. 489-509, 2006.
8. M. Lustig, D. L. Donoho, and J. M. Pauly, "Rapid mr imaging with compressed sensing and randomly under-sampled 3dft trajectories," in Proceedings of 14th Annual Meeting of ISMRM. Citeseer, 2006.
9. M. Lustig, J. H. Lee, D. L. Donoho, and J. M. Pauly, "Faster imaging with randomly perturbed, under-sampled spirals and l1 reconstruction," in Proceedings of the 13th Annual Meeting of ISMRM, Miami Beach, 2005, p. 685.
10. K. Gedalyahu and Y. C. Eldar, "Time-delay estimation from low-rate samples: A union of subspaces approach," IEEE Transactions on Signal Processing, vol. 58, no. 6, pp. 3017-3031, 2010.
11. M. Mishali and Y. C. Eldar, "Blind multiband signal reconstruction: Compressed sensing for analog signals," IEEE Transactions on Signal Processing, vol. 57, no. 3, pp. 993-1009, 2009.
12. M. F. Duarte, M. A. Davenport, D. Takhar, J. N. Laska, T. Sun, K. F. Kelly, and R. G. Baraniuk, "Single-pixel imaging via compressive sampling," IEEE Signal Processing Magazine, vol. 25, no. 2, pp. 83-91, 2008.
13. R. F. Marcia, Z. T. Harmany, and R. M. Willett, "Compressive coded aperture imaging," in IS&T/SPIE Electronic Imaging. International Society for Optics and Photonics, 2009, pp. 72460G-72460G.
14. M. F. Duarte, S. Sarvotham, D. Baron, M. B. Wakin, and R. G. Baraniuk, "Distributed compressed sensing of jointly sparse signals," in Asilomar Conference on Signals, Systems, and Computing, 2005, pp. 1537-1541.
15. M. A. Davenport, C. Hegde, M. F. Duarte, and R. G. Baraniuk, "Joint manifolds for data fusion," IEEE Transactions on Image Processing, vol. 19, no. 10, pp. 2580-2594, 2010.
16. H. Lee, A. Battle, R. Raina, and A. Ng, "Efficient sparse coding algorithms," in Advances in neural information processing systems, 2006, pp. 801-808.
17. K. Kreutz-Delgado, J. F. Murray, B. D. Rao, K. Engan, T.-W. Lee, and T. J. Sejnowski, "Dictionary learning algorithms for sparse representation," Neural computation, vol. 15, no. 2, pp. 349-396, 2003.
18. M. Elad and M. Aharon, "Image denoising via sparse and redundant representations over learned dictionaries," IEEE Transactions on Image Processing, vol. 15, no. 12, pp. 3736-3745, 2006.
19. M. S. Lewicki and T. J. Sejnowski, "Learning overcomplete representations," Neural computation, vol. 12, no. 2, pp. 337-365, 2000.
20. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, "Supervised dictionary learning," arXiv preprint arXiv:0809.3083, 2008.
21. R. Rubinstein, A. M. Bruckstein, and M. Elad, "Dictionaries for sparse representation modeling," Proceedings of the IEEE, vol. 98, no. 6, pp. 1045-1057, 2010.
22. M. Aharon, M. Elad, and A. Bruckstein, "K-svd: Design of dictionaries for sparse representation," Proceedings of SPARS, vol. 5, pp. 9-12, 2005.
23. K. Engan, S. O. Aase, and J. Hakon Husoy, "Method of optimal directions for frame design," in Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing. IEEE, 1999, vol. 5, pp. 2443-2446.
24. B. A. Olshausen and D. J. Field, "Sparse coding with an overcomplete basis set: A strategy employed by v1?," Vision research, vol. 37, no. 23, pp. 3311-3325, 1997.
25. Z. Jiang, G. Zhang, and L. S. Davis, "Submodular dictionary learning for sparse coding," in IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2012, pp. 3418-3425.
26. B. K. Natarajan, "Sparse approximate solutions to linear systems," SIAM journal on computing, vol. 24, no. 2, pp. 227-234, 1995.
27. M. Razaviyayn, M. Hong, and Z.-Q. Luo, "A unified convergence analysis of block successive minimization methods for nonsmooth optimization," SIAM Journal on Optimization, vol. 23, no. 2, pp. 1126-1153, 2013.
28. P. O. Hoyer, "Non-negative matrix factorization with sparseness constraints," The Journal of Machine Learning Research, vol. 5, pp. 1457-1469, 2004.
29. V. K. Potluru, S. M. Plis, J. L. Roux, B. A. Pearlmutter, V. D. Calhoun, and T. P. Hayes, "Block coordinate descent for sparse nmf," arXiv preprint arXiv:1301.3527, 2013.
30. J. Kim and H. Park, "Sparse nonnegative matrix factorization for clustering," 2008.
31. D. P. Bertsekas, "Nonlinear programming," 1999.
32. L. J. Hong, Y. Yang, and L. Zhang, "Sequential convex approximations to joint chance constrained programs: A monte carlo approach," Operations Research, vol. 59, no. 3, pp. 617-630, 2011.