Direct optimization of the dictionary learning problem

IEEE Transactions on Signal Processing

Volumn 61, Issue 22, 2013, Pages 5495-5506

  Rakotomamonjy, Alain ^a  

^a UNIVERSITÉ DE ROUEN   (France)

Author keywords

Dictionary learning; non convex proximal; one step block coordinate descent

Indexed keywords

ALTERNATING OPTIMIZATIONS; COEFFICIENT MATRIX; DICTIONARY LEARNING; DIRECT OPTIMIZATION; NON-CONVEX PROXIMAL; NONCONVEX OPTIMIZATION; ONE-STEP BLOCK-COORDINATE DESCENT; SPARSE DECOMPOSITION;

ALGORITHMS; CONVEX OPTIMIZATION;

OPTIMIZATION;

EID: 84887055598     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2013.2278158     Document Type: Article

References (26)

1. M. Aharon, E. Elad, and A. Bruckstein, "K-svd: And algorithm for designing overcomplete dictionaries for sparse representations," IEEE Trans. Signal Process., vol. 54, no. 11, pp. 4311-4322, 2006. (Pubitemid 44637761)
2. H. Attouch, J. Bolte, and B. F. Svaiter, "Convergence of descent methods for semi-algebraic and tame problems: Proximal algorithms, forward-backward splitting, and regularized Gauss seidel methods," Math. Program., vol. 137, no. 1-2, pp. 91-129, 2013.
3. A. Beck and M. Teboulle, "A fast iterative shrinkage-thresholding algorithm for linear inverse problems," SIAM J. Imag. Sci., vol. 2, pp. 183-202, 2009.
4. D. Bertsekas, Nonlinear Programming. NewYork,NY, USA: Athena Scientific, 1999.
5. J. Chen and X. Huo, "Sparse representations for multiple measurements vectors (mmv) in an overcomplete dictionary," in Proc IEEE Int. Conf Acoust., Speech Signal Process., 2005, vol. 4, pp. 257-260.
6. P. Combettes and V. Wajs, "Signal recovery by proximal forwardbackward splitting," Multiscale Model. Simul., vol. 4, pp. 1168-1200, 2005. (Pubitemid 44932287)
7. P. L. Combettes and J.-C. Pesquet, "Proximal splitting methods in signal processing," in Fixed-Point Algorithms for Inverse Problems in Science and Engineering, H. H. Bauschke, R. Burachik, P. L. Combettes, V. Elser, D. R. Luke, and H. Wolkowicz, Eds. New York, NY, USA: Springer-Verlag, 2010.
8. S. Cotter, B. Rao, K. Engan, and K. Kreutz-Delgado, "Sparse solutions to linear inverse problems with multiple measurement vectors," IEEE Trans. Signal Process., vol. 53, no. 7, pp. 2477-2488, 2005. (Pubitemid 41174440)
9. W. Dai, T. Xu, and W. Wang, "Simultaneous codeword optimization (simco) for dictionary update and learning," IEEE Trans. Signal Process., vol. 60, no. 12, pp. 6340-6353, 2012.
10. M. Elad and M. Aharon, "Image denoising via sparse and redundant representations over learned dictionaries," IEEE Trans. Image Process., vol. 54, pp. 3736-3745, 2006. (Pubitemid 44811686)
11. K. Engan, S.Aase, and J.Hakon-Husoy, "Method of optimal directions for frame designs," in Proc. Int. Conf. Acoust., Speech Signal Process., 1999.
12. M. Figueiredo, R. Nowak, and S. Wright, "Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems," IEEE J. Sel. Topics Signal Process. (Special Issue on Convex Optimiz. Methods for Signal Process.), vol. 1, no. 4, pp. 586-598, 2007.
13. R. Jenatton, J. Mairal, G. Obozinski, and F. Bach, "Proximal methods for sparse hierarchical dictionary learning," in Proc. Int. Conf. Mach. Learn., 2010.
14. R. Jenatton, G. Obozinski, and F. Bach, "Structured sparse principal component analysis," in Proc. Int. Conf. Artif. Intell. Statist., 2010.
15. K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski, "Dictionary learning algorithms for sparse representation," Neural Computat., vol. 15, pp. 349-396, 2003. (Pubitemid 37049825)
16. H. Lee, A. Battle, R. Raina, and A. Ng, "Efficient sparse coding algorithms," Adv. Neural Inf. Process. Syst., 2007.
17. J.Mairal, F. Bach, J. Ponce, andG. Sapiro, "Online learning for matrix factorization and sparse coding," J. Mach. Learn. Res., vol. 11, pp. 19-60, 2010.
18. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, "Supervised dictionary learning," Proc. Neural Inf. Process. Syst., 2008.
19. B. Olshausen and D. Field, "Sparse coding with an overcomplete basis set: A strategy employed by v1?," Vision Res., vol. 37, pp. 3311-3325, 1997. (Pubitemid 27493805)
20. A. Rakotomamonjy, "Surveying and comparing simultaneous sparse approximation (or group lasso) algorithms," Signal Process., vol. 91, no. 7, pp. 1505-1526, 2011.
21. K. Skretting and K. Engan, "Recursive least squares dictionary learning algorithm," IEEE Trans. Signal Process., vol. 58, no. 4, pp. 2121-2130, 2010.
22. S. Sra, "Nonconvex proximal splitting: Batch and incremental algorithms," Adv. Neural Inf. Process. Syst. (NIPS), 2012.
23. J. Tropp, "Algorithms for simultaneous sparse approximation. Part II: Convex relaxation," Signal Process., vol. 86, pp. 589-602, 2006. (Pubitemid 43106573)
24. P. Tseng, "Convergence of block coordinate descent method for nondifferentiable minimization," J. Optimiz. Theory Appl., vol. 109, pp. 475-494, 2001.
25. M. Yaghoobi, T. Blumensath, and M. Davies, "Dictionary learning for sparse approximations with the majorization method," IEEE Trans. Signal Process., vol. 57, no. 6, pp. 2178-2191, 2009.
26. H. Zou, "The adaptive lasso and its oracle properties," J. Amer. Statist. Assoc., vol. 101, no. 476, pp. 1418-1429, 2006.