Applying alternating direction method of multipliers for constrained dictionary learning

Neurocomputing

Volumn 106, Issue , 2013, Pages 126-136

  Rakotomamonjy, A ^a  

^a UNIVERSITÉ DE ROUEN   (France)

Author keywords

Alternate direction method of multipliers; Dictionary learning; Mixed constraints; Sparse coding

Indexed keywords

ALTERNATING DIRECTION METHOD OF MULTIPLIERS; ALTERNATING OPTIMIZATIONS; APPROXIMATION COEFFICIENTS; DICTIONARY LEARNING; ITERATIVE SHRINKAGES; METHOD OF MULTIPLIERS; MIXED CONSTRAINTS; SPARSE CODING;

APPROXIMATION ALGORITHMS; ATOMS;

ITERATIVE METHODS;

ARTICLE; CALCULATION; DATA ANALYSIS; IMAGE ANALYSIS; IMAGE PROCESSING; LEARNING ALGORITHM; MATHEMATICAL COMPUTING; PRIORITY JOURNAL; PROCESS OPTIMIZATION;

EID: 84875404240     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2012.10.024     Document Type: Article

References (36)

1. Afonso M., Bioucas-Dias J., Figueiredo M. Fast image recovery using variable splitting and constrained optimization. IEEE Trans. Image Process. 2010, 19(9):2345-2356.
2. Aharon M., Elad E., Bruckstein A. K-svd. and algorithm for designing overcomplete dictionaries for sparse representations. IEEE Trans. Signal Process. 2006, 54(11):4311-4322.
3. Beck A., Teboulle M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2009, 2:183-202.
4. Boyd S., Parikh N., Chu E., Peleato B., Eckstein J. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 2011, 3:1-122. URL 〈http://www.stanford.edu/~boyd/papers/distr_opt_stat_learning_admm.html〉.
5. Pesquet J.-P., Pustelnik N. Nested iterative algorithms for convex constrained image recovery problems. SIAM J. Imaging Sci. 2009, 2(2):730-762.
6. J. Chen, X. Huo, Sparse representations for multiple measurements vectors (mmv) in an overcomplete dictionary, in: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 4, 2005, pp. 257-260.
7. Combettes P., Pesquet J.-C. A Douglas-Rachford splitting approach to nonsmooth convex variational signal recovery. IEEE J. Sel. Top. Signal Process. 2007, 1:564-574.
8. Combettes P., Wajs V. Signal recovery by proximal forward-backward splitting. Multiscale Modeling and Simulation 2005, 4:1168-1200.
9. Combettes P.L., Pesquet J.-C. Proximal splitting methods in signal processing. Fixed-Point Algorithms for Inverse Problems in Science and Engineering 2010, Springer-Verlag. H.H. Bauschke, R. Burachik, P.L. Combettes, V. Elser, D.R. Luke, H. Wolkowicz (Eds.).
10. Cotter S., Rao B., Engan K., Kreutz-Delgado K. Sparse solutions to linear inverse problems with multiple measurement vectors. IEEE Trans. Signal Process. 2005, 53(7):2477-2488.
11. Eckstein J., Bertsekas D. On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Programming 1992, 5:293-318.
12. Elad M., Aharon M. Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Process. 2006, 54:3736-3745.
13. K. Engan, S. Aase, J. Hakon-Husoy, Method of optimal directions for frame designs, in: Proceedings of the International Conference on Acoustics, Speech and Signal Processing, 1999.
14. E. Esser, Applications of Lagrangian-based Alternating Direction Methods and Connections to Split Bregman, Technical Report, University of California, Irvine, 2009.
15. Figueiredo M., Nowak R., Wright S. Gradient projection for sparse reconstruction. application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Signal Process. Special Issue on Convex Optim. Methods Signal Process. 2007, 1(4):586-598.
16. Fornasier M., Rauhut H. Iterative thresholding algorithms. Appl. Comput. Harmonic Anal. 2008, 25(2):187-208.
17. Fornasier M., Rauhut H. Recovery algorithms for vector valued data with joint sparsity constraints. SIAM J. Numer. Anal. 2008, 46(2):577-613.
18. R. Jenatton, J. Mairal, G. Obozinski, F. Bach, Proximal methods for sparse hierarchical dictionary learning, in: Proceedings of International Conference on Machine Learning, 2010.
19. R. Jenatton, G. Obozinski, F. Bach, Structured sparse principal component analysis, in: Proceedings of the International Conference on Artificial Intelligence and Statistics, 2010.
20. Kreutz-Delgado K., Murray J., Rao B., Engan K., SejnowskiLee T. Dictionary learning algorithms for sparse representation. Neural Comput. 2003, 15:349-396.
21. Labusch K., Barth E., Martinetz T. Soft-competitive learning of sparse codes and its application to image reconstruction. Neurocomputing 2011, 74(9):1418-1428.
22. H. Lee, A. Battle, R. Raina, A. Ng, Efficient sparse coding algorithms, in: Advances in Neural Information Processing Systems, 2007.
23. P. Machart, S. Anthoine, L. Baldassare, Optimal Computational Trade-off of Inexact Proximal Methods, Technical Report, Aix-Marseille University, 2012.
24. Mairal J., Bach F., Ponce J., Sapiro G. Online learning for matrix factorization and sparse coding. J. Mach. Learn. Res. 2010, 11:19-60.
25. J. Mairal, F. Bach, J. Ponce, G. Sapiro, A. Zisserman, Supervised dictionary learning, in: Proceedings of Neural Information Processing Systems, 2008.
26. Moreau J.-J. Proximité et dualité dans un espace hilbertien. Bull. Soc. Math. France 1965, 93:273-299.
27. Nocedal J., Wright S. Numerical Optimization 2000, Springer.
28. Pustelnik N., Chaux C., Pesquet J.-C. Parallel proximal algorithm for image restoration using hybrid regularization. IEEE Trans. Image Process. 2011, 20(9):2450-2462.
29. Rakotomamonjy A. Surveying and comparing simultaneous sparse approximation (or group Lasso) algorithms. Signal Process. 2011, 91(7):1505-1526.
30. Skretting K., Engan K. Recursive least squares dictionary learning algorithm. IEEE Trans. Signal Process. 2010, 58(4):2121-2130.
31. Tropp J. Algorithms for simultaneous sparse approximation. Part II. convex relaxation. Signal Process. 2006, 86:589-602.
32. Tseng P. Convergence of block coordinate descent method for nondifferentiable minimization. J. Optim. Theory Appl. 2001, 109:475-494.
33. Yaghoobi M., Blumensath T., Davies M. Dictionary learning for sparse approximations with the majorization method. IEEE Trans. Signal Process. 2009, 57(6):2178-2191.
34. Yang S., Liu Z., Wang M., Sun F., Jiao L. Multitask dictionary learning and sparse representation based single-image super-resolution reconstruction. Neurocomputing 2011, 74(17):3193-3203.
35. Q. Zhang, B. Li, Discriminative K-SVD for dictionary learning in face recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2010.
36. Zou H. The adaptive Lasso and its oracle properties. J. Am. Stat. Assoc. 2006, 101(476):1418-1429.