Inertial proximal alternating linearized minimization (iPALM) for nonconvex and nonsmooth problems

SIAM Journal on Imaging Sciences

Volumn 9, Issue 4, 2016, Pages 1756-1787

  Pock, Thomas ^a,b   Sabach, Shoham ^c  

^a GRAZ UNIVERSITY OF TECHNOLOGY   (Austria)

^b AIT AUSTRIAN INSTITUTE OF TECHNOLOGY   (Austria)

^c TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Alternating minimization; Blind image deconvolution; Block coordinate descent; Dictionary learning; Heavy ball method; Kurdyka ojasiewicz property; Nonconvex and nonsmooth minimization; Sparse nonnegative matrix factorization

Indexed keywords

FACTORIZATION; LINEARIZATION; NUMERICAL METHODS; OPTIMIZATION;

ALTERNATING MINIMIZATION; BLIND IMAGE DECONVOLUTION; BLOCK COORDINATE DESCENTS; DICTIONARY LEARNING; HEAVY BALL METHOD; NONCONVEX; SPARSE NON-NEGATIVE MATRIX FACTORIZATIONS;

MATRIX ALGEBRA;

EID: 85007346120     PISSN: None     EISSN: 19364954     Source Type: Journal    
DOI: 10.1137/16M1064064     Document Type: Article

References (36)

1. M. Aharon, M. Elad, and A. Bruckstein, K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation, IEEE Trans. Signal Process., 54 (2006), pp. 4311-4322.
2. F. Alvarez and H. Attouch, An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping, Set-Valued Anal., 9 (2001), pp. 3-11.
3. H. Attouch and J. Bolte, On the convergence of the proximal algorithm for nonsmooth functions involving analytic features, Math. Program., 116 (2009), pp. 5-16.
4. H. Attouch, J. Bolte, P. Redont, and A. Soubeyran, Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the Kurdyka-Lojasiewicz inequality, Math. Oper. Res., 35 (2010), pp. 438-457.
5. H. Attouch, J. Bolte, and B. F. Svaiter, Convergence of descent methods for semi-algebraic and tame problems: Proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods, Math. Program., 137 (2013), pp. 91-129.
6. A. Beck and M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci., 2 (2009), pp. 183-202.
7. D. P. Bertsekas and J. N. Tsitsiklis, Parallel and Distributed Computation, Prentice-Hall, Englewood Cliffs, NJ, 1989.
8. J. Bolte, A. Daniilidis, and A. Lewis, The Lojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems, SIAM J. Optim., 17 (2006), pp. 1205-1223.
9. J. Bolte, A. Daniilidis, O. Ley, and L. Mazet, Characterizations of Łojasiewicz inequalities: Subgradient flows, talweg, convexity, Trans. Amer. Math. Soc., 362 (2010), pp. 3319-3363.
10. J. Bolte, S. Sabach, and M. Teboulle, Proximal alternating linearized minimization for nonconvex and nonsmooth problems, Math. Program. Ser. A, 146 (2014), pp. 459-494.
11. P. L. Combettes and V. R. Wajs, Signal recovery by proximal forward-backward splitting, Multiscale Model. Simul., 4 (2005), pp. 1168-1200.
12. Y. Drori and M. Teboulle, Performance of first-order methods for smooth convex minimization: A novel approach, Math. Program., 145 (2014), pp. 451-482.
13. J. Duchi, S. Shalev-Shwartz, Y. Singer, and T.Chandra, Efficient projections onto the l1-ball for learning in high dimensions, in Proceedings of the 25th International Conference on Machine Learning, ICML '08, 2008, pp. 272-279.
14. R. I. Boţ and E. R. Csetnek, An inertial Tseng’s type proximal algorithm for nonsmooth and nonconvex optimization problems, J. Optim. Theory Appl., (2015), pp. 1-17.
15. D. D. Lee and H. S. Seung, Learning the parts of objects by nonnegative matrix factorization, Nature, 401 (1999), pp. 788-791.
16. A. Levin, Y. Weiss, F. Durand, and W. T. Freeman, Understanding and evaluating blind deconvolution algorithms, in Proceedings of Computer Vision and Patter Recognition (CVPR), 2009.
17. P. L. Lions and B. Mercier, Splitting algorithms for the sum of two nonlinear operators, SIAM J. Appl. Math., 16 (1979), pp. 964-979.
18. D. C. Liu and J. Nocedal, On the limited memory BFGS method for large scale optimization, Math. Program., 45 (1989), pp. 503-528.
19. B. S. Mordukhovich, Variational Analysis and Generalized Differentiation: I, Grundlehren Math. Wiss. 330, Springer-Verlag, Berlin, 2006.
20. J. J. Moreau, Proximitéet dualité dans un espace Hilbertien, Bull. Soc. Math. France, 93 (1965), pp. 273-299.
21. Y. Nesterov, Introductory Lectures on Convex Optimization, Appl. Optim. 87, Kluwer Academic, Boston, 2004.
22. 2), Dokl. Akad. Nauk SSSR, 269 (1983), pp. 543-547.
23. J. Nocedal and S. J. Wright, Numerical Optimization, 2nd ed., Springer-Verlag, Berlin, 2006.
24. P. Ochs, Y. Chen, T. Brox, and T. Pock, iPiano: Inertial proximal algorithm for nonconvex optimization, SIAM J. Imaging Sci., 7 (2014), pp. 1388-1419.
25. B. A. Olshausen and D. J. Field, Sparse coding with an overcomplete basis set: A strategy employed by v1?, Vis. Res., 37 (1997), pp. 3311-3325.
26. P. Paatero and U. Tapper, Positive matrix factorization: A nonnegative factor model with optimal utilization of error estimates of data values, Environmetrics, 5 (1994), pp. 111-126.
27. R. Peharz and F. Pernkopf, Sparse nonnegative matrix factorization with l0-constraints, Neurocomputing, 80 (2012), pp. 38-46.
28. D. Perrone, R. Diethelm, and P. Favaro, Blind deconvolution via lower-bounded logarithmic image priors, in Proceedings of the International Conference on Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR), 2015.
29. B. T. Polyak, Some methods of speeding up the convergence of iteration methods, USSR Comput. Math. Math. Phys., 4 (1964), pp. 1-17.
30. B. T. Polyak, Some methods of speeding up the convergence of iteration methods, USSR Comput. Math. Math. Phys., 4 (1964), pp. 1-17.
31. R. T. Rockafellar and R. J.-B. Wets, Variational Analysis, Grundlehren Math. Wiss. 317, Springer-Verlag, Berlin, 1998.
32. F. Samaria and A. Harter, Parameterisation of a stochastic model for human face identification, in WACV, IEEE, 1994, pp. 138-142.
33. S. Sra and I. S. Dhillon, Generalized nonnegative matrix approximations with Bregman divergences, in Advances in Neural Information Processing Systems 18, Y. Weiss, B. Schölkopf, and J. C. Platt, eds., MIT Press, Cambridge, MA, 2006, pp. 283-290.
34. Y. Xu and W. Yin, A Globally Convergent Algorithm for Nonconvex Optimization Based on Block Coordinate Update, Technical report, preprint, arXiv: 1410: 1386, 2014.
35. S. K. Zavriev and F. V. Kostyuk, Heavy-ball method in nonconvex optimization problems, Comput. Math. Model., 4 (1993), pp. 336-341.
36. M. Zeiler, D. Krishnan, G. Taylor, and R. Fergus, Deconvolutional networks, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010, pp. 2528-2535.