On the Linear Convergence of the Approximate Proximal Splitting Method for Non-smooth Convex Optimization

Journal of the Operations Research Society of China

Volumn 2, Issue 2, 2014, Pages 123-141

  Kadkhodaie, Mojtaba ^a   Sanjabi, Maziar ^a   Luo, Zhi Quan ^a  

^a UNIVERSITY OF MINNESOTA   (United States)

Author keywords

Block coordinate descent method; Convergence rate analysis; Convex optimization; Local error bound; Proximal splitting method

Indexed keywords

CONVEX OPTIMIZATION; FUNCTIONS; REGRESSION ANALYSIS;

APPROXIMATE GRADIENT; BLOCK COORDINATE DESCENTS; COMPRESSIVE SENSING; CONVERGENCE RATES; LOCAL ERROR BOUND; MINIMIZATION PROBLEMS; NON-SMOOTH CONVEX OPTIMIZATIONS; SPLITTING METHOD;

PROBLEM SOLVING;

EID: 84903559962     PISSN: 2194668X     EISSN: 21946698     Source Type: Journal    
DOI: 10.1007/s40305-014-0047-x     Document Type: Article

References (36)

1. Agarwal, A., Negahban, S., Wainwright, M.: Fast global convergence of gradient methods for high-dimensional statistical recovery. Ann. Stat. 40, 2452-2482 (2012).
2. Bach, F.: Consistency of the group lasso and multiple kernel learning. J. Mach. Learn. Res. 9, 1179-1225 (2008).
3. Berg, E. V. D., Schmidt, M., Friedlander, M., Murphy, K.: Group sparsity via linear-time projection, Technical Report TR-2008-09. Department of Computer Science, University of British Columbia, 2008.
4. Combettes, P. L., Wajs, V. R.: Signal recovery by proximal forward-backward splitting. Multiscale Model. Simul. 4, 1168-1200 (2005).
5. Dembo, R. S., Tulowizki, U.: Local convergence analysis for successive inexact quadratic programming methods, Working Paper. School of Organization and Management, Yale University, New Haven, (1984).
6. Hong, M., Wang, X., Razaviyayn, M., Luo, Z.-Q.: Iteration complexity analysis of block coordinate descent methods. arXiv preprint, http://arxiv. org/abs/1310. 6957arXiv: 1310. 6957 (2013).
7. Kim, D., Sra, S., Dhillon, I.: A scalable trust-region algorithm with applications to mixed-norm regression. International Conference of Machine Learning (ICML), vol. 1, 2010.
8. Korpelevich, G. M.: The extra-gradient method for finding saddle points and other problems. Ekon. i Mat Metody 12, 747-756 (1976). Translated to English as Matecon.
9. Li, W.: Remarks on the convergence of the matrix splitting algorithm for the symmetric linear complementarity problem. SIAM J. Optim. 3, 155-163 (1993).
10. Liu, J., Ji, S., Ye, J.: Sparse learning with efficient projections. Arizona State University, (2009).
11. Luo, Z.-Q., Tseng, P.: Error bound and convergence analysis of matrix splitting algorithm for the affine variational inequality problem. SIAM J. Optim. 2, 43-54 (1992).
12. Luo, Z.-Q., Tseng, P.: On the linear convergence of descent methods for convex essentially smooth minimization. SIAM J. Control Optim. 3, 408-425 (1992).
13. Luo, Z.-Q., Tseng, P.: Error bounds and convergence analysis of feasible descent methods: a general approach. Ann. Oper. Res. 46, 157-178 (1993).
14. Luo, Z.-Q., Tseng, P.: Analysis of an approximate gradient projection method with applications to the back-propagation algorithm. Optim. Methods Softw. 4, 85-101 (1994).
15. Ma, S., Song, X., Huang, J.: Supervised group lasso with applications to microarray data analysis. BMC Bioinform. 8, 60 (2007).
16. Mairal, J.: Incremental majorization-minimization optimization with application to large-scale machine learning. arXiv preprint, http://arxiv. org/abs/1402. 4419 (2014).
17. Mangasarian, O. L.: Convergence of iterates of an inexact matrix splitting algorithm for the symmetric monotone linear complementarity problem. SIAM J. Optim. 1, 114-122 (1991).
18. Meier, L., Geer, S. V. D., Buhlmann, P.: The group lasso for logistic regression. J. R. Stat. Soc. B 70, 53-71 (2008).
19. Necoara, I., Clipici, D.: Distributed random coordinate descent method for composite minimization. arXiv preprint http://arxiv. org/abs/1312. 5302 (2013).
20. Pang, J.-S.: On the convergence of the basic iterative method for the implicit complementarity problem. J. Optim. Theory Appl. 37, 149-162 (1982).
21. Pang, J.-S.: Inexact newton methods for nonlinear complementarity problem. Math. Prog. 36(1), 54-71 (1986).
22. Pang, J.-S.: A posteriori error bounds for the linearly-constrained variational inequality problem. Math. Oper. Res. 12(3), 474-484 (1987).
23. Razaviyayn, M., Hong, M., Luo, Z.-Q.: A unified convergence analysis of block successive minimization methods for nonsmooth optimization. SIAM J. Optim. 23, 1126-1153 (2013).
24. Rockafellar, R. T.: Convex Analysis. Princeton University Press, Princeton (1970).
25. Rosen, J. B.: The gradient projection method for non-linear programming, part I linear constraints. J. Soc. Ind. Appl. Math. 8, 181-217 (1960).
26. Rosen, J. B.: The gradient projection method for non-linear programming, part II non-linear constraints. J. Soc. Ind. Appl. Math. 9, 514-532 (1961).
27. Roth, V., Fischer, B.: The group-lasso for generalized linear models: uniqueness of solutions and efficient algorithms. In: Proceedings of the 25th International Conference on Machine Learning, ACM, 2008, pp. 848-855.
28. Schmidt, M., Roux, N., Bach, F.: Convergence rates of inexact proximal-gradient methods for convex optimization. arxiv preprint, http://arxiv. org/abs/1109. 2415 (2011).
29. Tibshirani, R.: Regression shrinkage and selection via lasso. J. R. Stat. Soc. B 58, 267-288 (1996).
30. Tseng, P.: Convergence of block coordinate method for non-differentiable minimization. J. Optim. Theory Apppl. 109, 475-494 (2001).
31. Tseng, P.: Approximation accuracy, gradient methods, and error bound for structured convex optimization. Math. Program. 125(2), 263-295 (2009).
32. Tseng, P., Yun, S.: A coordinate gradient descent method for nonsmooth separable minimization. Math. Program. 117, 387-423 (2009).
33. Wright, S., Nowak, R., Figueiredo, M.: Sparse reconstruction by separable approximation. IEEE Trans. Signal Process. 57, 2479-2493 (2009).
34. Yang, J., Sun, D., Toh, K.-C.: A proximal point algorithm for log-determinant optimization with group lasso regularization. SIAM J. Optim. 23, 857-893 (2013).
35. Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. B 68, 49-67 (2006).
36. Zhang, H., Jiang, J., Luo, Z.-Q.: On the linear convergence of proximal splitting method for a class of nonsmooth convex minimization problems. J. Oper. Res. Soc. China 1, 163-186 (2013).