Bundle-level type methods uniformly optimal for smooth and nonsmooth convex optimization

Mathematical Programming

Volumn 149, Issue 1-2, 2015, Pages 1-45

  Lan, Guanghui ^a  

^a J Crayton Pruitt Family Department of Biomedical Engineering   (United States)

Author keywords

Bundle level; Complexity; Convex programming; Optimal methods

Indexed keywords

APL (PROGRAMMING LANGUAGE); CONVEX OPTIMIZATION; ITERATIVE METHODS; STOCHASTIC PROGRAMMING; STOCHASTIC SYSTEMS;

BUNDLE-LEVEL; COMPLEXITY; LIPSCHITZ CONSTANT; NON-SMOOTH CONVEX OPTIMIZATIONS; OPTIMAL COMPLEXITY; OPTIMAL METHODS; SADDLE POINT PROBLEMS; SEMI-DEFINITE PROGRAMMING;

NUMERICAL METHODS;

EID: 84890945640     PISSN: 00255610     EISSN: 14364646     Source Type: Journal    
DOI: 10.1007/s10107-013-0737-x     Document Type: Article

References (48)

1. Ahmed, S.: Smooth Minimization of Two-Stage Stochastic Linear Programs. Manuscript, Georgia Institute of Technology (2006)
2. Auslender, A., Teboulle, M.: Interior gradient and proximal methods for convex and conic optimization. SIAM J. Optim. 16, 697–725 (2006)
3. Becker, S., Bobin, J., Candès, E.: NESTA: A Fast and Accurate First-Order Method for Sparse Recovery. Manuscript, California Institute of Technology (2009)
4. Ben-Tal, A., Nemirovski, A.S.: Lectures on Modern Convex Optimization: Analysis, Algorithms. Engineering Applications. MPS-SIAM Series on Optimization. SIAM, Philadelphia (2000)
5. Ben-Tal, A., Nemirovski, A.S.: Non-euclidean restricted memory level method for large-scale convex optimization. Math. Program. 102, 407–456 (2005)
6. Chenny, E.W., Goldstein, A.A.: Newton’s methods for convex programming and tchebytcheff approximation. Numer. Math. 1, 253–268 (1959)
7. d’Aspremont, A., Banerjee, O., El Ghaoui, L.: First-order methods for sparse covariance selection. SIAM J. Matrix Anal. Appl. 30, 56–66 (2008)
8. Devolder, O., Glineur, F., Nesterov, Y.E.: First-Order Methods of Smooth Convex Optimization with Inexact Oracle. Manuscript, CORE, Université catholique de Louvain, Louvain-la-Neuve, Belgium (Dec 2010)
9. Ghadimi, S., Lan, G.: Optimal stochastic approximation algorithms for strongly convex stochastic composite optimization, II: shrinking procedures and optimal algorithms. Technical report, SIAM J. Optim. (to appear) (2010)
10. Ghadimi, S., Lan, G.: Optimal stochastic approximation algorithms for strongly convex stochastic composite optimization, I: a generic algorithmic framework. SIAM J. Optim. 22, 1469–1492 (2012)
11. Helmberg, C., Rendl, F.: A spectral bundle method for semidefinite programming. SIAM J. Optim. 10, 673–696 (2000)
12. Hoffman, A.: On approximate solutions of systems of linear inequalities. J. Res. Natl. Bureau Stand Sect. B Math. Sci. 49, 263–265 (1952)
13. Juditsky, A., Nemirovski, A.S., Tauvel, C.: Solving Variational Inequalities with Stochastic Mirror-Prox Algorithm. Manuscript, Georgia Institute of Technology, Atlanta, GA (2008)
14. Kelley, J.E.: The cutting plane method for solving convex programs. J. SIAM 8, 703–712 (1960)
15. Kiwiel, K.C.: An aggregate subgradient method for nonsmooth convex minimization. Math. Program. 27, 320–341 (1983)
16. Kiwiel, K.C.: Proximity control in bundle methods for convex nondifferentiable minimization. Math. Program. 46, 105–122 (1990)
17. Kiwiel, K.C.: Proximal level bundle method for convex nondifferentable optimization, saddle point problems and variational inequalities. Math. Program. Ser. B 69, 89–109 (1995)
18. Lan, G., Lu, Z., Monteiro, R.D.C.: Primal-dual first-order methods with $${\cal O}(1/\epsilon)$$O(1/ϵ) iteration-complexity for cone programming. Math. Program. 126, 1–29 (2011)
19. Lan, G.: An optimal method for stochastic composite optimization. Math. Program. 133(1), 365–397 (2012)
20. Lan, G., Nemirovski, A.S., Shapiro, A.: Validation analysis of mirror descent stochastic approximation method. Math. Program. 134, 425–458 (2012)
21. Lemaréchal, C.: An extension of davidon methods to non-differentiable problems. Math. Program. Study 3, 95–109 (1975)
22. Lemaréchal, C., Nemirovski, A.S., Nesterov, Y.E.: New variants of bundle methods. Math. Program. 69, 111–148 (1995)
23. Lewis, A.S., Wright, S.J.: A Proximal Method for Composite Minimization. Manuscript, Cornell University, Ithaca, NY (2009)
24. Linderoth, J., Wright, S.: Decomposition algorithms for stochastic programming on a computational grid. Comput. Optim. Appl. 24, 207–250 (2003)
25. Linderoth, J., Shapiro, A., Wright, S.: The empirical behavior of sampling methods for stochastic programming. Ann. Oper. Res. 142, 215–241 (2006)
26. Lu, Z.: Smooth optimization approach for sparse covariance selection. SIAM J. Optim. 19, 1807–1827 (2009)
27. Mak, W.K., Morton, D.P., Wood, R.K.: Monte carlo bounding techniques for determining solution quality in stochastic programs. Oper. Res. Lett. 24, 47–56 (1999)
28. Mosek. The Mosek Optimization Toolbox for Matlab Manual. Version 6.0 (revision 93). http://www.mosek.com
29. Nemirovski, A.S., Yudin, D.: Problem Complexity and Method Efficiency in Optimization. Wiley-Interscience Series in Discrete Mathematics. Wiley, XV (1983)
30. Nemirovski, A.S.: Efficient Methods in Convex Programming. Lecture notes, Technion (1994)
31. Nemirovski, A.S.: Prox-method with rate of convergence $$o(1/t)$$o(1/t) for variational inequalities with lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM J. Optim. 15, 229–251 (2005)
32. Nemirovski, A.S., Juditsky, A., Lan, G., Shapiro, A.: Robust stochastic approximation approach to stochastic programming. SIAM J. Optim. 19, 1574–1609 (2009)
33. Nemirovskii, A.S., Nesterov, Y.E.: Optimal methods for smooth convex minimization. Zh. Vichisl. Mat. Fiz. 25, 356–369 (1985). (In Russian)
34. Nesterov, Y.E.: Efficient Methods in Nonlinear Programming. Radio i Sviaz, Moscow (1989)
35. Nesterov, Y.E.: Gradient methods for minimizing composite objective functions. Technical report, Center for Operations Research and Econometrics (CORE), Catholic University of Louvain, September (2007)
36. Nesterov, Y.E.: A method for unconstrained convex minimization problem with the rate of convergence $$O(1/k^2)$$O(1/k2). Doklady AN SSSR 269, 543–547 (1983)
37. Nesterov, Y.E.: On an approach to the construction of optimal methods of minimization of smooth convex functions. Ekonomo. i. Mat. Metody 24, 509–517 (1988)
38. Nesterov, Y.E.: Introductory Lectures on Convex Optimization: A Basic Course. Kluwer, Massachusetts (2004)
39. Nesterov, Y.E.: Excessive gap technique in nonsmooth convex minimization. SIAM J. Optim. 16, 235–249 (2005)
40. Nesterov, Y.E.: Smooth minimization of nonsmooth functions. Math. Program. 103, 127–152 (2005)
41. Nesterov, Y.E.: Smoothing technique and its applications in semidefinite optimization. Math. Program. 110, 245–259 (2007)
42. Oliveira, W., Sagastizábal, C., Scheimberg, S.: Inexact bundle methods for two-stage stochastic programming. SIAM J. Optim. 21(2), 517–544 (2011)
43. Peña, J.: Nash equilibria computation via smoothing techniques. Optima 78, 12–13 (2008)
44. Richtárik, P.: Approximate level method for nonsmooth convex minimization. J. Optim. Theory Appl. 152(2), 334–350 (2012)
45. Ruszczyński, A.: Nonlinear Optimization, 1st edn. Princeton University Press, Princeton (2006)
46. Sagastizábal, C.: Nonsmooth Optimization: Thinking Outside of the Black Box. SIAG/OPT Views-and-News, pp. 1–9 (2011)
47. Sen, S., Doverspike, R.D., Cosares, S.: Network planning with random demand. Telecommun. Syst. 3, 11–30 (1994)
48. Tseng, P.: On Accelerated Proximal Gradient Methods for Convex-Concave Optimization. Manuscript, University of Washington, Seattle (May 2008)