Random Gradient-Free Minimization of Convex Functions

Foundations of Computational Mathematics

Volumn 17, Issue 2, 2017, Pages 527-566

  Nesterov, Yurii ^a   Spokoiny, Vladimir ^b  

^a UNIVERSITÉ CATHOLIQUE DE LOUVAIN   (Belgium)

^b HUMBOLDT UNIVERSITY   (Germany)

Author keywords

Complexity bounds; Convex optimization; Derivative free methods; Random methods; Stochastic optimization

Indexed keywords

APPROXIMATION THEORY; CONVEX OPTIMIZATION; FUNCTIONS; GRADIENT METHODS; NORMAL DISTRIBUTION; OPTIMIZATION;

COMPLEXITY BOUNDS; COMPUTATIONAL EXPERIMENT; DERIVATIVE-FREE METHODS; ITERATION COUNTERS; NONCONVEX FUNCTIONS; RANDOM METHODS; RATE OF CONVERGENCE; STOCHASTIC OPTIMIZATIONS;

ITERATIVE METHODS;

EID: 84948964679     PISSN: 16153375     EISSN: 16153383     Source Type: Journal    
DOI: 10.1007/s10208-015-9296-2     Document Type: Article

References (22)

1. A. Agarwal, O. Dekel, and L. Xiao, Optimal algorithms for online convex optimization with multi-point bandit feedback, in Proceedings of the 23rd Annual Conference on Learning, 2010, pp. 2840.
2. A. Agarwal, D. Foster, D. Hsu, S. Kakade, and A. Rakhlin, Stochastic convex optimization with bandit feedback,. SIAM J. on Optimization, 23 (2013), pp. 213-240.
3. D. Bertsimas and S. Vempala, Solving convex programs by random walks, J. of the ACM, 51 (2004), pp. 540-556.
4. F. Clarke, Optimization and nonsmooth analysis, Wliley, New York, 1983.
5. A. Conn, K. Scheinberg, and L. Vicente, Introduction to derivative-free optimization. MPS-SIAM series on optimization, SIAM, Philadelphia, 2009.
6. C. Dorea, Expected number of steps of a random optimization method, JOTA, 39 (1983), pp. 165-171.
7. J. Duchi, M.I. Jordan, M.J. Wainwright, and A. Wibisono, Finite sample convergence rate of zero-order stochastic optimization methods, in NIPS, 2012, pp. 1448-1456.
8. A. D. Flaxman, A.T. Kalai, and B.H. Mcmahan, Online convex optimization in the bandit setting: gradient descent without a gradient, in Proceedings of the 16th annual ACM-SIAM symposium on Discrete Algorithms, 2005, pp. 385-394.
9. R. Kleinberg, A. Slivkins, and E. Upfal, Multi-armed bandits in metric spaces, in Proceedings of the 40th annual ACM symposium on Theory of Computing, 2008, pp. 681-690.
10. J. C. Lagarias, J. A. Reeds, M. H. Wright, and P. E. Wright, Convergence properties of the Nelder-Mead Simplex Algorithm in low dimensions, SIAM J. Optimization, 9 (1998), pp. 112-147.
11. J. C. Lagarias, B. Poonen, and M. H. Wright, Convergence of the restricted Nelder-Mead algorithm in two dimensions, SIAM J. Optimization, 22 (2012), pp. 501-532.
12. J. Matyas, Random optimization. Automation and Remote Control, 26 (1965), pp. 246-253.
13. J. A. Nelder and R. Mead, A simplex method for function minimization, Computer Journal, 7 (1965), pp. 308–3013
14. A. Nemirovski, A. Juditsky, G. Lan, and A.Shapiro, Robust Stochastic Approximation approach to Stochastic Programming, SIAM J. on Optimization, 19 (2009), pp. 1574-1609.
15. A. Nemirovsky and D.Yudin, Problem complexity and method efficiency in optimization, John Wiley and Sons, New York, 1983.
16. Yu. Nesterov, Introductory Lectures on Convex Optimization, Kluwer, Boston, 2004.
17. Yu. Nesterov, Lexicographic differentiation of nonsmooth functions’, Mathematical Programming, 104 (2005), pp. 669-700.
18. Yu. Nesterov, Random gradient-free minimization of convex functions, CORE Discussion Paper # 2011/1, (2011).
19. Yu. Nesterov, Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM J. on Optimization, 22 (2012), pp. 341-362.
20. B. Polyak, Introduction to Optimization. Optimization Software - Inc., Publications Division, New York, 1987.
21. V. Protasov, Algorithms for approximate calculation of the minimum of a convex function from its values, Mathematical Notes, 59 (1996), pp. 69-74.
22. M. Sarma, On the convergence of the Baba and Dorea random optimization methods, JOTA, 66 (1990), pp. 337-343.