A hypothesis about the rate of global convergence for optimal methods (Newton's type) in smooth convex optimization

Computer Research and Modeling

Volumn 10, Issue 3, 2018, Pages 305-314

  Gasnikov, A V ^a,b   Kovalev, D A ^a  

^a MOSCOW INSTITUTE OF PHYSICS AND TECHNOLOGY   (Russian Federation)

^b INSTITUTE FOR INFORMATION TRANSMISSION PROBLEMS   (Russian Federation)

Author keywords

Chebyshev's type methods; Hesse matrix; Lower bounds; Newton method; Superliner rate of convergence

Indexed keywords

EID: 85049776636     PISSN: 20767633     EISSN: 20776853     Source Type: Journal    
DOI: 10.20537/2076-7633-2018-10-3-305-314     Document Type: Article

References (32)

1. Bayandina A. S., Gasnikov A. V., Lagunovskaya A. A. Bezgradiyentnyye dvukhtochechnyye metody resheniya zadach stokhasticheskoy negladkoy vypukloy optimizatsii pri nalichii malykh shumov ne sluchaynoy prirody [Gradient-less two-point methods for solving stochastic nonsmooth convex optimization problems in the presence of small non-random noises] // Automatics and telemechanics. - 2018. - URL: https://arxiv.org/ftp/arxiv/papers/1701/1701.03821.pdf (in Russian).
2. Vasiliev F. P. Metody optimizatsii [Optimization methods]. - Vol. 2. - Moscow: MCCME, 2011. - P. 433. (in Russian).
3. Vorontsova E. A., Gasnikov A. V., Gorbunov E. A. Uskorennyye spuski po sluchaynomu napravleniyu i bezgradiyentnyye metody s neyevklidovoy proks-strukturoy [Accelerated descents in a random direction and gradientless methods with non-euclidean prox-structure] // Automatics and telemechanics. - 2018. - URL: https://arxiv.org/pdf/1710.00162.pdf (in Russian).
4. Gasnikov A. V. Effektivnyye chislennyye metody poiska ravnovesiya v bol'shikh transportnykh setyakh: dissertatsiya na soiskaniye uchenoy stepeni d. f.-m. n. po spetsial'nosti 05.13.18 [Effective numerical methods for finding equilibrium in large transport networks: thesis for PhD on the specialty 05.13.18] - Matematicheskoye modelirovaniye, chislennyye metody, kompleksy programm [Mathematical modeling, numerical methods, program complexes]. - Moscow: MFTI, 2016. - 487 p. (in Russian).
5. Goodfellow Ya., Bengio I., Courville A. Glubokoye obucheniye [Deep Learning]. - DMK Press, 2017. - 652 p. (in Russian).
6. Dikin I. Metod vnutrennikh tochek v lineynom i nelineynom programmirovanii [Interior point methods in linear and nonlinear programming]. - Moscow: KRASAND, 2010. - 120 p. (in Russian).
7. Zorich V. A. Matematicheskiy analiz zadach yestestvoznaniya [Mathematical analysis of problems in the natural sciences]. - Moscow: MCCME, 2017. - 160 p. (in Russian).
8. Evtushenko Yu.G. Optimizatsiya i bystroye avtomaticheskoye differentsirovaniye [Evtushenko. Optimization and fast automatic differentiation]. - Moscow: CC RAS, 2013. - 144 p. (in Russian).
9. Karmanov V. G. Matematicheskoye programmirovaniye [Mathematical programming]. - M.: Science, 1986. - 288 p. (in Russian).
10. Cormen T., Leiserson C., Rivest R. Algoritmy: postroyeniye i analiz [Introduction to Algorithms]. - Moscow: MCCME, 2002. - 960 p. (in Russian).
11. Nemirovsky A. S., Yudin D. B. Slozhnost' zadach i effektivnost' metodov optimizatsii [Problem complexity and method efficiency in optimization]. - Moscow: Science, 1979. - 384 p. (in Russian).
12. Nesterov Yu. E. Vvedeniye v vypukluyu optimizatsiyu [Introductory lectures on convex optimization]. - Moscow: MCCME, 2010. - 262 p. (in Russian).
13. Protasov V. Yu. K voprosu ob algoritmakh priblizhennogo vychisleniya minimuma vypukloy funktsii po yeye znacheniyam [On the question of algorithms for the approximate calculation of the minimum of a convex function from its values] // Mat. zametki [Math. notes]. - 1996. - Vol. 59, No. 1. - P. 95-102. (in Russian).
14. Razborov A. A. Algebraicheskaya slozhnost' [Algebraic complexity]. - Moscow: MCCME, 2016. - 32 p. (in Russian).
15. Agarwal N., Allen-Zhu Z., Bullins B., Hazan E., Ma T. Finding approximate local minima faster than gradient descent // In Proceedings of the Forty-Ninth Annual ACM Symposium on the Theory of Computing, 2017.
16. Arjevani Y., Shamir O., Shiff R. Oracle complexity of second-order methods for smooth convex optimization // e-print, 2017. - URL: https://arxiv.org/pdf/1705.07260.pdf
17. Baes M. Estimate sequence methods: extensions and approximations // e-print, 2009. - URL: http://www.optimization-online.org/DB-FILE/2009/08/2372.pdf
18. Baydin A. G., Pearlmutter B. A., Radul A. A., Siskand J. M. Automatic differentiation in machine learning: a survey // e-print, 2015. - URL: https://arxiv.org/pdf/1502.05767.pdf
19. Bubeck S. Convex optimization: algorithms and complexity // In Foundations and Trends in Machine Learning. - 2015. - Vol. 8, No. 3-4. - P. 231-357.
20. Carmon Y., Duchi J. C., Hinder O., Sidford A. Accelerated methods for non-convex optimization // e-print, 2017. - URL: https://arxiv.org/pdf/1611.00756.pdf
21. Dvurechensky P., Gasnikov A., Tiurin A. Randomized Similar Triangles Method: A Unifying Framework for Accelerated Randomized Optimization Methods (Coordinate Descent, Directional Search, Derivative-Free Method) // SIAM J. Optim. - 2017 (Submitted). - URL: https://arxiv.org/pdf/1707.08486.pdf
22. Ghadimi S., Liu H., Zhang T. Second-order methods with cubic regularization under inexact information // e-print, 2017. - URL: https://arxiv.org/pdf/1710.05782.pdf
23. Grapiglia G. N., Nesterov Yu. Regularized Newton methods for minimazing functions with Hölder continuous Hessian // SIAM J. Optim. - 2017. - Vol. 27 (1). - P. 478-506.
24. Lee Y.-T., Sidford A., Wong S. C.-W. A faster cutting plane method and its implications for combinatorial and convex optimization // e-print, 2015. - URL: https://arxiv.org/pdf/1508.04874.pdf
25. Monteiro R., Svaiter B. An accelerated hybrid proximal extragradient method for convex optimization and its implications to second-order methods // SIAM Journal on Optimization. - 2013. - Vol. 23 (2). - P. 1092-1125.
26. Nemirovski A. Lectures on modern convex optimization analysis, algorithms, and engineering applications. - Philadelphia: SIAM, 2015. - URL: http://www2.isye.gatech.edu/nemirovs/ Lect-ModConvOpt.pdf
27. Nesterov Yu. Accelerating the cubic regularization of Newton's method on convex problems // Math. Prog., Ser. A. - 2008. - Vol. 112. - P. 159-181.
28. Nesterov Yu. Implementable tensor methods in unconstrained convex optmization // CORE Discussion Papers 2018005. - 2018. - URL: https://ideas.repec.org/p/cor/louvco/2018005.html
29. Nesterov Yu. Minimizing functions with bounded variation of subgradients // CORE Discussion Papers. 2005/79. - 2005. - 13 p. - URL: http://webdoc.sub.gwdg.de/ebook/serien/e/CORE/ dp2005-79.pdf
30. Nesterov Yu., Polyak P. Cubic regularization of Newton method and its global performance // Math. Program. Ser. A. - 2006. - Vol. 108. - P. 177-205.
31. Nesterov Yu., Spokoiny V. Random gradient-free minimization of convex functions // Foundations of Computational Mathematics. - 2017. - Vol. 17 (2). - P. 527-566.
32. Nocedal J., Wright S. Numerical optimization. - Springer, 2006.