New adaptive stepsize selections in gradient methods

Journal of Industrial and Management Optimization

Volumn 4, Issue 2, 2008, Pages 299-312

  Frassoldati, Giacomo ^a   Zanni, Luca ^a   Zanghirati, Gaetano ^b  

^a UNIVERSITY OF MODENA AND REGGIO EMILIA   (Italy)

^b UNIVERSITY OF FERRARA   (Italy)

Author keywords

Adaptive stepsize selections; Gradient methods; Strictly convex quadratics; Unconstrained optimization

Indexed keywords

EID: 56649089614     PISSN: 15475816     EISSN: 1553166X     Source Type: Journal    
DOI: 10.3934/jimo.2008.4.299     Document Type: Article

References (23)

1. H. Akaike, On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method, Ann. Inst. Statist. Math. Tokyo, 11 (1959), 1-17.
2. J. Barzilai and J. M. Borwein, Two-point step size gradient methods, IMA J. Numer. Anal., 8 (1988), 141-148.
3. E. G. Birgin, J. M. Martínez and M. Raydan, Nonmonotone spectral projected gradient methods on convex sets, SIAM J. Optim., 10 (2000), 1196-1211.
4. A. Cauchy, Méthode générale pour la résolution des systèmes d'equations simultanées, Comp. Rend. Sci. Paris, 25 (1847), 46-89.
5. Y. H. Dai, Alternate stepsize gradient method, Optimization, 52 (2003), 395-415.
6. Y. H. Dai and R. Fletcher, On the asymptotic behaviour of some new gradient methods, Math. Prog. (Series A), 103 (2005), 541-559.
7. Y. H. Dai and R. Fletcher, Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming, Numer. Math., 100 (2005), 51-47.
8. Y. H. Dai and R. Fletcher, New algorithms for singly linearly constrained quadratic programs subject to lower and upper bounds, Math. Prog. (Series A), 106 (2006), 403-421.
9. W. W. Hager and H. Zhang, A new active set algorithm for box constrained optimization, SIAM J. Optim., 17 (2006), 526-557.
10. Y. H. Dai, W. W. Hager, K. Schittkowski and H. Zhang, The cyclic Barzilai-Borwein method for unconstrained optimization, IMA J. Numer. Anal., 26 (2006), 604-627.
11. Y. H. Dai and Y. X. Yuan, Alternate minimization gradient method, IMA J. Numer. Anal., 23 (2003), 377-393.
12. Y. H. Dai and Y. X. Yuan, Analysis of monotone gradient methods, J. Ind. Manag. Optim., 1 (2005), 181-192.
13. R. Fletcher, On the Barzilai-Borwein method, in "Optimization and Control with Applications", Appl. Optim., vol. 96, Springer, New York, (2005), 235-256.
14. A. Friedlander, J. M. Martínez, B. Molina and M. Raydan, Gradient method with retards and generalization, SIAM J. Numer. Anal., 36 (1999), 275-289.
15. M. Raydan, The Barzilai and Borwein gradient method for the large scale unconstrained minimization problems, SIAM J. Optim., 7 (1997), 26-33.
16. M. Raydan and B. F. Svaiter, Relaxed steepest descent and Cauchy-Barzilai-Borwein method, Comput. Optim. Appl., 21 (2002), 155-167.
17. K. Schittkowski, Optimal parameter selection in support vector machines, J. Ind. Manag. Optim., 1 (2005), 465-476.
18. T. Serafini, G. Zanghirati and L. Zanni, Gradient projection methods for quadratic programs and applications in training support vector machines, Optim. Meth. Soft., 20 (2005), 353-378.
19. Y. X. Yuan, A new stepsize for the steepest descent method, J. Comput. Math., 24 (2006), 149-156.
20. 2 exact penalty function in global optimization, J. Ind. Manag. Optim., 1 (2005), 533-547.
21. L. Zanni, An improved gradient projection-based decomposition technique for support vector machines, Comput. Manag. Sci., 3 (2006), 131-145.
22. L. Zanni, T. Serafini and G. Zanghirati, Parallel software for training large scale support vector machines on multiprocessor systems, J. Mach. Learn. Res., 7 (2006), 1467-1492.
23. B. Zhou, L. Gao and Y. H. Dai, Gradient methods with adaptive step-sizes, Comput. Optim. Appl., 35 (2006), 69-86.