Open problems in nonlinear conjugate gradient algorithms for unconstrained optimization

Bulletin of the Malaysian Mathematical Sciences Society

Volumn 34, Issue 2, 2011, Pages 319-330

  Andrei, Neculai ^a,b  

^a NATIONAL INSTITUTE FOR RESEARCH AND DEVELOPMENT IN INFORMATICS   (Romania)

^b ACADEMY OF ROMANIAN SCIENTISTS   (Romania)

Author keywords

Conjugate gradient method; Newton method; Quasi Newton methods; Unconstrained optimization

Indexed keywords

EID: 79955418173     PISSN: 01266705     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (67)

1. J. C. Allwright, Improving the conditioning of optimal control problems using simple methods, In O. J. Bell (Ed.), Recent Mathematical Developments in Control, Academic Press, London, 1972.
2. N. Andrei, Scaled conjugate gradient algorithms for unconstrained optimization, Comput. Optim. Appl. 38 (2007), no. 3, 401-416.
3. N. Andrei, Scaled memoryless BFGS preconditioned conjugate gradient algorithm for unconstrained optimization, Optim. Methods Softw. 22 (2007), no. 4, 561-571.
4. N. Andrei, A scaled BFGS preconditioned conjugate gradient algorithm for unconstrained optimization, Appl. Math. Lett. 20 (2007), no. 6, 645-650.
5. N. Andrei, A scaled nonlinear conjugate gradient algorithm for unconstrained optimization, Optimization 57 (2008), no. 4, 549-570.
6. N. Andrei, A Dai-Yuan conjugate gradient algorithm with sufficient descent and conjugacy conditions for unconstrained optimization, Appl. Math. Lett. 21 (2008), no. 2, 165-171.
7. N. Andrei, A hybrid conjugate gradient algorithm with modified secant condition for unconstrained optimization, ICI Technical Report, February 6, 2008.
8. N. Andrei, A hybrid conjugate gradient algorithm for unconstrained optimization as a convex combination of Hestenes-Stiefel and Dai-Yuan, Studies in Informatics and Control 17 (2008), 55-70.
9. N. Andrei, An acceleration of gradient descent algorithm with backtracking for unconstrained optimization, Numer. Algorithms 42 (2006), no. 1, 63-73.
10. N. Andrei, Numerical comparison of conjugate gradient algorithms for unconstrained optimization, Studies in Informatics and Control 16 (2007), no. 4, 333-352.
11. N. Andrei, Accelerated conjugate gradient algorithm with modified secant condition for unconstrained optimization, ICI Technical Report, March 3, 2008.
12. N. Andrei, Accelerated scaled memoryless BFGS preconditioned conjugate gradient algorithm for unconstrained optimization, ICI Technical Report, March 24, 2008.
13. N. Andrei, 40 conjugate gradient algorithms for unconstrained optimization. A survey on their definition. ICI Technical Report No. 13/08, March 14, 2008.
14. N. Andrei, Acceleration of conjugate gradient algorithms for unconstrained optimization, Appl. Math. Comput. 213 (2009), no. 2, 361-369.
15. N. Andrei, New accelerated conjugate gradient algorithms for unconstrained optimization, ICI Technical Report, June 18, 2008.
16. N. Andrei, New hybrid conjugate gradient algorithms as a convex combination of PRP and DY for unconstrained optimization, ICI Technical Report, October 1, 2007.
17. N. Andrei, Hybrid conjugate gradient algorithm for unconstrained optimization, J. Optim. Theory Appl. 141 (2009), no. 2, 249-264.
18. N. Andrei, Another nonlinear conjugate gradient algorithm for unconstrained optimization, Optim. Methods Softw. 24 (2009), no. 1, 89-104.
19. N. Andrei, Accelerated conjugate gradient algorithm with finite difference Hessian/vector product approximation for unconstrained optimization, J. Comput. Appl. Math. 230 (2009), no. 2, 570-582.
20. N. Andrei, Hybrid conjugate gradient algorithm for unconstrained optimization, J. Optim. Theory Appl. 141 (2009), no. 2, 249-264.
21. N. Andrei, Performance profiles of conjugate gradient algorithms for unconstrained optimization. Encyclopedia of Optimization, second edition, C. A. Floudas and P. Pardalos (Eds.), Springer Science + Business Media, New York, 2009, vol. P, pp. 2938-2953.
22. M. Arioli and D. Loghin, Stopping criteria for mixed finite element problems, Electron. Trans. Numer. Anal. 29 (2007/08), 178-192.
23. L. Armijo, Minimization of functions having Lipschitz continuous first partial derivatives, Pacific J. Math. 16 (1966), 1-3.
24. E. M. L. Beale, A derivation of conjugate gradients, in Numerical Methods for Non-linear Optimization (Conf., Univ. Dundee, Dundee, 1971), 39-43, Academic Press, London.
25. E. G. Birgin and J. M. Martinez, A spectral conjugate gradient method for unconstrained optimization, Appl. Math. Optim. 43 (2001), no. 2, 117-128.
26. A. R. Conn, N. Gould and P. L. Toint, Improving the decomposition of partially separable functions in the context of large-scale optimization: a first approach, in Large scale optimization (Gainesville, FL, 1993), 82-94, Kluwer Acad. Publ., Dordrecht.
27. A. R. Conn, N. I. M. Gould and Ph. L. Toint, LANCELOT, Springer Series in Computational Mathematics, 17, Springer, Berlin, 1992.
28. H. Crowder and P. Wolfe, Linear convergence of the conjugate gradient method, IBM J. Res. Develop. 16 (1972), 431-433.
29. Y.-H. Dai and L.-Z. Liao, New conjugacy conditions and related nonlinear conjugate gradient methods, Appl. Math. Optim. 43 (2001), no. 1, 87-101.
30. Y.-H. Dai and Y. Yuan, A nonlinear conjugate gradient method with a strong global convergence property, SIAM J. Optim. 10 (1999), no. 1, 177-182 (electronic).
31. J. W. Daniel, The conjugate gradient method for linear and nonlinear operator equations, SIAM J. Numer. Anal. 4 (1967), 10-26.
32. J. E. Dennis, Jr. and R. B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice Hall Series in Computational Mathematics, Prentice Hall, Englewood Cliffs, NJ, 1983.
33. L. C. W. Dixon, P. G. Ducksbury and P. Singh, A new three-term conjugate gradient method, J. Optim. Theory Appl. 47 (1985), no. 3, 285-300.
34. J. A. Fessler and S. D. Booth, Conjugate-gradient preconditioning methods for shift-variant PET image reconstruction, IEEE Trans. Image Process. 8 (1999), no. 5, 688-699.
35. R. Fletcher and C. M. Reeves, Function minimization by conjugate gradients, Comput. J. 7 (1964), 149-154.
36. J. C. Gilbert and J. Nocedal, Global convergence properties of conjugate gradient methods for optimization, SIAM J. Optim. 2 (1992), no. 1, 21-42.
37. A. A. Goldstein, On steepest descent, J. Soc. Indust. Appl. Math. Ser. A Control 3 (1965), 147-151.
38. L. Grippo, F. Lampariello and S. Lucidi, A nonmonotone line search technique for Newton's method, SIAM J. Numer. Anal. 23 (1986), no. 4, 707-716.
39. W. W. Hager and H. Zhang, A new conjugate gradient method with guaranteed descent and an efficient line search, SIAM J. Optim. 16 (2005), no. 1, 170-192.
40. W. W. Hager and H. Zhang, A survey of nonlinear conjugate gradient methods, Pac. J. Optim. 2 (2006), no. 1, 35-58.
41. M. R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, J. Research Nat. Bur. Standards 49 (1952), 409-436 (1953).
42. Y. F. Hu and C. Storey, Global convergence result for conjugate gradient methods, J. Optim. Theory Appl. 71 (1991), no. 2, 399-405.
43. A. V. Knyazev and I. Lashuk, Steepest descent and conjugate gradient methods with variable preconditioning, SIAM J. Matrix Anal. Appl. 29 (2007), no. 4, 1267-1280.
44. C. Lemarechal, A view of line-searches, in Optimization and Optimal Control (Proc. Conf., Math. Res. Inst., Oberwolfach, 1980), 59-78, Lecture Notes in Control and Information Sci., 30, Springer, Berlin.
45. D. C. Liu and J. Nocedal, On the limited memory BFGS method for large scale optimization, Math. Programming 45 (1989), no. 3, (Ser. B), 503-528.
46. J. J. More and D. J. Thuente, Line search algorithms with guaranteed sufficient decrease, ACM Trans. Math. Software 20 (1994), no. 3, 286-307.
47. L. Nazareth, A conjugate direction algorithm without line searches, J. Optimization Theory Appl. 23 (1977), no. 3, 373-387.
48. J. Nocedal, Conjugate gradient methods and nonlinear optimization, in Linear and Nonlinear Conjugate Gradient-related Methods (Seattle, WA, 1995), 9-23, SIAM, Philadelphia, PA.
49. E. Polak and G. Ribière, Note sur la convergence de directions conjuguée, Rev. Francaise Informat Recherche Operationelle, 3e Année 16 (1969), 35-43.
50. B. T. Polyak, The conjugate gradient method in extreme problems, USSR Comp. Math. Math. Phys. 9 (1969), 94-112.
51. F. A. Potra and Y. Shi, Efficient line search algorithm for unconstrained optimization, J. Optim. Theory Appl. 85 (1995), no. 3, 677-704.
52. M. J. D. Powell, Some global convergence properties of a variable metric algorithm for minimization without exact line searches, in Nonlinear Programming (Proc. Sympos., New York, 1975), 53-72. SIAM-AMS Proc., IX, Amer. Math. Soc., Providence, RI.
53. M. J. D. Powell, Some convergence properties of the conjugate gradient method, Math. Programming 11 (1976), no. 1, 42-49.
54. M. J. D. Powell, Restart procedures for the conjugate gradient method, Math. Programming 12 (1977), no. 2, 241-254.
55. J. K. Reid, On the method of conjugate gradients for the solution of large sparse systems of linear equations, in Large Sparse Sets of Linear Equations (Proc. Conf., St. Catherine's Coll., Oxford, 1970), 231-254, Academic Press, London.
56. D. F. Shanno, Conjugate gradient methods with inexact searches, Math. Oper. Res. 3 (1978), no. 3, 244-256.
57. D. F. Shanno, K. H. Phua, Algorithm 500, Minimization of unconstrained multivariate functions, ACM Trans. on Math. Software 2 (1976), 87-94.
58. D. Touati-Ahmed and C. Storey, Efficient hybrid conjugate gradient techniques, J. Optim. Theory Appl. 64 (1990), no. 2, 379-397.
59. Z. Wei, G. Li and L. Qi, New nonlinear conjugate gradient formulas for large-scale unconstrained optimization problems, Appl. Math. Comput. 179 (2006), no. 2, 407-430.
60. Z. X. Wei, G. Y. Li and L. Q. Qi, Global convergence of the Polak-Ribière-Polyak conjugate gradient method with an Armijo-type inexact line search for nonconvex unconstrained optimization problems, Math. Comp. 77 (2008), no. 264, 2173-2193.
61. P. Wolfe, Convergence conditions for ascent methods, SIAM Rev. 11 (1969), 226-235.
62. H. Yabe and M. Takano, Global convergence properties of nonlinear conjugate gradient methods with modified secant condition, Comput. Optim. Appl. 28 (2004), no. 2, 203-225.
63. Y. Yuan, Some problems in nonlinear programming, Numerical Linear Algebra and Optimization, Science Press, Beijing/New York, 2003, pp. 90.
64. G. Yuan and X. Lu, A modified PRP conjugate gradient method, Ann. Oper. Res. 166 (2009), 73-90.
65. G. Yuan, Modified nonlinear conjugate gradient methods with sufficient descent property for large-scale optimization problems, Optimization Letters (D0I:10.1007/s11590-008-0086-5).
66. J. Z. Zhang, N. Y. Deng and L. H. Chen, New quasi-Newton equation and related methods for unconstrained optimization, J. Optim. Theory Appl. 102 (1999), no. 1, 147-167.
67. J. Zhang and C. Xu, Properties and numerical performance of quasi-Newton methods with modified quasi-Newton equations, J. Comput. Appl. Math. 137 (2001), no. 2, 269-278.