Dynamic scaling based preconditioning for truncated Newton methods in large scale unconstrained optimization

Optimization Methods and Software

Volumn 20, Issue 6, 2005, Pages 693-713

  Roma, Massimo ^a  

^a SAPIENZA UNIVERSITY OF ROME   (Italy)

Author keywords

Conjugate gradient (CG) method; Equilibrated matrix; Preconditioning; Row column scaling; Truncated Newton method

Indexed keywords

ALGORITHMS; LINEAR SYSTEMS; MATRIX ALGEBRA; OPTIMIZATION; VECTORS;

CONJUGATE GRADIENT (CG) METHOD; EQUILIBRATED MATRIX; PRECONDITIONING; ROW-COLUMN SCALING; TRUNCATED NEWTON METHOD;

COMPUTATIONAL METHODS;

EID: 29744436841     PISSN: 10556788     EISSN: 10294937     Source Type: Journal    
DOI: 10.1080/10556780410001727709     Document Type: Article

References (37)

1. Nocedal, J., 1997, Large scale unconstrained optimization. In: A. Watson and I. Duff (Eds) The State of the Art in Numerical Analysis (Oxford: Oxford University Press), pp. 311-338.
2. Roma, M., 2001, Large scale unconstrained optimization. In: C. Floudas and P. Pardalos (Eds) Encyclopedia of Optimization, Vol. III (Kluwer Academic Publishers), pp. 143-150.
3. Nash, S.G., 2000, A survey of truncated-Newton methods. Journal of Computational and Applied Mathematics, 124, 45-59.
4. O'Leary, D.P., 1982, A discrete Newton algorithm for minimizing a function of many variables. Mathematical Programming, 23, 20-33.
5. Morales, J.L. and Nocedal, J., 2000, Automatic preconditioning by limited memory quasi-Newton updating. SIAM Journal on Optimization, 10, 1079-1096.
6. Van der Sluis, A., 1969, Condition number and equilibration of matrices. Numerische Mathematik, 14, 14-23.
7. Bauer, F.L., 1963, Optimally scaled matrices. Numerische Mathematik, 5, 73-87.
8. Averick, B.M., Carter, R.G., Moré, J.J. and Xue, G., 1992, The MINPACK-2 test problems collection. Preprint MCS-P153-0692, Argonne National Laboratoty, Argonne, IL.
9. Bongartz, I., Conn, A.R., Gould, N.I.M. and Toint, Ph.L., 1995, CUTE: Constrained and unconstrained testing environment. ACM Transaction on Mathematical Software, 21, 123-160.
10. Dembo, R.S. and Steihaug, T., 1983, Truncated-Newton algorithms for large-scale unconstrained optimization. Mathematical Programming, 26, 190-212.
11. Dembo, R.S., Eisenstat, S.C. and Steihaug, T., 1982, Inexact Newton methods. SIAM Journal on Numerical Analysis, 19, 400-408.
12. Golub, G.H. and Van Loan, C.F., 1996, Matrix Computations (3rd edn) (Baltimore: The John Hopkins Press).
13. Barret, R., Berry, M., Chan, T.F., Demmel, J., Donato, J., Dongarra, J., Eijkhout, V., Pozo, R., Romine, C. and Van der Vorst, H., 1994, Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods (Philadelphia, PA: SIAM).
14. Greenbaum, A., 1997, Iterative Methods for Solving Linear Systems (Philadelphia, PA: SIAM).
15. Van Der Vorst, H. and Dekker, K., 1988, Conjugate gradient type methods and preconditioning. Journal of Computational and Applied Mathematics, 24, 73-87.
16. Axelsson, O., 1994, Iterative Solution Methods (Cambridge: Cambridge University Press).
17. Meurant, G., 1999, Computer Solution of Large Linear Systems (Elsevier, Amsterdam: North Holland).
18. Lin, C.-J. and Moré, J., 1999, Incomplete Cholesky factorization with limited memory. SIAM Journal on Scientific Computing, 21, 24-45.
19. Schlick, T., 1993, Modified Cholesky factorization for sparse preconditioners. SIAM Journal on Scientific Computing, 14, 424-445.
20. Bouaricha, A., Moré, J. and Wu, Z., 1998, Preconditioning Newton's method. Technical Report CRPC-TR98762, Center for Research on Parallel Computing, Rice University, Houston, TX.
21. Daydé, M.J., L'Excellent, J.-Y. and Gould, N.I.M., 1997, Element-by-element preconditioners for large-scale partially separable optimization problems. SIAM Journal on Scientific Computing, 18, 1767-1787.
22. Griewank, A., 1989, On automatic differentiation. In: M. Iri and K. Tanake (Eds) Mathematical Programming: Recent Developments and Applications (Kluwer Academic Publishers), pp. 83-108.
23. Nash, S.G., 1985, Preconditioning of truncated-Newton methods. SIAM Journal on Scientific and Statistical Computing, 6, 599-616.
24. Nash, S.G. and Nocedal, J., 1991, A numerical study of the limited memory BFGS method and the truncated-Newton method for large scale optimization. SIAM Journal on Optimization, 1, 358-372.
25. Morales, J.L. and Nocedal, J., 2001, Algorithm PREQN: Fortran 77 subroutine for preconditioning the conjugate gradient method. ACM Transaction on Mathematical Software, 27, 83-91.
26. Duff, I.S., Erisman, A.M. and Reid, J.K., 1986, Direct Methods for Sparse Matrices (London: Oxford University Press).
27. Schneider, M.H. and Zenios, S., 1990, A comparative study of algorithms for matrix balancing. Operations Research, 38, 439-455.
28. Ruiz, D., 2001, A scaling algorithm to equilibrate both rows and columns norms in matrices. Technical Report RAL-TR-2001-034, Computational Sciences and Engineering Department, Rutherford Appleton Laboratory, Oxon, UK. (Submitted to Linear Algebra and Applications.)
29. Bunch, J.R., 1971, Equilibration of symmetric matrices in the max-norm. Journal of ACM, 18, 566-572.
30. Parlett, B.N. and Landis, T.L., 1982, Methods for scaling to double stochastic form. Linear Algebra and Applications, 48, 53-79.
31. Higham, N.J., 2002, Accuracy and Stability of Numerical Algorithms (2nd edn) (Philadelphia, PA: SIAM).
32. Liu, D.C. and Nocedal, J., 1989, On the limited memory BFGS method for large scale optimization. Mathematical Programming, 45, 503-528.
33. Griewank, A. and Toint, Ph.L., 1982, Partitioned variable metric updates for large structured optimization problems. Numerische Mathematik, 39, 119-137.
34. Barlow, J.L. and Toraldo, G., 1995, The effect of diagonal scaling on projected gradient methods for bound constrained quadratic programming problems. Optimization Methods and Software, 5, 235-245.
35. Nash, S.G. and Sofer, A., 1990, Assessing a search direction within a truncated-Newton method. Operations Research Letter, 9, 219-221.
36. Roma, M., 2002, Dynamic scaling based preconditioning for truncated Newton methods in large scale unconstrained optimization: the complete results. Technical Report R. 579, IASI-CNR. Revised 2003. Available at http://www.dis.uniroma1.it/~roma/R579.pdf.
37. Wang, W. and O'Leary, D.P., 2000, Adaptative use of iterative methods in predictor-corrector interior point methods for linear programming. Numerical Algorithms, 25, 387-406.