Superlinear convergence and implicit filtering

SIAM Journal on Optimization

Volumn 10, Issue 4, 2000, Pages 1149-1162

  Choi, T D ^a   Kelley, C T ^b  

^a Intelligent Information Systems   (United States)

^b NORTH CAROLINA STATE UNIVERSITY   (United States)

Author keywords

BFGS algorithm; Implicit filtering; Noisy optimization; Superlinear convergence

Indexed keywords

EID: 0034345430     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/S1052623499354096     Document Type: Article

References (53)

1. K. R. BAILEY AND B. G. FITZPATRICK, Estimation of Groundwater Flow Parameters Using Least Squares, Tech. Rep. CRSC-TR96-13, North Carolina State University, Center for Research in Scientific Computation, Raleigh, NC, April 1996.
2. K. R. BAILEY, B. G. FITZPATRICK, AND M. A. JEFFRIES, Least Squares Estimation of Hydraulic Conductivity from Field Data, Tech. Rep. CRSC-TR95-8, North Carolina State University, Center for Research in Scientific Computation, Raleigh, NC, February 1995.
3. H. T. BANKS AND H. T. TRAN, Mathematical and Experimental Modeling of Physical Processes, unpublished lecture notes for MA 573-4, Department of Mathematics, North Carolina State University, Raleigh, NC, 1997.
4. D. M. BORTZ AND C. T. KELLEY, The simplex gradient and noisy optimization problems, in Computational Methods in Optimal Design and Control, J. T. Borggaard, J. Burns, E. Cliff, and S. Schreck, eds., Progr. Systems Control Theory 24, Birkhäuser, Boston, 1998, pp. 77-90.
5. C. G. BROYDEN, Quasi-Newton methods and their application to function minimization, Math. Comp., 21 (1967), pp. 368-381.
6. C. G. BROYDEN, A new double-rank minimization algorithm, AMS Notices, 16 (1969), p. 670.
7. C. G. BROYDEN, J. E. DENNIS, JR., AND J. J. MORE, On the local and superlinear convergence of quasi-Newton methods, J. Inst. Math. Appl., 12 (1973), pp. 223-245.
8. L. J. CAMPBELL, Y. EYSSA, P. GILMORE, P. PERNAMBUCO-WISE, D. M. PARKIN, D. G. RICKEL, J. B. SCHILLIG, AND H. J. SCHNEIDER-MUNTAU, The US 100-T magnet project, Phys. B, 211 (1995), pp. 52-55.
9. R. G. CARTER, On the global convergence of trust region algorithms using inexact gradient information, SIAM J. Numer. Anal., 28 (1991), pp. 251-265.
10. R. G. CARTER, Numerical experience with a class of algorithms for nonlinear optimization using inexact function and gradient information, SIAM J. Sci. Comput., 14 (1993), pp. 368-388.
11. R. G. CARTER, Private communication, 1999.
12. R. G. CARTER, W. W. SCHROEDER, AND T. D. HARBICK, Some causes and effect of discontinuities in modeling and optimizing gas transmission networks, paper PSIG-9308, Proceedings of the Pipeline Simulation Interest Group, Pittsburgh, PA, 1993.
13. T. D. CHOI, O. J. ESLINGER, C. T. KELLEY, J. W. DAVID, AND M. ETHERIDGE, Optimization of Automotive Valve Train Components with Implicit Filtering, Tech. Rep. CRSC-TR98-44, North Carolina State University, Center for Research in Scientific Computation, Raleigh, NC, December 1998. Optim. Engrg., to appear.
14. A. R. CONN, K. SCHEINBERG, AND P. L. TOINT, On the convergence of derivative-free methods for unconstrained optimization, in Approximation Theory and Optimization: Tributes to M. J. D. Powell, A. Iserles and M. Buhmann, eds., Cambridge University Press, Cambridge, UK, 1997, pp. 83-108.
15. A. R. CONN Recent progress in unconstrained nonlinear optimization without derivatives, Math. Programming Ser. B, 79 (1997), pp. 397-414.
16. A. R. CONN AND P. L. TOINT, An Algorithm Using Quadratic Interpolation for Unconstrained Derivative-Free Optimization, Tech. Rep. 95/6, Facultès Universitaires de Namur, Namur, Belgium, 1995.
17. J. W. DAVID, C. Y. CHENG, T. D. CHOI, C. T. KELLEY, AND J. GABLONSKY, Optimal Design of High Speed Mechanical Systems, Tech. Rep. CRSC-TR97-18, North Carolina State University, Center for Research in Scientific Computation, Raleigh, NC, July 1997. Math. Model. Sci. Comput., to appear.
18. J. W. DAVID, C. T. KELLEY, AND C. Y. CHENG, Use of an implicit filtering algorithm for mechanical system parameter identification, in Modeling of CI and SI Engines, SAE Paper 960358, 1996 SAE International Congress and Exposition Conference Proceedings, Society of Automotive Engineers, Washington, DC, 1996, pp. 189-194.
19. J. E. DENNIS AND J. J. MORÉ, A characterization of superlinear convergence and its application to quasi-Newton methods, Math. Comp., 28 (1974), pp. 549-560.
20. J. .E. DENNIS, Quasi-Newton methods, motivation and theory, SIAM Rev., 19 (1977), pp. 46-89.
21. J. E. DENNIS, JR. AND R. B. SCHNABEL, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Classics Appl. Math. 16, SIAM, Philadelphia, 1996.
22. J. E. DENNIS, JR., AND V. TORCZON, Direct search methods on parallel machines, SIAM J. Optim., 1 (1991), pp. 448-474.
23. J. E. DENNIS AND H. F. WALKER, Inaccuracy in quasi-Newton methods: Local improvement theorems, in Mathematical Programming Study 22: Mathematical Programming at Oberwolfach II, North-Holland, Amsterdam, 1984, pp. 70-85.
24. A. V. FIACCO AND G. P. MCCORMICK, Nonlinear Programming: Sequential Unconstrained Minimization Techniques, Classics Appl. Math. 4, SIAM, Philadelphia, 1990.
25. R. FLETCHER, A new approach to variable metric methods, Comput. J., 13 (1970), pp. 317-322.
26. S. J. FORTUNE, D. M. GAY, B. W. KERNIGHAN, O. LANDRON, R. A. VALENZUELA, AND M. H. WRIGHT, WISE design of indoor wireless systems, IEEE Comput. Sci. Engrg., 2 (1995), pp. 58-68.
27. P. GILMORE AND C. T. KELLEY, An implicit filtering algorithm for optimization of functions with many local minima, SIAM J. Optim., 5 (1995), pp. 269-285.
28. P. GILMORE, C. T. KELLEY, C. T. MILLER, AND G. A. WILLIAMS, Implicit filtering and optimal design problems, in Proceedings of the workshop on Optimal Design and Control, Blacksburg, VA, April 8-9, 1994, J. Borggaard, J. Burkhardt, M. Gunzburger, and J. Peterson, eds., Progr. Systems Control Theory 19, Birkhäuser, Boston, 1995, pp. 159-176.
29. P. GILMORE, P. PERNAMBUCO-WISE, AND Y. EYSSA, An Optimization Code for Pulse Magnets, Tech. Rep., National High Magnetic Field Laboratory, Florida State University, Tallahassee, FL, August 1994.
30. P. A. GILMORE, S. S. BERGER, R. F. BURR, AND J. A. BURNS, Automated optimization techniques for phase change piezoelectric ink jet performance enhancement, in Proceedings 1997 International Conference on Digital Printing Technologies, Society for Imaging Science and Technology, Springfield, VA, 1997, pp. 716-721.
31. D. GOLDFARB, A family of variable metric methods derived by variational means, Math. Comp., 24 (1970), pp. 23-26.
32. R. HOOKE AND T. A. JEEVES, 'Direct search' solution of numerical and statistical problems, J. ACM, 8 (1961), pp. 212-229.
33. D. R. JONES, C. C. PERTTUNEN, AND B. E. STUCKMAN, Lipschitzian optimization without the Lipschitz constant, J. Optim. Theory Appl., 79 (1993), pp. 157-181.
34. C. T. KELLEY, Iterative Methods for Linear and Nonlinear Equations, Frontiers Appl. Math. 16, SIAM, Philadelphia, 1995.
35. C. T. KELLEY, Detection and remediation of stagnation in the Nelder-Mead algorithm using a sufficient decrease condition, SIAM J. Optim., 10 (1999), pp. 43-55.
36. C. T. KELLEY, Iterative Methods for Optimization, Frontiers Appl. Math. 18, SIAM, Philadelphia, 1999.
37. J. C. LAGARIAS, J. A. REEDS, M. H. WRIGHT, AND P. E. WRIGHT, Convergence properties of the Nelder-Mead simplex algorithm in low dimensions, SIAM J. Optim., 9 (1998), pp. 112-147.
38. S. LUCIDI AND M. SCIANDRONE, On the Global Convergence of Derivative Free Methods for Unconstrained Optimization, preprint, Università di Roma "La Sapienza," Dipartimento di Informatica e Sistemistica, Rome, Italy, 1997.
39. S. LUCIDI AND M. SCIANDRONE, A Derivative-free Algorithm for Bound Constrained Optimization, preprint, Institute di Analisi dei Sistemi ed Informatica, Consiglio Nazionale delle Richerche, Rome, Italy, 1999.
40. R. MIFFLIN, A superlinearly convergent algorithm for minimization without derivatives, Math. Programming, 9 (1975), pp. 100-117.
41. C. T. MILLER, G. A. WILLIAMS, AND C. T. KELLEY, Transformation Approaches for Simulating Flow in Variably Saturated Porous Media, Tech. Rep. CRSC-TR98-01, North Carolina State University, Center for Research in Scientific Computation, Raleigh, NC, January 1998. Water Resources Research, to appear.
42. J. A. NELDER AND R. MEAD, A simplex method for function minimization, Comput. J., 7 (1965), pp. 308-313.
43. P. PERNAMBUCO-WISE, P. GILMORE, AND Y. EYSSA, An optimization code for pulse magnets, Phys. B, to appear.
44. P. PERNAMBUCO-WISE, P. GILMORE, B. LESCH, Y. EYSSA, AND H. J. SCHNEIDER-MUNTAU, Systematic failure testing of internally reinforced magnets, IEEE Trans. Magnetics, 4 (1996), pp. 2458-2461.
45. L. F. SHAMPINE AND M. W. REICHELT, The MATLAB ODE suite, SIAM J. Sci. Comput., 18 (1997), pp. 1-22.
46. D. F. SHANNO, Conditioning of quasi-Newton methods for function minimization, Math. Comp., 24 (1970), pp. 647-657.
47. D. STONEKING, G. BILBRO, R. TREW, P. GILMORE, AND C. T. KELLEY, Yield optimization using a GaAs process simulator coupled to a physical device model, IEEE Trans. Microwave Theory and Techniques, 40 (1992), pp. 1353-1363.
48. D. E. STONEKING, G. L. BILBRO, R. J. TREW, P. GILMORE, AND C. T. KELLEY, Yield optimization using a GaAs process simulator coupled to a physical device model, in Proceedings IEEE/Cornell Conference on Advanced Concepts in High Speed Devices and Circuits, IEEE, Piscataway, NJ, 1991, pp. 374-383.
49. V. TORCZON, Multidirectional Search, Ph.D. thesis, Rice University, Houston, TX, 1989.
50. V. TORCZON, On the convergence of the multidirectional search algorithm, SIAM J. Optim., 1 (1991), pp. 123-145.
51. T. A. WINSLOW, R. J. TREW, P. GILMORE, AND C. T. KELLEY, Doping profiles for optimum class B performance of GaAs mesfet amplifiers, in Proceedings IEEE/Cornell Conference on Advanced Concepts in High Speed Devices and Circuits, IEEE, Piscataway, NJ, 1991, pp. 188-197.
52. T. A. WINSLOW, R. J. TREW, P. GILMORE, AND C. T. KELLEY, Simulated performance optimization of GaAs MESFET amplifiers, in Proceedings IEEE/Cornell Conference on Advanced Concepts in High Speed Devices and Circuits, IEEE, Piscataway, NJ, 1991, pp. 393-402.
53. S. K. ZAVRIEV, On the global optimization properties of finite-difference local descent algorithms, J. Global Optim., 3 (1993), pp. 67-78.