New sequential and parallel derivative-free algorithms for unconstrained minimization

SIAM Journal on Optimization

Volumn 13, Issue 1, 2003, Pages 79-96

  Garcia Palomares U M ^a   Rodriguez J F ^b  

^a UNIVERSIDAD SIMÓN BOLÍVAR   (Venezuela)

^b IUT Dr Federico Rivero Palacio   (Venezuela)

Author keywords

Derivative free algorithm; Necessary and sufficient conditions; Nonsmooth function; Parallel algorithms; Unconstrained minimization

Indexed keywords

EID: 0037289130     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/S1052623400370606     Document Type: Article

References (46)

1. L. ARMIJO, Minimization of functions having Lipschitz continuous first partial derivatives, Pacific J. Math., 16 (1966), pp. 1-3.
2. C. AUDET, A Counter-Example for the Derivative-Free Algorithm of García-Palomares and Rodríguez, personal communication, École Polytechnique de Montréal and GERAD, Département de Mathématiques et de Génie Industriel, Montreal, 2000.
3. C. AUDET AND J.E. DENNIS, Analysis of Generalized Pattern Searches, Technical report TR00-07, Department of Computational and Applied Mathematics, Rice University, Houston, TX, 2000.
4. M. AVRIEL, Nonlinear Programming: Analysis and Methods, Prentice-Hall, Englewood Cliffs, NJ, 1976.
5. D.M. BORTZ AND C.T. KELLEY, The simplex gradient and noisy optimization problems, in Computational Methods in Optimal Design and Control, J.T. Borggaard et al., eds., Birkhaüser Boston, Cambridge, MA, 1998, pp. 77-90.
6. J. BRAUNINGER, A variable metric algorithm for unconstrained minimization without evaluation of derivatives, Numer. Math., 36 (1981), pp. 359-373.
7. F.H. CLARKE, Optimization and Nonsmooth Analysis, Classics Appl. Math. 5, SIAM, Philadelphia, 1990.
8. A.R. CONN, K. SCHEINBERG, AND PH.L. TOINT, Recent progress in unconstrained nonlinear optimization without derivatives, Math. Programming, 79 (1997), pp. 397-414.
9. J.E. DENNIS, JR., AND V. TORCZON, Direct search methods on parallel machines, SIAM J. Optim., 1 (1991), pp. 448-474.
10. J.E. DENNIS AND D.J. WOODS, Optimization on microcomputers: The Nelder-Mead simplex algorithm, in New Computing Environments: Microcomputers in Large-Scale Computing, A. Wouk, ed., SIAM, Philadelphia, 1987, pp. 116-122.
11. L.C.W. DIXON, Neural networks and unconstrained optimization, in Algorithms for Continuous Optimization: The State of the Art, E. Spedicato, ed., Kluwer Academic Publishers, Norwell, MA, 1994, pp. 513-530.
12. U.M. GARCÍA-PALOMARES, Análisis y Teorema de Convergencia de un Algoritmo de Minimización sin el Cálculo de Derivadas, Acta Cient. Venezolana, 27 (1976), pp. 187-189.
13. U.M. GARCÍA-PALOMAES AND J.F. RODRÍGUEZ, Second-order Information in the Adaptive Search Exploration Algorithm, presented at the 8th AIAA/USAF/NASA/ISSMO/Symposium on Multidisciplinary Analysis and Optimization, Long Beach, CA, 2000, paper AIAA-2000-4765.
14. F. GIANESSI, General optimality conditions via a separation scheme, in Algorithms for Continuous Optimization: The State of the Art, E. Spedicato, ed., Kluwer Academic Publishers, Norwell, MA, 1994, pp. 1-23.
15. P.D. HOUGH, T.G. KOLDA, AND V.J. TORCZON, Asynchronous parallel pattern search for nonlinear optimization, SIAM J. Sci. Comput., 23 (2001), pp. 134-156.
16. L.R. HUANG AND K.F. NG, Second-order necessary and sufficient conditions in nonsmooth optimization, Math. Programming, 66 (1994), pp. 379-402.
17. S.L.S. JACOBY, J.S. KOWALIK, AND J.T. PIZZO, Iterative Methods for Nonlinear Optimization Problems, Prentice-Hall, Englewood Cliffs, NJ, 1972.
18. C.T. KELLEY, Detection and remediation of stagnation in the Nelder-Mead algorithm using a sufficient decrease condition, SIAM J. Optim., 10 (1999), pp. 43-55.
19. C.T. KELLEY, Iterative Methods for Optimization, Frontiers Appl. Math., SIAM, Philadelphia, 1999.
20. J.C. LAGARIAS, J.A. REEDS, M.H. WRIGHT, AND P.E. WRIGHT, Convergence properties of the Nelder-Mead simplex method in low dimensions, SIAM J. Optim., 9 (1998), pp. 112-147.
21. R.M. LEWIS AND V. TORCZON, Rank Ordering and Positive Basis in Pattern Search Algo rithms, Technical report TR96-71, ICASE, Langley Research Center, Hampton, VA, 1996.
22. R.M. LEWIS AND V. TORCZON, A globally convergent augmented Lagrangian pattern search algorithm for optimization with general constraints and simple bounds, SIAM J. Optim., 12 (2002), pp. 1075-1089.
23. R.M. LEWIS AND V. TORCZON, Pattern search algorithms for bound constrained minimization, SIAM J. Optim., 9 (1999), pp. 1082-1099.
24. S. LUCIDI AND M. SCIANDRONE, On the Global Convergence of Derivative Free Methods for Unconstrained Optimization, Technical report, Università di Roma "La Sapienza", Dipartimento di Informatica e Sistemistica, Rome, 1997.
25. K.I.M. McKINNON, Convergence of the Nelder-Mead simplex method to a nonstationary point, SIAM J. Optim., 9 (1998), pp. 148-158.
26. R. MIFFLIN, A superlinearly convergent algorithm for minimization without evaluating derivatives, Math. Programming, 9 (1975), pp. 100-117.
27. J.A. NELDER AND R. MEAD, A simplex method for function minimization, The Computer Journal, 7 (1965), pp. 308-313.
28. J. ORTEGA AND W.C. RHEINBOLDT, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, 1970.
29. J.-S. PANG, Newton's methods for B-differentiable equations, Math. Oper. Res., 15 (1990), pp. 311-341.
30. J.-S. PANG, S.-P. HAN, AND N. RANGARAJ, Minimization of locally Lipschitzian functions, SIAM J. Optim., 1 (1991), pp. 57-82.
31. M.J.D. POWELL, An efficient method of finding the minimum of a function of several variables without calculating derivatives, The Computer Journal, 7 (1964), pp. 155-162.
32. B.N. PSENICHNY, A method of minimizing functions without computing derivatives, Dokl. Akad. SSSR, 235 (1977), pp. 1097-1100.
33. L. QI, A. RUSZCZYŃSKI, AND R. WOMERSLEY, EDS., Computational Nonsmooth Optimization, Mathematical Programming Series B 3-1, Elsevier Science, New York, 1997.
34. S.M. ROBINSON, Local structure of feasible sets in nonlinear programming, Part III: Stability and sensitivity, Mathematical Programming Study, 30 (1987), pp. 45-66.
35. R.T. ROCKAFELLAR, Convex Analysis, Princeton Math. Ser. 28, 2nd ed., Princeton University Press, Princeton, NJ, 1972.
36. A.S. RYKOV, Simplex algorithms for unconstrained optimization, Probl. Control Inform. Theory, 12 (1983), pp. 195-208.
37. G.W. STEWART, A modification of Davidon's method to accept difference approximations of derivatives, J. ACM, 14 (1967), pp. 72-83.
38. D. STONEKING, G. BILBRO, R. TREW, P. GILMORE, AND C.T. KELLEY, Yield optimization using a GaAs process simulator coupled to a physical device model, IEEE Trans. Microwave Theory and Techniques, 40 (1992), pp. 1353-1363.
39. V. TORCZON, Multi-Directional Search: A Direct Search Algorithm for Parallel Machines, Ph.D. thesis, Department of Mathematical Sciences, Rice University, Houston, TX, 1989.
40. V. TORCZON, On the convergence of the multidirectional search algorithm, SIAM J. Optim., 1 (1991), pp. 123-145.
41. V. TORCZON, On the convergence of pattern search algorithms, SIAM J. Optim., 7 (1997), pp. 1-25.
42. P. TSENG, Fortified-descent simplicial search method: A general approach, SIAM J. Optim., 10 (1999), pp. 269-288.
43. M.N. VRAHATIS, G.S. ANDROULAKIS, AND G.E. MANOUSSAKIS, A new unconstrained optimization method for imprecise function and gradient values, J. Math. Anal. Appl., 197 (1996), pp. 586-607.
44. G.R. WALSH, Methods of Optimization, Wiley and Sons, New York, 1975.
45. P. WOLFE, On the convergence of gradient methods under descent, IBM J. Research and Development, 16 (1972), pp. 407-411.
46. B. XIAO AND P.T. HARKER, A nonsmooth Newton method for variational inequalities, I: Theory, Math. Programming, 65 (1994), pp. 151-194.