Optimization by direct search: New perspectives on some classical and modern methods

SIAM Review

Volumn 45, Issue 3, 2003, Pages 385-482

  Kolda, Tamara G ^a   Lewis, Robert Michael ^b   Torczon, Virginia ^b  

^a SANDIA NATIONAL LABORATORIES   (United States)

^b The College of William and Mary   (United States)

Author keywords

Direct search; Generating set search; Global convergence analysis; Local convergence analysis; Nonlinear optimization; Nonlinear programming; Pattern search; Positive bases; Simplex search

Indexed keywords

CONSTRAINT THEORY; CONVERGENCE OF NUMERICAL METHODS; DISTRIBUTED COMPUTER SYSTEMS; GLOBAL OPTIMIZATION; PARALLEL PROCESSING SYSTEMS; SEARCH ENGINES; SET THEORY;

DIRECT SEARCH; GENERATING SET SEARCH; GLOBAL CONVERGENCE ANALYSIS; LOCAL CONVERGENCE ANALYSIS; NONLINEAR OPTIMIZATION; PATTERN SEARCH; SIMPLEX SEARCH;

NONLINEAR PROGRAMMING;

EID: 0242667172     PISSN: 00361445     EISSN: None     Source Type: Journal    
DOI: 10.1137/S003614450242889     Document Type: Article

References (282)

1. E. R. Åberg and A. G. T. Gustavsson, Design and evaluation of modified simplex methods, Analytica Chimica Acta, 144 (1982), pp. 39-53.
2. M. A. Abramson, Pattern Search Algorithms for Mixed Variable General Constrained Optimization Problems, Ph.D. thesis, Department of Computational and Applied Mathematics, Rice University, Houston, TX, 2002.
3. M. A. Abramson, C. Audet, and J. E. Dennis, Jr., Generalized Pattern Searches with Derivative Information, Tech. Rep. 02-10, Department of Computational and Applied Mathematics, Rice University, Houston, TX, 2002.
4. J. W. Akitt, Function minimisation using the Nelder and Mead simplex method with limited arithmetic precision: The self regenerative simplex, Comput. J., 20 (1977), pp. 84-85.
5. P. Alberto, F. Nogueira, H. Rocha, and L. N. Vicente, Pattern Search Methods for User-Provided Points: Application to Molecular Geometry Problems, Tech. Rep. 00-20, Departamento de Matemática, Universidade de Coimbra, Portugal, 2000 (revised 2002).
6. E. J. Anderson and M. C. Ferris, A direct search algorithm for optimization with noisy function evaluations, SIAM J. Optim., 11 (2001), pp. 837-857.
7. H. L. Anderson, W. C. Davidon, M. G. Glicksman, and U. E. Kruse, Scattering of positive pions by hydrogen at 189 MeV, Phys. Rev., 100 (1955), pp. 279-287.
8. Anonymous, A new algorithm for optimization, Math. Programming, 3 (1972), pp. 124-128.
9. L. Armijo, Minimization of functions having Lipschitz continuous first partial derivatives, Pacific J. Math., 16 (1966), pp. 1-3.
10. C. Audet, Convergence Results for Pattern Search Algorithms Are Tight, Tech. Rep. G-2002-56, Les Cahiers du GERAD, École Polytechnique de Montréal, Département de Mathématiques et de Génie Industriel, Montréal, QC, Canada, 2002.
11. C. Audet and J. E. Dennis, Jr., Pattern search algorithms for mixed variable programming, SIAM J. Optim., 11 (2000), pp. 573-594.
12. C. Audet and J. E. Dennis Jr., A Pattern Search Filter Method for Nonlinear Programming without Derivatives, Tech. Rep. 00-09, Department of Computational and Applied Mathematics, Rice University, Houston, TX, 2000.
13. C. Audet and J. E. Dennis, Jr., Analysis of generalized pattern searches, SIAM J. Optim., 13 (2003), pp. 889-903.
14. D. M. Avis and K. Fukuda, A pivoting algorithm for convex hulls and vertex enumeration of arrangements and polyhedra, Discrete Comput. Geom., 8 (1992), pp. 295-313.
15. M. Avriel, Nonlinear Programming: Analysis and Methods, Prentice-Hall, Englewood Cliffs, NJ, 1976.
16. J. W. Bandler and P. A. McDonald, Optimization of microwave networks by razor search, IEEE Trans. Microwave Theory Tech., 17 (1969), pp. 552-562.
17. G. K. Barnes, A Comparative Study of Nonlinear Optimization Codes, Master's thesis, University of Texas at Austin, 1967.
18. R. A. Barneson, N. F. Brannock, J. G. Moore, and C. Morris, Picking optimization methods, Chemical Engrg., 77 (1970), pp. 132-142.
19. M. Bell and M. C. Pike, Remark on algorithm 178, Comm. ACM, 9 (1966), pp. 684-685.
20. P. R. Benyon, Remark AS R15. Function minimization using a simplex procedure, Appl. Statist., 25 (1976), p. 97.
21. G. Berman, Minimization by successive approximation, SIAM J. Numer. Anal., 3 (1966), pp. 123-133.
22. G. Berman, Lattice approximations to the minima of functions of several variables, J. ACM, 16 (1969), pp. 286-294.
23. D. Betteridge, A. P. Wade, and A. G. Howard, Reflections on the modified simplex - I, Talanta, 32 (1985), pp. 709-722.
24. D. Betteridge, A. P. Wade, and A. G. Howard, Reflections on the modified simplex - II, Talanta, 32 (1985), pp. 723-734.
25. C. Bischof, A. Carle, P. Khademi, and A. Mauer, The ADIFOR 2.0 system for the automatic differentiation of Fortran 77 programs, IEEE Comput. Sci. Engrg., 3 (1996), pp. 18-32.
26. C. Bischof, A. Carle, P. Khademi, A. Mauer, and P. Hovland, ADIFOR 2.0 User's Guide (Revision C), Tech. Rep. CRPC-95516-S, Center for Research on Parallel Computation, Rice University, Houston, TX, 1995.
27. C. Brischof, L. Roh, and A. Mauer, ADIC - An Extensible Automatic Differentiation Tool for ANSI-C, Tech. Rep. ANL/CS-P626-1196, Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL, 1996.
28. C. H. Bischof, A. Carle, G. corliss, A. O. Griewank, and P. Hovland, ADIFOR: Generating derivative codes from Fortran programs, Sci. Programming, 1 (1992), pp. 11-29.
29. P. T. Boggs and J. E. Dennis, Jr., A stability analysis for perturbed nonlinear iterative methods, Math. Comp., 30 (1976), pp. 199-215.
30. A. J. Booker, J. E. Dennis, Jr., P. D. Frank, D. B. Serafini, and V. Torczon, Optimization using surrogate objectives on a helicopter test example, in Optimal Design and Control, J. Borggaard, J. Burns, E. Cliff, and S. Schreck, eds., Progr. Systems Control Theory, Birkhäuser, Cambridge, MA, 1998, pp. 49-58.
31. A. J. Booker, J. E. Dennis Jr., P. D. Frank, D. B. Serafini, V. Torczon, and M. W. Trosset, A rigorous framework for optimization of expensive functions by surrogates, Structural Optimization, 17 (1999), pp. 1-13.
32. J. Borggaard, D. Pelletier, and K. Vugrin, On Sensitivity Analysis for Problems with Numerical Noise, AIAA Paper 2002-5553; presented at the 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, Atlanta, GA, 2002.
33. D. M. Bortz and C. T. Kelley, The simplex gradient and noisy optimization problems, in Optimal Design and Control, J. Borggaard, J. Burns, E. Cliff, and S. Schreck, eds., Progr. Systems Control Theory, Birkhäuser, Cambridge, MA, 1998, pp. 77-90.
34. G. E. P. Box, The exploration and exploitation of response surfaces: Some general considerations and examples, Biometrics, 10 (1954), pp. 16-60.
35. G. E. P. Box, Evolutionary operation: A method for increasing industrial productivity, Appl. Statist., 6 (1957), pp. 81-101.
36. G. E. P. Box and N. R. Draper, Empirical Model-Building and Response Surfaces, Wiley, New York, 1987.
37. G. E. P. Box and N. R. Draper, Evolutionary Operation: A Statistical Method for Process Improvement, Wiley, New York, 1998.
38. G. E. P. Box and K. B. Wilson, On the experimental attainment of optimum conditions, J. Roy. Statist. Soc. Ser. B, 13 (1951), pp. 1-45.
39. M. J. Box, A new method of constrained optimization and a comparison with other methods, Comput. J., 8 (1965) pp. 42-52.
40. M. J. Box, A comparison of several current optimization methods, and the use of transformations in constrained problems, Comput. J., 9 (1966), pp. 67-77.
41. M. J. Box, D. Davies, and W. H. Swann, Non-linear Optimization Techniques, ICI Monograph 5, Oliver & Boyd, Edinburgh, UK, 1969.
42. R. P. Brent, Algorithms for Minimization without Derivatives, Prentice-Hall, Englewood Cliffs, NJ, 1973; reissued by Dover Publications, Mineaola, NY, 2002.
43. G. F. Brissey, R. B. Spencer, and C. L. Wilkins, High-speed algorithm for simplex optimization calculations, Analytical Chem., 51 (1979), pp. 2295-2297.
44. K. W. Brodlie, A new direction set method for unconstrained minimization without evaluating derivatives, J. Inst. Math. Appl., 15 (1975), pp. 385-396.
45. A. Brooke, D. Kendrick, and A. Meeraus, GAMS: A Users' Guide, The Scientific Press, Danvers, MA, 1988.
46. S. H. Brooks, A comparison of maximum-seeking methods, Oper. Res., 7 (1959), pp. 430-457.
47. S. H. Brooks and M. R. Mickey, Optimum estimation of gradient direction in steepest ascent experiments, Biometrics, 17 (1961), pp. 48-56.
48. C. G. Broyden, A class of methods for solving nonlinear simultaneous equations, Math. Comp., 19 (1965), pp. 577-593.
49. A. G. Buckley and H. Ma, A Derivative-Free Algorithm for Parallel and Sequential Optimization, tech. rep., Computer Science Department, University of Victoria, BC, Canada, 1994.
50. J. Burkardt, M. Gunzburger, and J. Peterson, Insensitive functionals, inconsistent gradients, spurious minima, and regularized functionals in flow optimization problems, Int. J. Comput. Fluid Dyn., 16 (2002), pp. 171-185.
51. K. W. C. Burton and G. Nickless, Optimisation via simplex: Part I. Background, definitions and a simple application, Chemometrics and Intelligent Laboratory Systems, 1 (1987), pp. 135-149.
52. P. H. Calamai and J. J. Moré, Projected gradient methods for linearly constrained problems, Math. Programming, 39 (1987), pp. 93-116.
53. F. M. Callier and P. L. Toint, Recent results on the accelerating property of an algorithm for function minimization without calculating derivatives, in Survey of Mathematical Programming, A. Prekopa, ed., Hungarian Academy of Sciences, Budapest, 1977, pp. 369-376.
54. C. W. Carroll, The created response surface technique for optimizing nonlinear, restrained systems, Oper. Res., 9 (1961), pp. 169-185; with "Comments on the Preceding Paper" by Anthony V. Fiacco.
55. M. R. Cave, An improved simplex algorithm for dealing with boundary conditions, Analytica Chimica Acta, 181 (1986), pp. 107-116.
56. J. Céa, Optimisation: Théorie et algorithmes, Dunod, Paris, 1971.
57. J. M. Chambers and J. E. Ertel, Remark AS R11. A remark on algorithm AS 47 'Function minimization using a simplex procedure,' Appl. Statist., 23 (1974), pp. 250-251.
58. J. P. Chandler, SIMPLEX - finds local minima of a function of several parameters (note), Behavioral Sci., 14 (1969), p. 82.
59. J. P. Chandler, STEPT: A family of routines for optimization and the fitting of data, Tech. Rep. QCPE 307, Quantum Chemistry Program Exchange, Chemistry Department, Indiana University, Bloomington, IN, 1975.
60. D. Chazan and W. Miranker, Chaotic relaxation, Linear Algebra Appl., 2 (1969), pp. 199-222.
61. D. Chazan and W. L. Miranker, A nongradient and parallel algorithm for unconstrained minimization, SIAM J. Control, 8 (1970), pp. 207-217.
62. D. H. Chen, Z. Saleem, and D. W. Grace, A new simplex procedure for function minimization, Internat. J. Modelling & Simulation, 6 (1986), pp. 81-85.
63. K.-Z. Chen and Y.-P. Wang, A New Direct Search Algorithm and Its Convergence, Department of Applied Mathematics, Northwest Telecommunication Engineering Institute, Xi'an, China.
64. F. L. Chernous'ko, A local variation method for the numerical solution of variational problems, U.S.S.R. Comput. Math. and Math. Phys., 5 (1965), pp. 234-242.
65. T. D. Choi and C. T. Kelley, Superlinear convergence and implicit filtering, SIAM J. Optim., 10 (2000), pp. 1149-1162.
66. F. H. Clarke, Optimization and Nonsmooth Analysis, Wiley, New York, 1983; reissued as Classics Appl. Math. 5, SIAM, Philadelphia, 1990.
67. A. R. Conn, N. I. M. Gould, and P. L. Toint, Global convergence of a class of trust region algorithms for optimization with simple bounds, SIAM J. Numer. Anal., 25 (1988), pp. 433-460.
68. A. R. Conn, N. I. M. Gould, and P. L. Toint, A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds, SIAM J. Numer. Anal., 28 (1991), pp. 545-572.
69. A. R. Conn, N. I. M. Gould, and P. L. Toint, Trust-Region Methods, MPS-SIAM Ser. Optim. 1, SIAM, Philadelphia, 2000.
70. A. R. Conn, K. Scheinberg, and P. L. Toint, DFO (Derivatives Free Optimization); software available at http://oss.software.ibm.com/developerworks/ opersource/coin/.
71. A. R. Conn, K. Scheinberg, and P. L. Toint, On the convergence of derivative-free methods for unconstrained optimization, in Approximation theory and Optimization: Tributes to M.J.D. Powell, M. D. Buhmann and A. Iserles, eds., Cambridge University Press, Cambridge, UK, 1997, pp. 83-108.
72. A. R. Conn, K. Scheinberg, and P. L. Toint, Recent progress in unconstrained nonlinear optimization without derivatives, in Lectures on Mathematical Programming: ismp97, T. M. Liebling and D. de Werra, eds., Mathematical Programming 79, Series B, Elsevier, Amsterdam, 1997, pp. 397-414.
73. A. R. Conn, K. Scheinberg, and P. L. Toint, A derivative free optimization algorithm in practice, in Proceedings of the 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, St. Louis, MO, 1998.
74. A. R. Conn and P. L. Toint, An algorithm using quadratic interpolation for unconstrained derivative free optimization, in Nonlinear Optimization and Applications, G. Di Pillo and F. Giannessi, eds., Kluwer Academic/Plenum Publishers, New York, 1996, pp. 27-47.
75. I. Coope and C. Price, Frame-based methods for unconstrained optimization, J. Optim. Theory Appl., 107 (2000), pp. 261-274.
76. I. D. Coope and C. J. Price, A direct search conjugate directions algorithm for unconstrained minimization, ANZIAM J., 42 (2000), pp. C478-498.
77. I. D. Coope and C. J. Price, On the convergence of grid-based methods for unconstrained optimization, SIAM J. Optim., 11 (2001), pp. 859-869.
78. I. D. Coope and C. J. Price, Positive bases in numerical optimization, Comput. Optim. Appl., 21 (2002), pp. 169-175.
79. R. J. Craig, The Simplex Search in Statistical Optimization Problems, Ph.D. thesis, University of Kentucky, Lexington, KY, 1980.
80. R. J. Craig, J. W. Evans, and D. M. Allen, The Simplex-Search in Non-linear Estimation, Tech. Rep. 155, Department of Statistics, University of Kentucky, Lexington, KY, 1980.
81. A. P. Dambrowski, Variable-step simplex optimization procedures, Tdch. Kibern., 1 (1970), p. 33.
82. W. C. Davidon, Variable Metric Method for Minimization, Tech. Rep. 5990, Argonne National Laboratory, Argonne, IL, 1959.
83. W. C. Davidon, Variable metric method for minimization, SIAM J. Optim., 1 (1991), pp. 1-17; with a belated preface for ANL 5990.
84. D. Davies and W. H. Swann, Review of constrained optimization, in Optimization, R. Fletcher, ed., Academic Press, New York, 1969, pp. 187-202.
85. O. L. Davies, The Design and Analysis of Industrial Experiments, Hafner Publishing, New York, 1954.
86. C. Davis, Theory of positive linear dependence, Amer. J. Math., 76 (1954), pp. 733-746.
87. R. De Leone, M. Gaudioso, and L. Grippo, Stopping criteria for linesearch methods without derivatives, Math. Programming, 30 (1984), pp. 285-300.
88. R. De Vogelaere, Remark on algorithm 178, Comm. ACM, 11 (1968), p. 498.
89. S. N. Deming and S. L. Morgan, Simplex optimization of variables in analytical chemistry, Analytical Chem., 45 (1973), pp. 278A-283A.
90. S. N. Deming and L. R. Parker, Jr., A review of simplex optimization in analytical chemistry, CRC Critical Rev. Analytical Chem., 7 (1978), pp. 187-202.
91. J. E. Dennis and V. Torczon, Managing approximation models in optimization, in Multidisciplinary Design Optimization: State of the Art, N. M. Alexandrov and M. Y. Hussaini, eds., SIAM, Philadelphia, 1997, pp. 330-347.
92. J. E. Dennis, Jr. and R. B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall, Englewood Cliffs, NJ, 1983; reissued as Classics Appl. Math. 16, SIAM, Philadelphia, 1996.
93. J. E. Dennis, Jr. and V. Torczon, Direct search methods on parallel machines, SIAM J. Optim., 1 (1991), pp. 448-474.
94. J. E. Dennis, Jr. and D. J. Woods, Optimization on microcomputers: The Nelder-Mead simplex algorithm, in New Computing Environments: Microcomputers in Large-Scale Computing, A. Wouk, ed., SIAM, Philadelphia, 1987, pp. 116-122.
95. L. C. W. Dixon, ACSIM - An accelerated constrained simplex technique, Comput. Aided Design, 5 (1973), pp. 22-32.
96. E. Dolan, A. Gurson, A. Shepherd, C. Sieffert, V. Torczon, and A. Yates, DirectSearch classes; software available at http://www.cs.wm.edu/~va/software/DirectSearch.
97. E. D. Dolan, Pattern Search Behavior in Nonlinear Optimization, Honors Thesis, Department of Computer Science, College of William & Mary, Williamsburg, VA, 1999.
98. E. D. Dolan, R. M. Lewis, and V. J. Torczon, On the local convergence properties of pattern search, SIAM J. Optim., to appear.
99. J. C. Dunn, Global and asymptotic convergence rate estimates for a class of projected gradient processes, SIAM J. Control Optim., 19 (1981), pp. 368-400.
100. C. Elster and A. Neumaier, A grid algorithm for bound constrained optimization of noisy functions, IMA J. Numer. Anal., 15 (1995), pp. 585-608.
101. S. V. Emel'yanov, S. K. Korovin, and A. S. Rykov, Design principles and properties of locally optimal simplicial direct-search procedures, Soviet Phys. Dokl., 24 (1979), pp. 532-534.
102. F. E. Emery and M. O'Hagan, Optimal design of matching networks for microwave transistor amplifiers, IEEE Trans. Microwave Theory Tech., 14 (1966), pp. 696-698.
103. J. W. Evans, Computer augmentation of experimental designs to maximize |X'X|, Technometrics, 21 (1979), pp. 321-330.
104. P. G. Fajer, R. L. H. Bennett, C. F. Polnaszek, E. A. Fajer, and D. D. Thomas, General method of multiparameter fitting of high-resolution EPR-spectra using a simplex algorithm, J. Magnetic Resonance, 88 (1990), pp. 111-125.
105. E. Fermi and N. Metropolis, Los Alamos Unclassified Report LS-1492, tech. rep., Los Alamos National Laboratory, Los Alamos, NM, 1952.
106. A. V. Fiacco and G. P. McCormick, Nonlinear Programming: Sequential Unconstrained Minimization Techniques, Wiley, New York, 1968; reissued as Classics Appl. Math. 4, SIAM, Philadelphia, 1990.
107. R. Fletcher, Function minimization without evaluating derivatives - A review, Comput. J., 8 (1995), pp. 33-41.
108. R. Fletcher and S. Leyffer, Nonlinear programming without a penalty function, Math. Programming, Ser. A, 91 (2002), pp. 239-269.
109. R. Fletcher, S. Leyffer, and P. L. Toint, On the global convergence of a filter-SQP algorithm, SIAM J. Optim., 13 (2002), pp. 44-59.
110. R. Fletcher and M. J. D. Powell, A rapidly convergent descent method for minimization, Comput. J., 6 (1963), pp. 163-168.
111. R. Fourer, D. M. Gay, and B. W. Kernighan, AMPL: A Modeling Language for Mathematical Programming, 2nd ed., Thomson/Brooks/Cole, Pacific Grove, CA, 2003.
112. B. Franck, Private communication, Department of Experimental and Clinical Audiology, Academic Medical Centre, Amsterdam, 2002.
113. P. Friedman and K. L. Pinder, Optimization of a simulation model of a chemical plan, Industrial & Engineering Chemistry Process Design and Development, 11 (1972), pp. 512-520.
114. J. Gablonsky and C. T. Kelley, A locally-biased form of the DIRECT algorithm, J. Global Optim., 21 (2001), pp. 27-37.
115. U. M. García-Palomares and J. F. Rodríguez, New sequential and parallel derivative-free algorithms for unconstrained minimization, SIAM J. Optim., 13 (2002), pp. 79-96.
116. U. M. García-Palomares and J. F. Rodríguez, Second-order information in the adaptive search exploration algorithm, in Proceedings of the 8th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, Long Beach, CA, 2000, AIAA Paper 2000-4765.
117. P. E. Gill, W. Murray, and M. H. Wright, Practical Optimization, Academic Press, London, 1981.
118. P. Gilmore, T. Choi, O. Eslinger, C. T. Kelley, H. A. Patrick, and J. Gablonsky, IFFCO (Implicit Filtering For Constrained Optimization); software available at http://www4.ncsu.edu/~ctk/iffco.html.
119. P. Gilmore and C. T. Kelley, An implicit filtering algorithm for optimization of functions with many local minima, SIAM J. Optim., 5 (1995), pp. 269-285.
120. P. Gilmore, C. T. Kelly, C. T. Miller, and G. A. Williams, Implicit filtering and optimal design problems, in Optimal Design and Control, J. Borggaard, J.Burkhardt, M. Gunzberger, and J. Peterson, eds., Progr. Systems Control Theory 19, Birkhäuser, Cambridge, MA, 1995, pp. 159-176.
121. T. Glad and A. Goldstein, Optimization of functions whose values are subject to small errors, BIT, 17 (1977), pp. 160-169.
122. H. Glass and L. Cooper, Sequential search: A method for solving constrained optimization problems, J. ACM, 12 (1965), pp. 71-82.
123. R. W. Glass and D. F. Bruley, REFLEX method for empirical optimization, Industrial and Engineering Chemistry Process Design and Development, 12 (1973), p. 6-10.
124. R. Glowinski and A. J. Kearsley, On the simulation and control of some friction constrained motions, SIAM J. Optim., 5 (1995), pp. 681-694.
125. A. A. Goldstein, Constructive Real Analysis, Harper & Row, New York, 1967.
126. A. O. Griewank, On automatic differentiation, in Mathematical Programming: Recent Developments and Applications, M. Iri and K. Tanabe, eds., Kluwer Academic, Dordrecht, The Netherlands, 1989, pp. 83-108.
127. L. Grippo, F. Lampariello, and S. Lucidi, Global convergence and stabilization of unconstrained minimization methods without derivatives, J. Optim. Theory Appl., 56 (1988), pp. 385-406.
128. J. A. Guin, Discussion and correspondence - modification of the complex method of constrained optima, Comput. J., 10 (1968), pp. 416-417.
129. A. P. Gurson, Simplex Search Behavior in Nonlinear Optimization, Honors Thesis, Department of Computer Science, College of William & Mary, Williamsburg, VA, 2000.
130. A. Gustavsson and J.-E. Sundkvist, Design and optimization of modified simplex methods, Analytica Chimica Acta, 167 (1985), pp. 1-10.
131. W. E. Hart, Evolutionary Pattern Search Algorithms, Tech. Rep. SAND95-2293, Sandia National Laboratories, Albuquerque, NM, 1995.
132. W. E. Hart, A stationary point convergence theory for evolutionary algorithms, in Foundations of Genetic Algorithms 4, Morgan Kaufmann, San Franscisco, CA, 1996, pp. 325-342.
133. W. E. Hart, A generalized stationary point convergence theory for evolutionary algorithms, in Proceedings of the International Conference on Genetic Algorithms, Morgan Kaufmann, San Francisco, CA, 1997, pp. 127-134.
134. P. Hedlund and A. Gustavsson, Design and evaluation of modified simplex methods having enhanced convergence ability, Analytica Chimica Acta, 259 (1992), pp. 243-256.
135. M. R. Hestenes, Multiplier and gradient methods, J. Optim. Theory Appl., 4 (1969), pp. 303-320.
136. I. D. Hill, Remark AS R28. A remark on algorithm AS 47: Function minimization using a simplex procedure, Appl. Statist., 27 (1978), pp. 380-382.
137. D. M. Himmelblau, Applied Nonlinear Programming, McGraw-Hill, New York, 1972.
138. J.-B. Hiriant-Urruty and C. Lemaréchal, Convex Analysis and Minimization Algorithms I, Springer-Verlag, Berlin, 1993.
139. R. Hooke and T. A. Jeeves, Direct search solution of numerical and statistical problems, J. ACM, 8 (1961), pp. 212-229.
140. R. Hough, T. G. Kolda, and V. J. Torczon, Asynchronous parallel pattern search for nonlinear optimization, SIAM J. Sci. Comput., 23 (2001), pp. 134-156.
141. P. D. Hough and J. C. Meza, A class of trust-region methods for parallel optimization, SIAM J. Optim., 13 (2002), pp. 264-282.
142. E C. Housos and O. Wing, Pseudo-conjugate directions for the solution of the nonlinear unconstrained optimization problem on a parallel computer, J. Optim. Theory Appl., 42 (1984).
143. P. Hovland, Automatic Differentiation and Its Role in Simulation-Based Optimization, IMA Workshop, Minneapolis, MN, 2003.
144. F. Y. Izakov, Practical example of simplex-planning for seeking the optimal regimes of technological processes, Industrial Laboratory, 37 (1971), pp. 420-422.
145. F. James and M. Roos, MINUIT - A system for function minimization and analysis of the parameter errors and correlations, Comput. Phys. Comm., 10 (1975), pp. 343-367.
146. M. G. Johnson, Nonlinear optimization using the algorithm of Hooke and Jeeves; software available in the Netlib Repository at http://www.netlib.org/hooke.c, 1994.
147. A. Jones, Spiral - A new algorithm for non-linear parameter estimation using least squares, Comput. J., 13 (1970), pp. 301-308.
148. D. R. Jones, C. D. Perttunen, and B. E. Stuckman, Lipschitzian optimization without the Lipschitz constant, J. Optim. Theory Appl., 79 (1993), pp. 15-181.
149. A. F. Kaupe Jr., Algorithm 178: Direct search, Comm. ACM, 6 (1963), pp. 313-314.
150. A. J. Keane, OPTIONS design exploration system, Computational Engineering and Design Centre, School of Engineering Sciences, Mechanical Engineering, University of Southampton, Highfield, Southampton, UK; software available at http://www.soton.ac.uk/~ajk/options/welcome.html
151. A. J. Kearsley, The Use of Optimization Techniques in the Solution of Partial Differential Equations from Science and Engineering, Ph.D. thesis, Department of Computational and Applied Mathematics, Rice University, Houston, TX, 1996; available as Tech. Rep. 96-11, Department of Computational and Applied Mathematics, Rice University, Houston, TX, 1996.
152. D. L. Keefer, Simpat: Self-bounding direct search method for optimization, Industrial and Engineering Chemistry Process Design and Development, 12 (1973), pp. 92-99.
153. C. T. Kelley, Detection and remediation of stagnation in the Nelder-Mead algorithm using a sufficient decrease condition, SIAM J. Optim., 10 (1999), pp. 43-55.
154. C. T. Kelley, Iterative Methods for Optimization, Frontiers Appl. Math. 18, SIAM, Philadelphia, 1999.
155. J. Kiefer, Optimum sequential search and approximation methods under minimum regularity assumptions, J. Soc. Indust. Appl. Math., 5 (1957), pp. 105-136.
156. P. G. King and S. N. Deming, UNIPLEX: Single-factor optimization of response in the presence of error, Analytical Chem., 46 (1974), pp. 1476-1481.
157. W. R. Klingman and D. M. Himmelblau, Nonlinear programming with the aid of a multiple-gradient summation technique, J. ACM, 11 (1964), pp. 400-415.
158. M. Kokkolaras, C. Audet, and J. E. Dennis, Jr., Mixed variable optimization of the number and composition of heat intercepts in a thermal insulation system, Optim. and Engrg., 2 (2001), pp. 5-29.
159. G. Kolda, P. D. Hough, G. Gray, S. Brown, D. Dunlavy, and H. A. Patrick, APPSPACK (Asynchronous Parallel Pattern Search package); software available at http://software.sandia.gov/appspack.
160. T. G. Kolda, R. M. Lewis, and V. J. Torczon, Stationarity results for generating set search for linearly constrained optimization, in preparation, 2003.
161. T. G. Kolda and V. J. Torczon, On the Convergence of Asynchronous Parallel Pattern Search, Tech. Rep. SAND2001-8696, Sandia National Laboratories, Livermore, CA, 2002
162. SIAM J. Optim., submitted.
163. T. G. Kolda and V. J. Torczon, Understanding asynchronous parallel pattern search, in High Performance Algorithms and Software for Nonlinear Optimization, G. DiPillo and A. Murli, eds., Kluwer Academic, Dordrecht, The Netherlands, 2003, pp. 316-335.
164. P. Krolak and L. Cooper, An extension of Fibonaccian search to several variables, Comm. ACM, 6 (1963), pp. 639-641.
165. J. C. Lagarias, J. A. Reeds, M. H. Wright and P. E. Wright, Convergence properties of the Nelder-Mead simplex method in low dimensions, SIAM J. Optim., 9 (1998), pp. 112-147.
166. J. P. Lawrence III and F. P. Emad, An adaptive randomized pattern search, in Proceedings of 1972 IEEE Conference on Decision Control and 11th Symposium on Adaptive Processes, New Orleans, 1972, pp. 421-425.
167. R. M. Lewis and V. Torczon, Rank Ordering and Positive Bases in Pattern Search Algorithms, Tech. Rep. 96-71, Institute for Computer Applications in Science and Engineering, NASA Langley Research Center, Hampton, VA, 1996.
168. R. M. Lewis and V. Torczon, Pattern search algorithms for bound constrained minimization, SIAM J. Optim., 9 (1999), pp. 1082-1099.
169. R. M. Lewis and V. Torczon, Pattern search methods for linearly constrained minimization, SIAM J. Optim., 10 (2000), pp. 917-941.
170. R. M. Lewis and V. Torczon, A globally convergent augmented Lagrangian pattern search algorithm for optimization with general constraints and simple bounds, SIAM J. Optim., 12 (2002), pp. 1075-1089.
171. R. M. Lewis, V. Torczon, and M. W. Trosset, Why pattern search works, Optima, 59 (1998), pp. 1-7; also available as ICASE Tech. Rep. 98-57, Institute for Computer Applications in Science and Engineering, NASA Langley Research Center, Hampton, VA, 1998.
172. R. M. Lewis, V. Torczon, and M. W. Trosset, Direct search methods: Then and now, J. Comput. Appl. Math., 124 (2000), pp. 191-207.
173. S. Lucidi and M. Sciandrone, Numerical results for unconstrained optimization without derivatives, in Nonlinear Optimization and Applications, G. Di Pillo and F. Giannessi, eds., Kluwer Academic/Plenum Publishers, New York, 1996, pp. 261-270.
174. S. Lucidi and M. Sciandrone, A derivative-free algorithm for bound constrained optimization, Comput. Optim. Appl., 21 (2002), pp. 119-142.
175. S. Lucidi and M. Sciandrone, On the global convergence of derivative-free methods for unconstrained optimization, SIAM J. Optim., 13 (2002), pp. 97-116.
176. S. Lucidi, M. Sciandrone, and P. Tseng, Objective-derivative-free methods for constrained optimization, Math. Program., 92 (2002), pp. 37-59.
177. R. Luus and T. H. I. Jaakola, Optimization by direct search and systematic reduction of the size of the search region, AIChE J., 19 (1973), pp. 760-766.
178. M. Marazzi and J. Nocedal, Wedge trust region methods for derivative free optimization, Math. Program., 91 (2002), pp. 289-305.
179. J. H. May, Linearly Constrained Nonlinear Programming: A Solution Method That Does Not Require Analytic Derivatives, Ph.D. thesis, Yale University, New Haven, CT, 1974.
180. K. I. M. McKinnon, Convergence of the Nelder-Mead simplex method to a nonstationary point, SIAM J. Optim., 9 (1998), pp. 148-158.
181. J. C. Meza, R. S. Judson, T. R. Faulkner, and A. M. Treasurywala, A comparison of a direct search method and a genetic algorithm for conformational searching, J. Comput. Chem., 17 (1996), pp. 1142-1151.
182. R. Mifflin, A superlinearly convergent algorithm for minimization without evaluating derivatives, Math. Programming, 9 (1975), pp. 100-117.
183. R. Mifflin and J.-J. Strodiot, A bracketing technique to ensure desirable convergence in univariate minimization, Math. Programming, 43 (1989), pp. 117-130.
184. R. Mifflin and J.-J. Strodiot, A rapidly convergent five-point algorithm for univariate minimization, Math. Programming, 62 (1993), pp. 299-319.
185. R. A. Mitchell and J. L. Kaplan, Nonlinear constrained optimization by a nonrandom complex method, J. Res. Nat. Bur. Standards, 72C (1968), pp. 249-258.
186. J. J. Moré, B. S. Garbow, and K. E. Hillstrom, Testing unconstrained optimization software, ACM Trans. Math. Software, 7 (1981), pp. 17-41.
187. J. J. Moreau, Décomposition orthgonale d'un espace hilbertien selon deux cônes mutuellement polaires, C. R. Acad. Sci. Paris, 255 (1962), pp. 238-240.
188. E. Morgan, K. W. Burton and G. Nickless, Optimization using the modified simplex method, Chemometrics and Intelligent Laboratory Systems, 7 (1990), pp. 209-222.
189. E. Morgan, K. W. Burton, and G. Nickless, Optimization using the super-modified simplex method, Chemometrics and Intelligent Laboratory Systems, 8 (1990), pp. 97-107.
190. S. L. Morgan and S. N. Deming, Simplex optimization of analytical chemical methods, Analytical Chem., 46 (1974), pp. 1170-1181.
191. R. A. Mugele, A program for optimal control of nonlinear processes, IBM Systems J., 1 (1962), pp. 2-17.
192. Multisimplex™ 2.0. Grabitech Solutions AB, Sundsvall, Sweden; software available at http://www.multisimplex.com, 2000.
193. S. G. Nash and A. Sofer, Linear and Nonlinear Programming, McGraw-Hill, New York, 1996.
194. L. Nazareth and P. Tseng, Gilding the lily: A variant of the Nelder-Mead algorithm based on golden-section search, Comput. Optim. Appl., 22 (2002), pp. 133-144.
195. J. A. Nelder and R. Mead, A simplex method for function minimization, Comput. J., 7 (1965), pp. 308-313.
196. L. S. Nelson, A sequential simplex procedure for non-linear least-squares estimation and other function minimization problems, in ASQC Technical Conference Transactions - Cleveland, 1972, American Society for Quality Control, Milwaukee, WI, 107-117.
197. J. Nocedal, Theory of algorithms for unconstrained optimization, Acta Numer., 1 (1992), pp. 199-242.
198. J. Nocedal and S. Wright, Numerical Optimization, Springer Ser. Oper. Res., Springer-Verlag, New York, 1999.
199. T. Öberg, Importance of the first design matrix in experimental simplex optimization, Chemometrics and Intelligent Laboratory Systems, 44 (1998), pp. 147-151.
200. D. M. Olsson and L. S. Nelson, The Nelder-Mead simplex procedure for function minimization, Technometrics, 17 (1975), pp. 45-51.
201. R. O'Neill, Algorithm AS 47. Function minimization using a simplex procedure, Appl. Statist., 20 (1971), pp. 338-345.
202. R. O'Neill, Corrigendum. Algorithm AS 47. Function minimization using a simplex procedure, Appl. Statist., 23 (1974), p. 252.
203. J. M. Ortega and W. C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, 1970; reissued as Classics Appl. Math. 30, SIAM, Philadelphia, 2000.
204. J. R. Palmer, An improved procedure for orthogonalising the search vectors in Rosenbrock's and Swann's direct search optimisation methods, Comput. J., 12 (1969), pp. 69-71.
205. L.-R. Parker Jr., M. R. Cave, and R. M. Barnes, Comparison of simplex algorithms, Analytica Chimica Acta, 175 (1985), pp. 231-237.
206. J. M. Parkinson and D. Huntchinson, A consideration of non-gradient algorithms for the unconstrained optimization of functions of high dimensionality, in Numerical Methods for Non-linear Optimization, F. A. Lootsma, ed., Academic Press, London, New York, 1972, pp. 99-113.
207. J. M. Parkinson and D. Hutchinson, An investigation into the efficiency of variants on the simplex method, in Numerical Methods for Non-linear Optimization, F. A. Lootsma, ed., Academic Press, London, New York, 1972, pp. 115-135.
208. D. Paviani and D. M. Himmelblau, Constrained nonlinear optimization by heuristic programming, Oper. Res., 17 (1969), pp. 872-882.
209. G. Peckham, A new method for minimising a sum of squares without calculating gradients, Comput. J., 13 (1970), pp. 418-420.
210. D. A. Phillips, A preliminary investigation of function optimisation by a combination of methods, Comput. J., 17 (1974), pp. 75-79.
211. C. Pillers Dobler, A matrix approach to finding a set of generators and finding the polar (dual) of a class of polyhedral cones, SIAM J. Matrix Anal. Appl., 15 (1994), pp. 796-803.
212. E. Polak, Computational Methods in Optimization: A Unified Approach, Academic Press, New York, 1971.
213. E. Polak and M. Wetter, Generalized Pattern Search Algorithms with Adaptive Precision Function Evaluations, Department of Electrical Engineering, University of California at Berkeley, and Simulation Research Group, Building Technologies Department, Environmental Energy Technologies Division, Lawrence Berkeley National Laboratory, Berkeley, CA, 2001.
214. M. J. D. Powell, An efficient method for finding the minimum of a function of several variables without calculating derivatives, Comput. J., 7 (1964), pp. 155-162.
215. M. J. D. Powell, A method for minimizing a sum of squares of non-linear functions without calculating derivatives, Comput. J., 7 (1965), pp. 303-307.
216. M. J. D. Powell, A method for nonlinear constraints in minimization problems, in Optimization, R. Fletcher, ed., Academic Press, New York, 1969, pp. 283-298.
217. M. J. D. Powell, A hybrid method for nonlinear equations, in Numerical Methods for Nonlinear Algebraic Equations, P. Rabinowitz, ed., Gordon & Breach, London, 1970, pp. 87-114.
218. M. J. D. Powell, On search directions for minimization algorithms, Math. Programming, 4 (1973), pp. 193-201.
219. M. J. D. Powell, Unconstrained minimization algorithms without computation of derivatives, Boll. Un. Mat. Ital. (4), 9, (1974), pp. 60-69.
220. M. J. D. Powell, A view of unconstrained minimization algorithms that do not require derivatives, ACM Trans. Math. Software, 1 (1975), pp. 97-107.
221. M. J. D. Powell, A direct search optimization method that models the objective and constraint functions by linear interpolation, in Advances in Optimization and Numerical Analysis, Proceedings of the 6th Workshop on Optimization and Numerical Analysis, Oaxaca, Mexico, S. Gomez and J.-P. Hennard, eds., Math. Appl. 275, Kluwer Academic, Dordrecht, The Netherlands, 1994, pp. 51-67.
222. M. J. D. Powell, Direct search algorithms for optimization calculations, Acta Numer., 7 (1998), pp. 287-336.
223. M. J. D. Powell, On the Lagrange Functions of Quadratic Models That Are Defined by Interpolation, Tech. Rep. DAMTP 2000/NA10, Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, UK, 2000.
224. C. J. Price and I. D. Coope, Frame based ray search algorithms in unconstrained optimization, J. Optim. Theory Appl., 116 (2003), pp. 259-377.
225. C. J. Price, I. D. Coope, and D. Byatt, A convergent variant of the Nelder Mead algorithm, J. Optim. Theory Appl., 113 (2002), pp. 5-19.
226. W. L. Price, A controlled random search procedure for global optimisation, Comput. J., 20 (1977), pp. 367-370.
227. R. T. Rockafellar, Convex Analysis, Princeton University Press, Princeton, NJ, 1970.
228. H. H. Rosenbrock, An automatic method for finding the greatest or least value of a function, Comput. J., 3 (1960), pp. 175-184.
229. M. W. Routh, P. A. Swartz, and M. B. Denton, Performance of the super modified simplex, Analytical Chem., 49 (1977), pp. 1422-1428.
230. T. H. Rowan, Functional Stability Analysis of Numerical Algorithms, Ph.D. thesis, University of Texas at Austin, 1990.
231. P. B. Ryan, R. L. Barr, and H. D. Todd, Simplex techniques for nonlinear optimization, Analytical Chem., 52 (1980), pp. 1460-1467.
232. A. S. Rykov, Simplex direct search algorithms, Automat. Remote Control, 41 (1980), pp. 784-793.
233. A. S. Rykov, Simplex methods of direct search, Engrg. Cybernetics, 18 (1980), pp. 12-18.
234. A. S. Rykov, A combined penalty-function and projection method, Soviet Phys. Dokl., 28 (1983), pp. 832-833.
235. A. S. Rykov, Simplex algorithms for unconstrained minimization, Problems Control Inform. Theory, 12 (1983), pp. 195-208.
236. R. W. H. Sargent, Minimization without constraints, in Optimization and Design, M. Avriel, M. J. Rijckaert, and D. J. Wilde, eds., Prentice-Hall, Englewood Cliffs, NJ, 1973, pp. 37-75.
237. M. H. Shao, Techniques and performance of the advanced regular simplex method, Analytica Chimica Acta, 248 (1991), pp. 31-39.
238. D. E. Shaw, R. W. M. Wedderburn, and A. Miller, MINIM: A program for function minimization using the simplex method; software available at StatLib: Data, Software and News from Statistics Community, Applied Statistics algorithms, http://lib.stat.cmu.edu/apstat/47, 1991.
239. H. Shekarforoush, M. Berthod, and J. Zerubia, Direct Search Generalized Simplex Algorithm for Optimization Non-Linear Functions, Tech. Rep. 2535, INRIA, France, 1995.
240. C. M. Siefert, Model-Assisted Pattern Search, Honors Thesis, Department of Computer Science, College of William, & Mary, Williamsburg, VA, 2000.
241. C. M. Siefert and A. Yates, MAPS (Model-Assisted Patter Search); software available at http://www.cs.wm.edu/~va/software/maps.
242. D. K. Smith, A minimization solver for many parameters, J. Quality Tech., 18 (1986), pp. 248-254.
243. L. B. Smith, Remark on algorithm 178, Comm. ACM, 12 (1969), p. 638.
244. H. A. Spang III, A review of minimization techniques for nonlinear functions, SIAM Rev., 4 (1962), pp. 343-365.
245. W. Spendley, Nonlinear least squares fittig using a modified simplex minimization method, in Optimization, R. Fletcher, ed., Academic Press, New York, 1969, pp. 259-270.
246. W. Spendley, G. R. Hext, and F. R. Himsworth, Sequential application of simplex designs in optimisation and evolutionary operation, Technometrics, 4 (1962), pp. 441-461.
247. M. Strasser, Übertangung des Optimerungsverfahrens von Nelder und Mead auf refstringierte Probleme, Diploma thesis, Numerical Mathematics Group, Technical University of Darmstadt, Germany, 1994.
248. J. C. Strikwerda, Finite Difference Schemes and Partial Differential Equations, Wadsworth & Brooks/Cole, Pacific Grove, CA, 1989.
249. N. Sugie, An extension of Fibonaccian searching to multidimensional cases, IEEE Trans. Automat. Control, 9 (1964), p. 105.
250. W. H. Swann, Report on the Development of a New Direct Search Method of Optimization, Tech. Rep. Research Note 64/3, I.C.I. Central Instrument Lab, 1964.
251. W. H. Swann, Direct search methods, in Numerical Methods for Unconstrained Optimization, W. Murray, ed., Academic Press, London, New York, 1972, pp. 13-28.
252. W. H. Swann, Constrained optimization by direct search, in Numerical Methods for Constrained Optimization, P. E. Gill and W. Murray, eds., Academic Press, London, New York, 1974, pp. 191-217.
253. S. W. Thomas, Sequential Estimation Techniques for Quasi-Newton Methods, Tech. Rep. TR75227, Department of Computer Science, Cornell University, Ithaca, NY, 1975.
254. P. L. Toint and F. M. Callier, On the accelerating property of an algorithm for function minimization without calculating derivatives, J. Optim. Theory Appl., 23 (1977), pp. 531-547; correction by F. M. Callier and Ph. L. Toint, Optim. Theory Appl., 26 (1978), pp. 465-467.
255. F. K. Tomlin and L. B. Smith, Remark on algorithm 178, Comm. ACM, 12 (1969), pp. 637-638.
256. V. Torczon, Multi-Directional Search: A Direct Search Algorithm for Parallel Machines, Ph.D. thesis, Department of Mathematical Sciences, Rice University, Houston, TX, 1989; available as Tech. Rep. 90-07, Department of Computational and Applied Mathematics, Rice University, Houston, TX, 1990.
257. V. Torczon, On the convergence of the multidirectional search algorithm, SIAM J. Optim., 1 (1991), 123-145.
258. V. Torczon, On the convergence of pattern search algorithms, SIAM J. Optim., 7 (1997), pp. 1-25.
259. V. Torczon and M. W. Trosset, From evolutionary operation to parallel direct search: Pattern search algorithms for numerical optimization, Comput. Sci. Statist., 29 (1998), pp. 396-401.
260. M. W. Trosset, I know it when I see it: Toward a definition of direct search methods, SIAG/OPT Views-and-News: A Forum for the SIAM Activity Group on Optimization, 9 (1997), pp. 7-10.
261. M. W. Trosset, On the Use of Direct Search Methods for Stochastic Optimization, Tech. Rep. 00-20, Department of Computational and Applied Mathematics, Rice University, Houston, TX, 2000.
262. M. W. Trosset and V. Torczon, Numerical Optimization Using Computer Experiments, Tech. Rep. 97-38, Institute for Computer Applications in Science and Engineering, NASA Langley Research Center, Hampton, VA, 1997; revised 1999 and available as Tech. Rep. 97-2, Department of Computational & Applied Mathematics, Rice University, Houston, TX, 1997.
263. P. Tseng, Fortified-descent simplicial search method: A general approach, SIAM J. Optim., 10 (1999), pp. 269-288.
264. P. Tseng, Convergence of block coordinate descent method for nondifferentiable minimization, J. Optim. Theory Appl., 109 (2001), pp. 475-494.
265. T. Umeda and A. Ichikawa, A modified complex method for optimization, Industrial and Engineering Chemistry Process Design and Development, 10 (1071), pp. 229-236.
266. P. F. A. van der Wiel, R. Maassen, and G. Kateman, The symmetry-controlled simplex optimization procedure, Analytica Chimica Acta, 153 (1983), pp. 83-92.
267. D. A. Walmsley, The Simplex Method for Minimisation of a General Function, Supplementary Report 686, Assessment Division, Transport Systems Department, Transport and Road Research Laboratory, Crowthorne, Berkshire, UK, 1981.
268. F. H. Walters, L. R. Parker, Jr., S. L. Morgan, and S. N. Deming, Sequential Simplex Optimization, Chemometrics Series, CRC Press, Boca Raton, FL, 1991.
269. D. Winfield, Function minimization by interpolation in a data table, J. Inst. Math. Appl., 12 (1973), pp. 339-347.
270. P. Wolfe, Convergence conditions for ascent methods, SIAM Rev., 11 (1969), pp. 226-235.
271. C. F. Wood, Review of design optimization techniques, IEEE Trans. Systems Sci. Cybernetics, 1 (1965), pp. 14-20.
272. D. J. Woods, An Interactive Approach for Solving Multi-Objective Optimization Problems, Ph.D. thesis, Department of Mathematical Sciences, Rice University, Houston, TX, 1985; also available as TR85-05, Department of Computational and Applied Mathematics, Rice University, Houston, TX, 1985.
273. M. H. Wright, Direct search methods: Once scorned, now respectable, in Numerical Analysis 1995 (Proceedings of the 1995 Dundee Biennial Conference in Numerical Analysis), D. F. Griffiths and G. A. Watson, eds., Pitman Res. Notes Math. Ser. 344, CRC Press, Boca Raton, FL, 1996, pp. 191-208.
274. M. H. Wright, What, If Anything, Is New in Optimization?, Tech. Rep. 00-4-08, Computing Sciences Research Center, Bell Laboratories, Murray Hill, NJ, 2000.
275. S. E. Wright, A note on positively spanning sets, Amer. Math. Monthly, 107 (2000), pp. 364-366.
276. Y.-X. Yuan, An example of non-convergence of trust region algorithms, in Advances in Nonlinear Programming, Y.-X. Yuan, ed., Kluwer Academic, Dordrecht, The Netherlands, 1998, pp. 205-215.
277. L. A. Yarbo and S. N. Deming, Selection and preprocessing of factors for simplex optimization, Analytica Chimica Acta, 73 (1974), pp. 391-398.
278. W. Yu, The convergence property of the simplex evolutionary techniques, Scientia Sinica, Special Issue of Mathematics, 1 (1979), pp. 68-77.
279. W. Yu, Positive basis and a class of direct search techniques, Scientia Sinica, Special Issue of Mathematics, 1 (1979), pp. 53-67.
280. W. I. Zangwill, Minimizing a function without calculating derivatives, Comput. J., 10 (1967), pp. 293-296.
281. E. H. Zarantonello, Projections on convex sets in Hilbert space and spectral theory, in Contributions to Nonlinear Functional Analysis, E. H. Zarantonello, ed., Academic Press, London, New York, 1971, pp. 237-424.
282. S. K. Zavriev, On the global optimization properties of finite-difference local descent algorithms, J. Global Optim., 3 (1993), pp. 67-78.