Derivative-free optimization: A review of algorithms and comparison of software implementations

Journal of Global Optimization

Volumn 56, Issue 3, 2013, Pages 1247-1293

  Rios, Luis Miguel ^a   Sahinidis, Nikolaos V ^a  

^a CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

Derivative free algorithms; Direct search methods; Surrogate models

Indexed keywords

DERIVATIVE-FREE ALGORITHM; DERIVATIVE-FREE OPTIMIZATION; DERIVATIVE-FREE OPTIMIZATION ALGORITHMS; DIRECT SEARCH METHODS; OBJECTIVE FUNCTION VALUES; SCIENCE AND ENGINEERING; SOFTWARE IMPLEMENTATION; SURROGATE MODEL;

CONSTRAINED OPTIMIZATION; EQUIPMENT TESTING; OPTIMAL SYSTEMS;

ALGORITHMS;

EID: 84879693300     PISSN: 09255001     EISSN: 15732916     Source Type: Journal    
DOI: 10.1007/s10898-012-9951-y     Document Type: Conference Paper

References (144)

1. Aarts E.H.L.; van Laarhoven P.J.M.: Statistical cooling: a general approach to combinatorial optimization problems. Phillips J. Res. 40, 193-226 (1985)
2. Abramson, M.A.: Pattern Search Algorithms for Mixed Variable General Constrained Optimization Problems. PhD thesis, Department of Computational and Applied Mathematics, Rice University, Houston (2002, Aug)
3. Abramson, M.A.: NOMADm Version 4.5 User's Guide. Air Force Institute of Technology, Wright-Patterson AFB, OH (2007)
4. Abramson M.A.; Asaki T.J.; Dennis J.E. Jr.; O'Reilly K.R.; Pingel R.L.: Quantitative object reconstruction via Abel-based X-ray tomography and mixed variable optimization. SIAM J. Imaging Sci. 1, 322-342 (2008)
5. Abramson M.A.; Audet C.: Convergence of mesh adaptive direct search to second-order stationary points. SIAM J. Optim. 17, 606-609 (2006)
6. Abramson, M.A.; Audet, C.; Couture, G.; Dennis, J.E. Jr.; LeDigabel, S.: The Nomad project. http://www.gerad.ca/nomad/
7. Abramson M.A.; Audet C.; Dennis J.E. Jr: Filter pattern search algorithms for mixed variable constrained optimization problems. Pac. J. Optim. 3, 477-500 (2007)
8. Abramson M.A.; Audet C.; Dennis J.E. Jr, Le Digabel S.: OrthoMADS: a deterministic MADS instance with orthogonal directions. SIAM J. Optim. 20, 948-966 (2009)
9. Audet C.: Convergence results for generalized pattern search algorithms are tight. Optim. Eng. 5, 101-122 (2004)
10. Audet C.; Béchard V.; Chaouki J.: Spent potliner treatment process optimization using a MADS algorithm. Optim. Eng. 9, 143-160 (2008)
11. Audet C.; Dennis J.E. Jr: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17, 188-217 (2006)
12. Audet C.; Dennis J.E. Jr: A progressive barrier for derivative-free nonlinear programming. SIAM J. Optim. 20, 445-472 (2009)
13. Awasthi, S.: Molecular Docking by Derivative-Free Optimization Solver. Master's thesis, Department of Chemical Engineering, Carnegie Mellon University, Pittsburgh (2008)
14. Barros P.A. Jr, Kirby M.R.; Mavris D.N.: Impact of sampling techniques selection on the creation of response surface models. SAE Trans. J. Aerosp. 113, 1682-1693 (2004)
15. Bartholomew-Biggs M.C.; Parkhurst S.C.; Wilson S.P.: Using DIRECT to solve an aircraft routing problem. Comput. Optim. Appl. 21, 311-323 (2002)
16. Barton, R.R.: Metamodeling: A state of the art review. In: Proceedings of the 1994 Winter Simulation Conference, pp. 237-244 (1994)
17. Bélisle C.J.; Romeijn H.E.; Smith R.L.: Hit-and-run algorithms for generating multivariate distributions. Math. Oper. Res. 18, 255-266 (1993)
18. Bethke, J.D.: Genetic Algorithms as Function Optimizers. PhD thesis, Department of Computer and Communication Sciences, University of Michigan, Ann Arbor (1980)
19. Björkman M.; Holmström K.: Global optimization of costly nonconvex functions using radial basis functions. Optim. Eng. 1, 373-397 (2000)
20. Boender C.G.E.; Rinnooy Kan A.H.G.; Timmer G.T.: A stochastic method for global optimization. Math. Program. 22, 125-140 (1982)
21. Boneh, A.; Golan, A.: Constraints' redundancy and feasible region boundedness by random feasible point generator (RFPG). In: 3rd European Congress on Operations Research (EURO III), Amsterdam (1979)
22. Booker, A.J.; Dennis, Jr. J.E.; Frank, P.D.; Serafini, D.B.; Torczon, V.J.; Trosset, M.W.: A rigorous framework for optimization of expensive functions by surrogates. In: ICASE Report, pp. 1-24 (1998)
23. Booker A.J.; Dennis J.E. Jr, Frank P.D.; Serafini D.B.; Torczon V.J.; Trosset M.W.: A rigorous framework for optimization of expensive functions by surrogates. Struct. Optim. 17, 1-13 (1999)
24. Booker A.J.; Meckesheimer M.; Torng T.: Reliability based design optimization using design explorer. Optim. Eng. 5, 179-205 (2004)
25. Brent R.P.: Algorithms for Minimization without Derivatives. Prentice-Hall, Englewood Cliffs (1973)
26. Chang, K.-F.: Modeling and Optimization of Polymerase Chain Reaction Using Derivative-Free Optimization. Master's thesis, Department of Chemical Engineering, Carnegie Mellon University, Pittsburgh (2011)
27. Chiang T.; Chow Y.: A limit theorem for a class of inhomogeneous Markov processes. Ann. Probab. 17, 1483-1502 (1989)
28. COIN-OR Project. Derivative Free Optimization. http://projects.coin-or. org/Dfo
29. COIN-OR Project. IPOPT 2.3.x A software package for large-scale nonlinear optimization. http://www.coin-or.org/Ipopt/ipopt-fortran.html
30. Conn A.R.; Gould N.; Lescrenier M.; Toint Ph.L.: Performance of a multifrontal scheme for partially separable optimization. In: Gomez, S.; Hennart, J.-P. (eds) Advances in Optimization and Numerical Analysis, pp. 79-96. Kluwer, Dordrecht (1994)
31. Conn A.R.; Scheinberg K.; Toint P.L.: On the convergence of derivative-free methods for unconstrained optimization. In: Buhmann, M.D.; Iserles, A. (eds) Approximation Theory and Optimization, Tribute to M. J. D. Powell, pp. 83-108. Cambridge University Press, Cambridge (1996)
32. Conn A.R.; Scheinberg K.; Toint P.L.: Recent progress in unconstrained nonlinear optimization without derivatives. Math. Program. 79, 397-414 (1997)
33. Conn, A.R.; Scheinberg, K.; Toint, P.L.: A derivative free optimization algorithm in practice. In: Proceedings of AIAA St Louis Conference, pp. 1-11 (1998)
34. Conn A.R.; Scheinberg K.; Vicente L.N.: Global convergence of general derivative-free trust-region algorithms to first and second order critical points. SIAM J. Optim. 20, 387-415 (2009)
35. Conn A.R.; Scheinberg K.; Vicente L.N.: Introduction to derivative-free optimization. SIAM, Philadelphia (2009)
36. Cox, D.D.; John, S.: SDO: A statistical method for global optimization. In: Multidisciplinary Design Optimization (Hampton, VA, 1995), pp. 315-329. SIAM, Philadelphia (1997)
37. Csendes T.; Pál L.; Sendín J.O.H.; Banga J.R.: The GLOBAL optimization method revisited. Optim. Lett. 2, 445-454 (2008)
38. Custódio A.L.; Dennis J.E. Jr, Vicente L.N.: Using simplex gradients of nonsmooth functions in direct search methods. IMA J. Numer. Anal. 28, 770-784 (2008)
39. Custódio, A.L.; Rocha, H.; Vicente, L.N.: Incorporating minimum Frobenius norm models in direct search. Comput. Optim. Appl. (to appear)
40. Custódio, A.L.; Vicente, L.N.: Using sampling and simplex derivatives in pattern search methods. SIAM J. Optim. 18, 537-555 (2007)
41. Custódio, A.L.; Vicente, L.N.: SID-PSM: A Pattern Search Method Guided by Simplex Derivatives for Use in Derivative-Free Optimization. Departamento de Matemática, Universidade de Coimbra, Coimbra (2008)
42. Deming S.N.; Parker L.R. Jr, Denton M.B.: A review of simplex optimization in analytical chemistry. Crit. Rev. Anal. Chem. 7, 187-202 (1974)
43. Desai, R.: A Comparison of Algorithms for Optimizing the Omega Function. Master's thesis, Department of Chemical Engineering, Carnegie Mellon University, Pittsburgh (2010)
44. Eberhart, R.;Kennedy, J.: A new optimizer using particle swarm theory. In: Proceedings of the 6th International Symposium on Micro Machine and Human Science, Nagoya, pp. 39-43 (1995)
45. Eldred, M.S.; Adams, B.M.; Gay, D.M.; Swiler, L.P.; Haskell, K.; Bohnhoff, W.J.; Eddy, J.P.; Hart, W.E.; Watson, J-P, Hough, P.D.; Kolda, T.G.; Williams, P.J.; Martinez-Canales, M.L.; DAKOTA, A.: Multilevel Parallel Object-Oriented Framework for Design Optimization, Parameter Estimation, Uncertainty Quantification, and Sensitivity Analysis: Version 4.2 User's Manual. Sandia National Laboratories, Albuquerque (2008)
46. Fan S.S.; Zahara E.: A hybrid simplex search and particle swarm optimization for unconstrained optimization. Eur. J. Oper. Res. 181, 527-548 (2007)
47. Finkel D.E.; Kelley C.T.: Additive scaling and the DIRECT algorithm. J. Glob. Optim. 36, 597-608 (2006)
48. Fowler K.R.; Reese J.P.; Kees C.E.; Dennis J.E. Jr, Kelley C.T.; Miller C.T.; Audet C.; Booker A.J.; Couture G.; Darwin R.W.; Farthing M.W.; Finkel D.E.; Gablonsky J.M.; Gray G.; Kolda T.G.: A comparison of derivative-free optimization methods for groundwater supply and hydraulic capture community problems. Adv. Water Resour. 31, 743-757 (2008)
49. Gablonsky, J.M.: Modifications of the DIRECT Algorithm. PhD thesis, Department of Mathematics, North Carolina State University, Raleigh (2001)
50. Gilmore P.; Kelley C.T.: An implicit filtering algorithm for optimization of functions with many local minima. SIAM J. Optim. 5, 269-285 (1995)
51. GLOBAL Library. http://www.gamsworld.org/global/globallib.htm
52. Gray G.; Kolda T.; Sale K.; Young M.: Optimizing an empirical scoring function for transmembrane protein structure determination. INFORMS J. Comput. 16, 406-418 (2004)
53. Gutmann H.-M.: A radial basis function method for global optimization. J. Glob. Optim. 19, 201-227 (2001)
54. Han J.; Kokkolaras M.; Papalambros P.Y.: Optimal design of hybrid fuel cell vehicles. J. Fuel Cell Sci. Technol. 5, 041014 (2008)
55. Hansen, N.: The CMA Evolution Strategy: A tutorial. http://www.lri.fr/ hansen/cmaesintro.html
56. Hansen N.: The CMA evolution strategy: a comparing review. In: Lozano, J.A.; Larranaga, P.; Inza, I.; Bengoetxea, E. (eds) Towards a New Evolutionary Computation. Advances on Estimation of Distribution Algorithms, pp. 75-102. Springer, Berlin (2006)
57. Hayes R.E.; Bertrand F.H.; Audet C.; Kolaczkowski S.T.: Catalytic combustion kinetics: using a direct search algorithm to evaluate kinetic parameters from light-off curves. Can. J. Chem. Eng. 81, 1192-1199 (2003)
58. Holland J.H.: Adaptation in Natural and Artificial Systems. The University of Michigan Press, Ann Arbor (1975)
59. Holmström, K.: Private Communication (2009)
60. Holmström, K.; Göran, A.O.; Edvall, M.M.: User's Guide for TOMLAB 7. Tomlab Optimization. http://tomopt.com
61. Holmström, K.; Göran, A.O.; Edvall, M.M.: User's Guide for TOMLAB/CGO. Tomlab Optimization (2007). http://tomopt.com
62. Holmström, K.; Göran, A.O.; Edvall, M.M.: User's Guide for TOMLAB/OQNLP. Tomlab Optimization (2007). http://tomopt.com
63. Holmström K.; Quttineh N.-H.; Edvall M.M.: An adaptive radial basis algorithm (ARBF) for expensive black-box mixed-integer constrained global optimization. Optim. Eng. 9, 311-339 (2008)
64. Hooke R.; Jeeves T.A.: Direct search solution of numerical and statistical problems. J. Assoc. Comput. Mach. 8, 212-219 (1961)
65. Huyer W.; Neumaier A.: Global optimization by multilevel coordinate search. J. Glob. Optim. 14, 331-355 (1999)
66. Huyer W.; Neumaier A.: SNOBFIT - Stable noisy optimization by branch and fit. ACM Trans. Math. Softw. 35, 1-25 (2008)
67. Hvattum L.M.; Glover F.: Finding local optima of high-dimensional functions using direct search methods. Eur. J. Oper. Res. 195, 31-45 (2009)
68. Ingber, L.: Adaptive Simulated Annealing (ASA). http://www.ingber.com/ #ASA
69. Järvi, T.: A Random Search Optimizer with an Application to a Max-Min Problem. Technical report, Pulications of the Institute for Applied Mathematics, University of Turku (1973)
70. Jones D.R.: A taxonomy of global optimization methods based on response surfaces. J. Glob. Optim. 21, 345-383 (2001)
71. Jones, D.R.: The DIRECT global optimization algorithm. In: Floudas, C.A.; Pardalos, P.M. (eds.) Encyclopedia of Optimization, vol. 1, pp. 431-440. Kluwer, Boston (2001)
72. Jones D.R.; Perttunen C.D.; Stuckman B.E.: Lipschitzian optimization without the Lipschitz constant. J. Optim. Theory Appl. 79, 157-181 (1993)
73. Jones D.R.; Schonlau M.; Welch W.J.: Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13, 455-492 (1998)
74. Kelley, C.T.: Users Guide for IMFIL version 1.0. http://www4.ncsu.edu/ ctk/imfil.html
75. Kelley C.T.: Detection and remediation of stagnation in the Nelder-Mead algorithm using a sufficient decrease condition. SIAM J. Optim. 10, 43-55 (1999)
76. Kelley C.T.: Iterative Methods for Optimization. SIAM, Philadelphia (1999)
77. Kennedy, J.; Eberhart, R.: Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks, Piscataway, pp. 1942-1948 (1995)
78. Kirkpatrick S.; Gelatt C.D.; Vecchi M.P.: Optimization by simulated annealing. Science 220, 671-680 (1983)
79. Kolda T.G.; Lewis R.M.; Torczon V.J.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45, 385-482 (2003)
80. Kolda T.G.; Torczon V.J.: On the convergence of asynchronous parallel pattern search. SIAM J. Optim. 14, 939-964 (2004)
81. Lagarias J.C.; Reeds J.A.; Wright M.H.; Wright P.E.: Convergence properties of the Nelder-Mead simplex method in low dimensions. SIAM J. Optim. 9, 112-147 (1998)
82. LeDigabel, S.: NOMAD User Guide Version 3.3. Technical report, Les Cahiers du GERAD (2009)
83. Lewis R.M.; Torczon V.J.: Pattern search algorithms for bound constrained minimization. SIAM J. Optim. 9, 1082-1099 (1999)
84. Lewis R.M.; Torczon V.J.: Pattern search algorithms for linearly constrained minimization. SIAM J. Optim. 10, 917-941 (2000)
85. Lewis R.M.; Torczon V.J.: A globally convergent augmented lagrangian pattern search algorithm for optimization with general constraints and simple bounds. SIAM J. Optim. 12, 1075-1089 (2002)
86. Liepins G.E.; Hilliard M.R.: Genetic algorithms: foundations and applications. Ann. Oper. Res. 21, 31-58 (1989)
87. Lin Y.; Schrage L.: The global solver in the LINDO API. Optim. Methods Softw. 24, 657-668 (2009)
88. Lucidi S.; Sciandrone M.: On the global convergence of derivative-free methods for unconstrained minimization. SIAM J. Optim. 13, 97-116 (2002)
89. Lukšan, L.; Vlček, J.: Test Problems for Nonsmooth Unconstrained and Linearly Constrained Optimization. Technical report, Institute of Computer Science, Academy of Sciences of the Czech Republic (2000). http://www3.cs.cas.cz/ics/reports/v798-00.ps
90. Marsden A.L.; Feinstein J.A.; Taylor C.A.: A computational framework for derivative-free optimization of cardiovascular geometries. Comput. Methods Appl. Mech. Eng. 197, 1890-1905 (2008)
91. Marsden A.L.; Wang M.; Dennis J.E. Jr, Moin P.: Optimal aeroacustic shape design using the surrogate management framework. Optim. Eng. 5, 235-262 (2004)
92. Marsden A.L.; Wang M.; Dennis J.E. Jr, Moin P.: Trailing-edge noise reduction using derivative-free optimization and large-eddy simulation. J. Fluid Mech. 5, 235-262 (2007)
93. Matheron G.: Principles of geostatistics. Econ. Geol. 58, 1246-1266 (1967)
94. McKinnon K.I.M.: Convergence of the Nelder-Mead simplex method to a nonstationary point. SIAM J. Optim. 9, 148-158 (1998)
95. Metropolis N.; Rosenbluth A.W.; Rosenbluth M.N.; Teller A.H.; Teller E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21, 1087-1092 (1953)
96. Mongeau M.; Karsenty H.; Rouzé V.; Hiriart-Urruty J.B.: Comparison of public-domain software for black box global optimization. Optim. Methods Softw. 13, 203-226 (2000)
97. Moré, J.; Wild, S.: Benchmarking derivative-free optimization algorithms. SIAM J. Optim. 20, 172-191 (2009)
98. Mugunthan P.; Shoemaker C.A.; Regis R.G.: Comparison of function approximation, heuristic, and derivative-based methods for automatic calibration of computationally expensive groundwater bioremediation models. Water Resour. Res. 41, W11427 (2005)
99. Nelder J.A.; Mead R.: A simplex method for function minimization. Comput. J. 7, 308-313 (1965)
100. Nesterov, Y.: Gradient methods for minimizing composite objective function. CORE Discussion Paper 2007/76 (2007)
101. Neumaier, A.: MCS: Global Optimization by Multilevel Coordinate Search. http://www.mat.univie.ac.at/neum/software/mcs/
102. Neumaier A.; Shcherbina O.; Huyer W.; Vinkó T.: A comparison of complete global optimization solvers. Math. Program. 103, 335-356 (2005)
103. Oeuvray, R.: Trust-Region Methods Based on Radial Basis Functions with Application to Biomedical Imaging. PhD thesis, Institute of Mathematics, Swiss Federal Institute of Technology, Lausanne (2005, March)
104. Orosz J.E.; Jacobson S.H.: Finite-time performance analysis of static simulated annealing algorithms. Comput. Optim. Appl. 21, 21-53 (2002)
105. Pintér, J.: Homepage of Pintér Consulting Services. http://www.pinterconsulting.com/
106. Pintér J.D.: Global Optimization in Action: Continuous and Lipschitz Optimization. Algorithms, Implementations and Applications. Nonconvex Optimization and its Applications. Kluwer, Dordrecht (1995)
107. Pintér, J.D.; Holmström, K.; Göran, A.O.; Edvall, M.M.: User's Guide for TOMLAB/LGO. Tomlab Optimization (2006). http://tomopt.com
108. Plantenga, T.D.: HOPSPACK 2.0 User Manual. Technical Report SAND2009-6265, Sandia National Laboratories, Albuquerque (2009)
109. Powell M.J.D.: A direct search optimization method that models the objective and constraint functions by linear interpolation. In: Gomez, S.; Hennart, J-P. (eds) (eds.) Advances in Optimization and Numerical Analysis, pp. 51-67. Kluwer, Dordrecht (1994)
110. Powell, M.J.D.: A direct search optimization method that models the objective by quadratic interpolation. In: Presentation at the 5th Stockholm Optimization Days (1994)
111. Powell, M.J.D.: Recent Research at Cambridge on Radial Basis Functions. Technical report, Department of Applied Mathematics and Theoretical Physics, University of Cambridge (1998)
112. Powell M.J.D.: UOBYQA: unconstrained optimization by quadratic approximation. Math. Program. 92, 555-582 (2002)
113. Powell M.J.D.: The NEWUOA software for unconstrained optimization without derivatives. In: Di Pillo, G.; Roma, M. (eds) Large-Scale Nonlinear Optimization, pp. 255-297. Springer, New York (2006)
114. Powell M.J.D.: Developments of NEWUOA for minimization without derivatives. IMA J. Numer. Anal. 28, 649-664 (2008)
115. Powell, M.J.D.: The BOBYQA Algorithm for Bound Constrained Optimization Without Derivatives. Technical report, Department of Applied Mathematics and Theoretical Physics, University of Cambridge (2009)
116. Princeton Library. http://www.gamsworld.org/performance/princetonlib/ princetonlib.htm
117. Regis R.G.; Shoemaker C.A.: Constrained global optimization of expensive black box functions using radial basis functions. J. Glob. Optim. 31, 153-171 (2005)
118. Regis R.G.; Shoemaker C.A.: Improved strategies for radial basis function methods for global optimization. J. Glob. Optim. 37, 113-135 (2007)
119. Richtarik, P.: Improved algorithms for convex minimization in relative scale. SIAM J. Optim. (2010, to appear). http://www.optimization-online.org/DB- FILE/2009/02/2226.pdf
120. Rios, L.M.: Algorithms for Derivative-Free Optimization. PhD thesis, Department of Industrial and Enterprise Systems Engineering, University of Illinois, Urbana (2009, May)
121. Romeo F.; Sangiovanni-Vincentelli A.: A theoretical framework for simulated annealing. Algorithmica 6, 302-345 (1991)
122. Sacks J.; Welch W.J.; Mitchell T.J.; Wynn H.P.: Design and analysis of computer experiments. Stat. Sci. 4, 409-423 (1989)
123. Sahinidis, N.V.; Tawarmalani, M.: BARON 7.5: Global Optimization of Mixed-Integer Nonlinear Programs, User's Manual (2005)
124. Sandia National Laboratories: The Coliny Project. https://software. sandia.gov/trac/acro/wiki/Overview/Projects
125. Scheinberg, K.: Manual for Fortran Software Package DFO v2.0 (2003)
126. Schonlau, M.: Computer Experiments and Global Optimization. PhD thesis, Department of Statistics, University of Waterloo, Waterloo (1997)
127. Serafini, D.B.: A Framework for Managing Models in Nonlinear Optimization of Computationally Expensive Functions. PhD thesis, Department of Computational and Applied Mathematics, Rice University, Houston (1998, Nov)
128. Shah, S.B.; Sahinidis, N.V.: SAS-Pro: Simultaneous residue assignment and structure superposition for protein structure alignment. PLoS ONE 7(5), e37493 (2012)
129. Shubert B.O.: A sequential method seeking the global maximum of a function. SIAM J. Numer. Anal. 9, 379-388 (1972)
130. Smith R.L.: Efficient Monte Carlo procedures for generating points uniformly distributed over bounded regions. Oper. Res. 32, 1296-1308 (1984)
131. Søndergaard, J.: Optimization Using Surrogate Models - by the Space Mapping Technique. PhD thesis, Technical University of Denmark, Department of Mathematical Modelling, Lingby (2003)
132. Spendley W.; Hext G.R.; Himsworth F.R.: Sequential application for simplex designs in optimisation and evolutionary operation. Technometrics 4, 441-461 (1962)
133. Tawarmalani M.; Sahinidis N.V.: A polyhedral branch-and-cut approach to global optimization. Math. Program. 103, 225-249 (2005)
134. Torczon V.J.: On the convergence of multidirectional search algorithms. SIAM J. Optim. 1, 123-145 (1991)
135. Torczon V.J.: On the convergence of pattern search algorithms. SIAM J. Optim. 7, 1-25 (1997)
136. Tseng P.: Fortified-descent simplicial search method: a general approach. SIAM J. Optim. 10, 269-288 (1999)
137. Vaz, A.I.F.: PSwarm Home Page. http://www.norg.uminho.pt/aivaz/pswarm/
138. Vaz A.I.F.; Vicente L.N.: A particle swarm pattern search method for bound constrained global optimization. J. Glob. Optim. 39, 197-219 (2007)
139. Wang, H.: Application of Derivative-Free Algorithms in Powder Diffraction. Master's thesis, Department of Chemical Engineering, Carnegie Mellon University, Pittsburgh (2011)
140. Wild S.M.; Regis R.G.; Shoemaker C.A.: ORBIT: Optimization by radial basis function interpolation in trust-regions. SIAM J. Sci. Comput. 30, 3197-3219 (2008)
141. Winfield, D.: Function and Functional Optimization by Interpolation in Data Tables. PhD thesis, Harvard University, Cambridge (1969)
142. Winslow, T.A.; Trew, R.J.; Gilmore, P.; Kelley, C.T.: Simulated performance optimization of gaas mesfet amplifiers. In: IEEE/Cornell Conference on Advanced Concepts in High Speed Semiconductor Devices and Circuits, Piscataway, pp. 393-402 (1991)
143. Zhao Z.; Meza J.C.; Van Hove M.: Using pattern search methods for surface structure determination of nanomaterials. J. Phys. Condens. Matter 18, 8693-8706 (2006)
144. Zheng, Y.: Pairs Trading and Portfolio Optimization. Master's thesis, Department of Chemical Engineering, Carnegie Mellon University, Pittsburgh (2011)