Stochastic optimization: A review

International Statistical Review

Volumn 70, Issue 3, 2002, Pages 315-349

  Fouskakis, Dimitris ^a   Draper, David ^b  

^a University of Bath   (United Kingdom)

^b University of California Santa Cruz   (United States)

Author keywords

Bayesian decision theory; Genetic algorithms; Global optimization; Heuristic methods; Hybrid algorithms; Local search; Maximization of expected utility; Simulated annealing; Tabu search; Variable selection

Indexed keywords

EID: 0036893886     PISSN: 03067734     EISSN: None     Source Type: Journal    
DOI: 10.1111/j.1751-5823.2002.tb00174.x     Document Type: Review

References (135)

1. Aarts, E.H.L. & Korst, J.H.M. (1989). Simulated Annealing and Boltzmann Machines. Chichester: Wiley.
2. Aarts, E. & Lenstra, J.K. (Eds.) (1997). Local Search in Combinatorial Optimization. New York: Wiley Interscience.
3. Ackley, D. (1987). A Connectionist Machine for Genetic Hill-climbing. Dordrecht: Kluwer.
4. Alander, J.T. (1992). On optimal population size of genetic algorithms. Proceedings CompEuro 92, pp. 65-70. IEEE Computer Society Press.
5. Augugliaro, A., Dusonchet, L. & Sanseverino, E.R. (1999). Genetic, simulated annealing, and tabu search algorithms: three heuristic methods for optimal reconfiguration and compensation of distribution networks. European Transactions on Electric Power, 1, 35-41.
6. Bäck, T. (1996). Evolutionary Algorithms in Theory and Practice. Oxford: Oxford University Press.
7. Baker, J.E. (1985). Adaptive selection methods for genetic algorithms. In Proceedings of an International Conference on Genetic Algorithms and Their Applications, Ed. L. Davis. Hillsdale, NJ: Lawrence Erlbaum Associates.
8. Beaty, S. (1991). Instruction Scheduling using Genetic Algorithms. Ph.D. Dissertation, Colorado State University.
9. Bergeret, F. & Besse, P. (1997). Simulated annealing, weighted simulated annealing and the genetic algorithm at work. Computational Statistics, 12, 447-465.
10. Bernardo, J.M. & Smith, A.F.M. (1994). Bayesian Theory. New York: Wiley.
11. Beyer, D. & Ogier, R. (1991). Tabu learning: a neural network search method for solving nonconvex optimization problems. Proceedings of the International Joint Conference on Neural Networks. Singapore: IEEE and INNS.
12. Beyer, H. (2001). The Theory of Evolution Strategies. New York: Springer.
13. Bland, J.A. & Dawson, G.P. (1991). Tabu search and design optimization. Computer-Aided Design, 23, 195-202.
14. Booker, L.B. (1987). Improving search in genetic algorithms. In Genetic Algorithms and Simulated Annealing, Ed L. Davis. San Mateo, CA: Morgan Kaufmann.
15. Brandimarte, P., Conterno, R. & Laface, P. (1987). FMS production scheduling by simulated annealing. Proceedings of the 3rd International Conference on Simulation in Manufacturing, pp. 235-245.
16. Brown, P.J., Fearn, T. & Vannucci, M. (1999). The choice of variables in multivariate regression: A non-conjugate Bayesian decision theory approach. Biometrika, 86, 635-648.
17. Caruua, R.A. & Schaffer, J.D. (1988). Representation and hidden bias: Gray vs. binary coding for genetic algorithms. Proceedings of the Fifth International Conference on Machine Learning. San Mateo, CA: Morgan Kaufmann.
18. Chakrapani, J. & Skorin-Kapov, J. (1993). Massively parallel tabu search for the quadratic assignment problem. Annals of Operations Research, 41, 327-341.
19. Cohoon, J.P., Hegde, S.U., Martin, W.N. & Richards, D. (1987). Punctuated equilibria: a parallel genetic algorithm. In Genetic Algorithms and Their Applications: Proceedings of the Second International Conference on Genetic Algorithms, Ed. J.J. Grefenstette. Hillsdale, NJ: Lawrence Erlbaum Associates.
20. Committee on the Next Decade of Operations Research (1988). Operations research: the next decade. Operations Research, 36, 619-637.
21. Connolly, D.T. (1990). An improved annealing scheme for the QAE European Journal of Operational Research, 46, 93-100.
22. Cvijovic, D. & Klinowski, J. (1995). Tabu search: An approach to the multiple minima problem. Science, 267, 664-666.
23. Davis, L. (1991). Handbook of Genetic Algorithms. New York: Van Nostrand Reinhold.
24. Daley, J., Jencks, S., Draper, D., Lenhart, G., Thomas, N. & Walker, J. (1988). Predicting hospital-associated mortality for Medicare patients with stroke, pneumonia, acute myocardial infarction, and congestive heart failure. Journal of the American Medical Association, 260, 3617-3624.
25. Dasgupta, D. & Michalewicz, Z. (Eds.) (1997). Evolutionary Algorithms in Engineering Applications. New York: Springer-Verlag.
26. De Jong, K.A. (1975). An Analysis of the Behavior of a Class of Genetic Adaptive Systems. Ph.D. Dissertation, University of Michigan.
27. de Werra, D. & Hertz, A. (1989). Tabu search techniques: a tutorial and an application to neural networks. OR Spektrum, 11, 131-141.
28. Dowsland, K.A. (1993). Some experiments with simulated annealing techniques for packing problems. European Journal of Operational Research, 68, 389-399.
29. Draper, D. (1995). Inference and hierarchical modeling in the social sciences (with discussion). Journal of Educational and Behavioral Statistics, 20, 115-147, 233-239.
30. Draper, D. (1996). Hierarchical models and variable selection. Technical Report, Department of Mathematical Sciences, University of Bath, UK.
31. Draper, D. & Fouskakis, D. (2000). A case study of stochastic optimization in health policy: problem formulation and preliminary results. Journal of Global Optimization, 18, 399-416.
32. Draper, D. & Fouskakis, D. (2002). Stochastic optimization methods for cost-effective quality assessment in health. Submitted.
33. Draper, D., Kahn, K., Reinisch, E., Sherwood, M., Carney, M., Kosecoff, J., Keeler, E., Rogers, W., Savitt, H., Allen, H., Wells, K., Reboussin, D. & Brook, R. (1990). Studying the effects of the DRG-based Prospective Payment System on quality of care: Design, sampling, and fieldwork. Journal of the American Medical Association, 264, 1956-1961.
34. Eshelman, L. (1991). The CHC adaptive search algorithm: How to have safe search when engaging in nontraditional genetic recombination. In Foundations of Genetic Algorithms, Ed. G. Rawlins. San Mateo, CA: Morgan Kaufmann.
35. Fairley, A. (1991). Comparison of methods of choosing the crossover point in the genetic crossover operation. Department of Computer Science, University of Liverpool.
36. Fiechter, C.N. (1994). A parallel tabu search algorithm for large traveling salesman problems. Discrete Applied Mathematics, 51, 243-267.
37. FOGA (2002). Foundations of genetic algorithms VII. http://polaris.lcc.uma.es/foga2002/
38. Fourer, R. (1998). Software for optimization: a survey of trends in mathematical programming systems for operations research and management science. OR/MS Today, December 1998, 41-43.
39. Fouskakis, D. (2001). Stochastic optimisation methods for cost-effective quality assessment in health. PhD dissertation, Department of Mathematical Sciences, University of Bath, UK (available at www.soe.ucsc.edu/̃draper).
40. Fouskakis, D. & Draper, D. (1999). Review of Tabu Search, by F. Glover and M. Laguna, Amsterdam: Kluwer (1997). The Statistician, 48, 616-619.
41. Franconi, L. & Jennison, C. (1997). Comparison of a genetic algorithm and simulated annealing in an application to statistical image reconstruction. Statistics and Computing, 7, 193-207.
42. Fu, H.P. & Su, C.T. (2000). A comparison of search techniques for minimizing assembly time in printed wiring assembly. International Journal of Production Economics, 63, 83-98.
43. Fudo, H., Toune, S. & Genji, T. (2000). An application of reactive tabu search for service restoration in distribution systems and its comparison with the genetic algorithm and parallel simulated annealing. Electrical Engineering in Japan, 133, 71-82.
44. Ganley, J.L. & Heath, L.S. (1998). An experimental evaluation of local search heuristics for graph partitioning. Computing, 60, 121-132.
45. Gelfand, A.E., Dey, D.K. & Chang, H. (1992). Model determination using predictive distributions with implementation via sampling-based methods. In Bayesian Statistics 4, Eds. J.M. Bernardo, J.O. Berger, A.P. Dawid and A.F.M. Smith, pp. 147-167. Oxford: University Press.
46. Glass, C.A. & Potts, C.N. (1996). A comparison of local search methods for flow shop scheduling. Annals of Operations Research, 63, 489-509.
47. Glover, F. (1977). Heuristics for integer programming using surrogate constraints. Decision Sciences, 8, 156-166.
48. Glover, F. (1986). Future paths for integer programming and links to artificial intelligence. Computers and Operations Research, 13, 533-549.
49. Glover, F. (1989). Tabu search-Part I. ORSA Journal on Computing, 1, 190-206.
50. Glover, F. (1990a). Tabu search-Part II. ORSA Journal on Computing, 2, 4-32.
51. Glover, F. (1990b). Tabu search: A tutorial. Interfaces, 20, 74-94.
52. Glover, F., Glover, R. & Klingman, D. (1986). The threshold assignment algorithm. Mathematical Programming Study, 26, 12-37.
53. Glover, F. & McMillan, C. (1986). The general employee scheduling problem: an integration of management science and artificial intelligence. Computer and Operational Research, 15, 563-593.
54. Glover, F., Taillard, E. & de Werra, D. (1993). A user'S guide to tabu search. Annals of Operations Research, 41, 3-28.
55. Goldberg, D.E. (1989). Genetic Algorithms in Search, Optimization and Machine Learning. Reading, MA: Addison-Wesley.
56. Goldberg, D.E. & Deb, K. (1991). A comparative analysis of selection schemes used in genetic algorithms. In Foundations of Genetic Algorithms, Ed. G. Rawlins. San Mateo, CA: Morgan Kaufmann.
57. Goldberg, D.E. & Lingle, R. (1985). Alleles, loci and the traveling salesman problem. In Genetic Algorithms and Their Applications: Proceedings of the Second International Conference on Genetic Algorithms, Ed. J.J. Grefenstette. Hillsdale, NJ: Lawrence Erlbaum Associates.
58. Goldberg, D.E. & Richardson, J. (1987). Genetic algorithms with sharing for multimodal function optimization. In Genetic Algorithms and Their Applications: Proceedings of the Second International Conference on Genetic Algorithms, Ed. J.J. Grefenstette. Hillsdale, NJ: Lawrence Erlbaum Associates.
59. Goldstein, H. & Spiegelhalter, D.J. (1996). League tables and their limitations: Statistical issues in comparisons of institutional performance (with discussion). Journal of the Royal Statistical Society, Series A, 159, 385-444.
60. Gray, P., Hart, W., Painton, L., Phillips, C., Trahan, M. & Wagner, J. (1997). A survey of global optimization methods. http://www.cs.sandia.gov/opt/survey/main.html
61. Greene, J.W. & Supowit, K.J. (1986). Simulated annealing without: rejected moves. IEEE Transactions on Computer-Aided Design, CAD-5, 221-228.
62. Grefenstette, J.J. (1986). Optimization of control parameters for genetic algorithms. IEEE Transactions on Systems, Man, and Cybernetics, 16, 122-128.
63. Hadorn, D., Draper, D., Rogers, W., Keeler, E. & Brook, R. (1992). Cross-validation performance of patient mortality prediction models. Statistics in Medicine, 11, 475-489.
64. Hansen, P. (1986). The steepest ascent mildest descent heuristic for combinatorial programming. Congress on Numerical Methods in Combinatorial Optimization. Capri, Italy.
65. Hansen, P. & Jaumard, B. (1990). Algorithms for the maximum satisfiability problem. Computing, 44, 279-303.
66. Hertz, A. & de Werra, D. (1987). Using tabu search techniques for graph coloring. Computing, 29, 345-351.
67. Holland, J.H. (1975). Adaptation in Natural and Artificial Systems. Ann Arbor: University of Michigan Press.
68. Hosmer, D.W. & Lemeshow, S. (1989). Applied Logistic Regression. New York: Wiley.
69. Ishibuchi, H., Misaki, S. & Tanaka, H. (1995). Modified simulated annealing algorithms for the flow shop sequencing problem. European Journal of Operational Research, 81, 388-398.
70. Jaumard, B., Hansen, P. & Poggi di Aragao, M. (1991). Column generation methods for probabilistic logic. ORSA Journal on Computing, 3, 135-148.
71. Jencks, S., Daley, J., Draper, D., Thomas, N., Lenhart, G. & Walker, J. (1988). Interpreting hospital mortality data: The role of clinical risk adjustment. Journal of the American Medical Association, 260, 3611-3616.
72. Jog, P., Suh, J.Y. & Gucht, D.V. (1989). The effects of population size, heuristic crossover and local improvement on a genetic algorithm for the traveling; salesman problem. In Proceedings of the Third International Conference on Genetic Algorithms, Ed. J.D. Schaffer. San Mateo, CA: Morgan Kaufmann.
73. Johnson, D.S., Aragon, C.R., McGeoch, L.A. & Schevon, C. (1989). Optimization by simulated annealing: an experimental evaluation; part I, graph partitioning. Operational Research, 37, 865-892.
74. Kahn, K., Brook, R., Draper, D., Keeler, E., Rubenstein, L., Rogers, W. & Kosecoff, I. (1988). Interpreting hospital mortality data: How can we proceed? Journal of the American Medical Association, 260, 3625-3628.
75. Kahn, K., Rogers, W., Rubenstein, L., Sherwood, M., Reinisch, E., Keeler, E., Draper, D., Kosecoff, J. & Brook, R. (1990). Measuring quality of care with explicit process criteria before and after implementation of the DRG-based Prospective Payment System. Journal of the American Medical Association, 264, 1969-1973.
76. Kahn, K., Rubenstein, L., Draper, D., Kosecoff, J., Rogers, W., Keeler, E. & Brook, R. (1990). The effects of the DRG-based Prospective Payment System on quality of care for hospitalized Medicare patients: An introduction to the series. Journal of the American Medical Association, 264, 1953-1955.
77. Kapsalis, A., Smith, G.D. & Rayward-Smith, V.J. (1993). Solving the graphical Steiner tree problem using genetic algorithms. Journal of the Operational Research Society, 44, 397-406.
78. Keeler, E., Kahn, K., Draper, D., Sherwood, M., Rubenstein, L., Reinisch, E., Kosecoff, J. & Brook, R. (1990). Changes in sickness at admission following the introduction of the Prospective Payment System. Journal of the American Medical Association, 264, 1962-1968.
79. Kirkpatrick, S., Gelatt, C.D. & Vecchi, M.P. (1983). Optimization by simulated annealing. Science, 220, 671-680.
80. Kitano, H. (1990). Designing neural networks using genetic algorithms with graph generation system. Complex Systems, 4, 461-476.
81. Laguna, M., Barnes, J.W. & Glover, F. (1991). Tabu search methods for a single machine scheduling problem. Journal of Intelligent Manufacturing, 2, 63-74.
82. Laguna, M. & Glover, F. (1993). Bandwidth packing: A tabu search approach. Management Science, 39, 492-500.
83. Laguna, M. & Gonzalez-Velarde, J.L. (1991). A search heuristic for just-in-time scheduling in parallel machines. Journal of Intelligent Manufacturing, 2, 253-260.
84. Lindley, D.V. (1968). The choice of variables in multiple regression (with discussion). Journal of the Royal Statistical Society, Series B, 30, 31-66.
85. Lundy, M. & Mees, A. (1986). Convergence of an annealing algorithm. Mathematical Programming, 34, 111-124.
86. Malek, M., Guruswamy, M., Pandya, M. & Owens, H. (1989). Serial and parallel simulated annealing and tabu search algorithms for the traveling salesman problem. Annals of Operations Research, 21, 59-84.
87. Manoharan, S. & Shanmuganathan, S. (1999). A comparison of search mechanisms for structural optimization. Computing Structures, 73, 363-372.
88. Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H. & Teller, E. (1953). Equation of state calculation by fast computing machines. Journal of Chemical Physics, 21, 1087-1091.
89. Meyer, T.P. & Packard, N.H. (1992). Local forecasting of high-dimensional chaotic dynamics. In Nonlinear Modeling and Forecasting, Eds. M. Casdagli and S. Eubank. Reading, MA: Addison-Wesley.
90. Michalewicz, Z. & Janikow, C.Z. (1991). Genetic Algorithms for numerical optimization. Statistics and Computing, 1, 75-91.
91. Montana, D.J. & Davies, L.D. (1989). Training feedforward networks using genetic algorithms. Proceedings of the International Joint Conference on Artificial Intelligence. San Mateo, CA: Morgan Kaufmann.
92. Mühlenbein, H., Gorges-Schleuter, M. & Krämer, O. (1988). Evolution algorithms in combinatorial optimization. Parallel Computing, 7, 65-85.
93. Murata, T., Ishibuchi, H. & Tanaka, H. (1996). Genetic algorithms for flow-shop scheduling problems. Computers and Industrial Engineering, 40, 1061-1071.
94. Neumaier, A. (2002). Global optimization, http://solon.cma.univie.ac.at/̃neum/glopt.html.
95. Ogbu, F.A. & Smith, D.K. (1990). The application of the simulated annealing algorithm to the solution of the n/m/Cmax flowshop problem. Computers and Operational Research, 17, 243-253.
96. Oliveira, S. & Stroud, G. (1989). A parallel version of tabu search and the path assignment problem. Heuristics for Combinatorial Optimization, 4, 1-24.
97. Pettey, C.B., Leuze, M.R. & Grefenstette, J.J. (1987). A parallel genetic algorithm, in Genetic Algorithms and Their Applications: Proceedings of the Second International Conference on Genetic Algorithms, Ed. J.J. Grefenstette. Hillsdale, NJ: Lawrence Erlbaum Associates.
98. Pintér, J. (1996). Continuous global optimization software: a brief review. Optima, 52, 1-8.
99. Rawlins, G.J.E. (1991). Foundations of Genetic Algorithms. San Mateo, CA: Morgan Kaufmann.
100. Reeves, C.R. (1993). Improving the efficiency of tabu search in machine sequencing problems. Journal of the Operations Research Society, 44, 375-382.
101. Reeves, C.R. (1995a). A genetic algorithm for flowshop sequencing. Computers and Operations Research, 22, 5-13.
102. Reeves, C.R. (1995b). Modern Heuristic Techniques for Combinatorial Problems. London: McGraw-Hill.
103. Reeves, C.R. (1997). Genetic algorithms for the operations researcher. Informs Journal on Computing, 9, 231-250.
104. Reeves, C.R. (1999). Landscapes, operators and heuristic search. Annals of Operations Research, 86, 473-490.
105. Reeves, C.R. & Yamada, T. (1998). Genetic algorithms, path relinking and the flowshop sequencing problem. Evolutionary Computation, 6, 45-60.
106. Reidys, C. & Stadler, P. (2002). Combinatorial landscapes. SIAM Review, 44, 3-54.
107. Schaffer, J.D., Caruana, R.A., Eshelman, L.J. & Das, R. (1989). A study of control parameters affecting online performance of genetic algorithms for function optimization. In Proceedings of the Third International Conference on Genetic Algorithms, Ed. J.D. Schaffer. San Mateo, CA: Morgan Kaufmann.
108. Schultz-Kremer, S. (1992). Genetic algorithms for protein tertiary structure prediction. In Parallel Problem Solving from Nature 2, Eds. E. Männer and B. Manderick. North-Holland.
109. Sechen, C., Braun, D. & Sangiovanni-Vincetelli, A. (1988). Thunderbird: A complete standard cell layout package. IEEE Journal of Solid-State Circuits, SC-23, 410-420.
110. Semet, F. & Taillard, E. (1993). Solving real-life vehicle routing problems efficiently using tabu search. Annals of Operations Research, 41, 469-488.
111. Sinclair, M. (1993). A comparison of the performance of modem heuristics for combinatorial optimization on real data. Computing and Operations Research, 20, 687-695.
112. Sirag, D.J. & Weisser, P.T. (1987). Towards a unified thermodynamic genetic operator. In Genetic Algorithms and Their Applications: Proceedings of the Second International Conference on Genetic Algorithms, Ed. J.J. Grefenstette. Hillsdale, NJ: Lawrence Erlbaum Associates.
113. Skorin-Kapov, J. (1990). Tabu search applied to the quadratic assignment problem. ORSA Journal on Computing, 2, 33-45.
114. Smith, K. (1999). Neural networks for combinatorial optimization: a review of more than a decade of research. http://citeseer.nj.nec.com/252443.html.
115. South, M.C., Wetherill, G.B. & Tham, M.T. (1993). Hitchhiker'S guide to genetic algorithm. Journal of Applied Statistics, 20, 153-175.
116. Stander, J. & Silverman, B.W. (1994). Temperature schedules for simulated annealing. Statistics and Computing, 4, 21-32.
117. Suh, J.Y. & Van Gucht, D. (1987). Incorporating heuristic information into genetic search. In Genetic Algorithms and Their Applications: Proceedings of the Second International Conference on Genetic Algorithms, Ed. J.J. Grefenstette. Hillsdale, NJ: Lawrence Erlbaum Associates.
118. Syswerda, G. (1989). Uniform crossover in genetic algorithms. In Proceedings of the Third International Conference on Genetic Algorithms, Ed. J.D. Schaffer. San Mateo, CA: Morgan Kaufmann.
119. Taillard, E. (1990). Some efficient heuristic methods for the flow shop sequencing problem. European Journal of Operational Research, 47, 65-74.
120. Taillard, E. (1991). Robust tabu search for the quadratic assignment problem. Parallel Computing, 17, 443-455.
121. Taillard, E. (1993). Parallel iterative search methods for vehicle-routing problems. Networks, 23, 661-673.
122. Talbi, E.G. & Bessière, P. (1991). A parallel genetic algorithm applied to the mapping problem. SIAM News, July 1991, 12-27.
123. Törn, A., Ali, M. & Viitanen, S. (1999). Stochastic global optimization: problem classes and solution techniques. Journal of Global Optimization, 14, 437-447.
124. Tovey, C.A. (1988). Simulated simulated annealing. AJMMS, 8, 389-407.
125. Ulder, N.L.J., Aarts, E.H.L., Bandelt, H.J., Van Laarhoven, P.J.M. & Pesch, E. (1991). Genetic local search algorithms for the traveling salesman problem. In Parallel Problem Solving from Nature, Eds. H.P. Schwefel and R. Manner. Berlin: Springer.
126. Vakharia, A.J. & Chang, Y.L. (1990). A simulated annealing approach to scheduling a manufacturing cell. Naval Research Logistics, 37, 559-577.
127. Van Laarhoven, P.J.M. & Aarts, E.H.L. (1988). Simulated Annealing and Boltzmann machines. Chichester: Wiley.
128. Vose, M. (1999). The Simple Genetic Algorithm. Boston, MA: MIT Press.
129. Weisberg, S. (1985). Applied Linear Regression, second edition. New York: Wiley.
130. Westhead, D.R., Clark, D.E. & Murray, C.W. (1997). A comparison of heuristic search algorithms for molecular docking. Journal of Computer-Aided Molecular Description, 11, 209-228.
131. Whitley, D. (1989). The GENITOR algorithm and selection pressure: why rank-based allocation of reproductive trials is best. In Proceedings of the Third International Conference on Genetic Algorithms, Ed. J.D. Schaffer. San Mateo, CA: Morgan Kaufmann.
132. Whitley, D. (1992). Foundations of Genetic Algorithms 2. San Marco, CA: Morgan Kaufmann.
133. Widmer, M. (1991). Job shop scheduling with tooling constraints: a tabu search approach. Journal of Operation Research Society, 42, 75-82.
134. Wolpert, D.H. & Macready, W.G. (1997). No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 1, 67-82.
135. Youssef, H., Sait, S.M. & Adiche, H. (2001). Evolutionary algorithms, simulated annealing and tabu search: a comparative study. Engineering Applications of Artificial Intelligence, 14, 167-181.