Sequential model-based parameter optimization: An experimental investigation of automated and interactive approaches

Experimental Methods for the Analysis of Optimization Algorithms

Volumn , Issue , 2010, Pages 363-414

  Hutter, Frank ^a   Bartz Beielstein, Thomas ^b   Hoos, Holger H ^a   Leyton Brown, Kevin ^a   Murphy, Kevin P ^a  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^b Gummersbach Campus   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84857604154     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1007/978-3-642-02538-9_15     Document Type: Chapter

References (56)

1. Adenso-Diaz B, Laguna M (2006) Fine-tuning of algorithms using fractional experimental design and local search. Operations Research 54(1):99-114.
2. Alexandrov NM, Lewis RM, Gumbert CR, Green LL, Newman PA (2001) Approximation and model management in aerodynamic optimization with variablefidelity models. Journal of Aircraft 38(6):1093-1101.
3. Audet C, Orban D (2006) Finding optimal algorithmic parameters using the mesh adaptive direct search algorithm. SIAM Journal on Optimization 17(3):642-664.
4. Balaprakash P, Birattari M, Stutzle T (2007) Improvement strategies for the frace algorithm: Sampling design and iterative refinement. In: Bartz-Beielstein T, Aguilera MJB, Blum C, Naujoks B, Roli A, Rudolph G, Sampels M (eds) 4th International Workshop on Hybrid Metaheuristics (HM'07), pp 108-122.
5. Bartz-Beielstein T (2003) Experimental analysis of evolution strategies-overview and comprehensive introduction. Interner Bericht des Sonderforschungsbereichs 531 Computational Intelligence, Universitat Dortmund, Germany.
6. Bartz-Beielstein T (2006) Experimental Research in Evolutionary Computation- The New Experimentalism. Natural Computing, Springer, Berlin, Heidelberg, New York.
7. Bartz-Beielstein T, Markon S (2004) Tuning search algorithms for real-world applications: A regression tree based approach. In: Greenwood GW (ed) Proceedings 2004 Congress on Evolutionary Computation (CEC'04), Portland OR, IEEE, Piscataway NJ, vol 1, pp 1111-1118.
8. Bartz-Beielstein T, Preuss M(2006) Considerations of budget allocation for sequential parameter optimization (SPO). In: Paquete L, et al. (eds) Workshop on Empirical Methods for the Analysis of Algorithms Proceedings, Reykjavik, Iceland, pp 35-40.
9. Bartz-Beielstein T, Parsopoulos KE, Vrahatis MN (2004a) Analysis of particle swarm optimization using computational statistics. In: Simos TE, Tsitouras C (eds) Proceedings International Conference Numerical Analysis and Applied Mathematics (ICNAAM), Wiley-VCH, Weinheim, Germany, pp 34-37.
10. Bartz-Beielstein T, Parsopoulos KE, Vrahatis MN (2004b) Design and analysis of optimization algorithms using computational statistics. Applied Numerical Analysis and Computational Mathematics (ANACM) 1(2):413-433.
11. Bartz-Beielstein T, de Vegt M, Parsopoulos KE, Vrahatis MN (2004c) Designing particle swarm optimization with regression trees. Interner Bericht des Sonderforschungsbereichs 531 Computational Intelligence, Universitat Dortmund, Germany.
12. Bartz-Beielstein T, Lasarczyk C, Preus M (2005) Sequential parameter optimization. In: McKay B, et al. (eds) Proceedings 2005 Congress on Evolutionary Computation (CEC'05), Edinburgh, Scotland, IEEE Press, Piscataway NJ, vol 1, pp 773-780.
13. Bartz-Beielstein T, Lasarczyk C, Preuss M (2008a) Sequential parameter optimization toolbox manual version 0.5, September 2008, available at http://www.gm.fh-koeln.de/imperia/md/content/personen/lehrende/ bartz-beielstein-thomas/spotdoc.pdf.
14. Bartz-Beielstein T, Zimmer T, Konen W (2008b) Parameterselektion fur komplexe modellierungsaufgaben der wasserwirtschaft - moderne CI-verfahren zur zeitreihenanalyse. In: Mikut R, Reischl M (eds) Proc. 18th Workshop Computational Intelligence, Universitatsverlag, Karlsruhe, pp 136-150.
15. Beachkofski B, Grandhi R (2002) Improved distributed hypercube sampling. American Institute of Aeronautics and Astronautics Paper 2002-1274.
16. Beielstein T (2003) Tuning evolutionary algorithms-overview and comprehensive introduction. Interner Bericht des Sonderforschungsbereichs 531 Computational Intelligence, Universitat Dortmund, Germany.
17. Beyer HG (2001) The Theory of Evolution Strategies. Springer, Berlin, Heidelberg, New York.
18. Birattari M, Stutzle T, Paquete L, Varrentrapp K (2002) A racing algorithm for configuring metaheuristics. In: Proc. of GECCO-02, pp 11-18.
19. Box G, Behnken D (1960) Some new three level designs for the study of quantitative variables. Technometrics 2:455-475.
20. Box GEP, Draper NR (1987) Empirical Model Building and Response Surfaces. Wiley, New York NY.
21. Box GEP, Hunter WG, Hunter JS (1978) Statistics for Experimenters. Wiley, New York NY.
22. Breiman L (2001) Random forests. Machine Learning 45(1):5-32.
23. Chen J, Chen C, Kelton D (2003) Optimal computing budget allocation of indifference-zone-selection procedures, taken from http://www.cba.uc.edu/ faculty/keltonwd. Cited 6 January 2005.
24. Coy SP, Golden BL, Runger GC, Wasil EA (2001) Using experimental design to find effective parameter settings for heuristics. Journal of Heuristics 7(1):77-97.
25. Dalgaard P (2002) Introductory Statistics with R. Springer, Berlin, Heidelberg, New York.
26. Hansen N (2006) The CMA evolution strategy: A comparing review. In: Lozano J, Larranaga P, Inza I, Bengoetxea E (eds) Towards a new evolutionary computation. Advances on estimation of distribution algorithms, Springer, pp 75-102.
27. Hansen N, Kern S (2004) Evaluating the CMA evolution strategy on multimodal test functions. In: Yao X, et al. (eds) Parallel Problem Solving from Nature PPSN VIII, Springer, LNCS, vol 3242, pp 282-291.
28. Hansen N, Ostermeier A (1996) Adapting arbitrary normal mutation distributions in evolution strategies: The covariance matrix adaptation. In: Proc. of CEC-96, Morgan Kaufmann, pp 312-317.
29. Hoos HH, Stutzle T (2005) Stochastic Local Search - Foundations & Applications. Morgan Kaufmann.
30. Huang D, Allen TT, Notz WI, Zeng N (2006) Global optimization of stochastic black-box systems via sequential kriging meta-models. Journal of Global Optimization 34(3):441-466.
31. Hutter F, Tompkins DAD, Hoos HH (2002) Scaling and probabilistic smoothing: Efficient dynamic local search for SAT. In: Proc. of CP-02, pp 233-248.
32. Hutter F, Hamadi Y, Hoos HH, Leyton-Brown K (2006) Performance prediction and automated tuning of randomized and parametric algorithms. In: Proc. of CP-06, pp 213-228.
33. Hutter F, Hoos HH, Stutzle T (2007) Automatic algorithm configuration based on local search. In: Proc. of AAAI-07, pp 1152-1157.
34. Hutter F, Hoos HH, Leyton-Brown K, Murphy KP (2009a) An experimental investigation of model-based parameter optimisation: SPO and beyond. In: Proc. of GECCO-09, pp 271-278.
35. Hutter F, Hoos HH, Leyton-Brown K, Stutzle T (2009b) ParamILS: an automatic algorithm configuration framework. Journal of Artificial Intelligence Research 36:267-306.
36. Ihaka R, Gentleman R (1996) R: A language for data analysis and graphics. Journal of Computational and Graphical Statistics 5(3):299-314.
37. Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black box functions. Journal of Global Optimization 13:455-492.
38. Kleijnen JPC (2008) Design and analysis of simulation experiments. Springer, New York NY.
39. Konen W, Zimmer T, Bartz-Beielstein T (2009) Optimierte Modellierung von Fullstanden in Regenuberlaufbecken mittels CI-basierter Parameterselektion. Automatisierungstechnik 57(3):155-166.
40. Lasarczyk CWG (2007) Genetische Programmierung einer algorithmischen Chemie. PhD thesis, Technische Universitat Dortmund.
41. Leyton-Brown K (2003) Resource allocation in competitive multiagent systems. PhD thesis, Stanford University.
42. Leyton-Brown K, Nudelman E, Shoham Y (2002) Learning the empirical hardness of optimization problems: The case of combinatorial auctions. In: Proc. of CP-02.
43. Lophaven SN, Nielsen HB, Sondergaard J (2002) Aspects of the Matlab toolbox DACE. Tech. Rep. IMM-REP-2002-13, Informatics and Mathematical Modelling, Technical University of Denmark, DK-2800 Kongens Lyngby, Denmark.
44. Mayer AS, Kelley C, Miller CT (2002) Optimal design for problems involving flow and transport phenomena in saturated subsurface systems. Advances in Water Resources 12:1233-1256.
45. Mockus J, Tiesis V, Zilinskas A (1978) The application of bayesian methods for seeking the extremum. Towards Global Optimisation 2:117-129, north Holland, Amsterdam.
46. Montgomery DC (2001) Design and Analysis of Experiments, 5th edn. Wiley, New York NY.
47. Pukelsheim F (1993) Optimal Design of Experiments. Wiley, New York NY.
48. Quinonero-Candela J, Rasmussen CE, Williams CK (2007) Approximation methods for gaussian process regression. In: Large-Scale Kernel Machines, Neural Information Processing, MIT Press, Cambridge, MA, USA, pp 203-223, URL http://mitpress.mit.edu/9780262026253.
49. Rasmussen CE, Williams CKI (2006) Gaussian Processes for Machine Learning. The MIT Press.
50. Sacks J, Welch WJ, Welch TJ, Wynn HP (1989) Design and analysis of computer experiments. Statistical Science 4(4):409-423.
51. Santner TJ, Williams BJ, Notz WI (2003) The Design and Analysis of Computer Experiments. Springer Verlag, New York.
52. Schonlau M, Welch WJ, Jones DR (1998) Global versus local search in constrained optimization of computer models. In: Flournoy N, RosenbergerW, WongW(eds) New Developments and Applications in Experimental Design, vol 34, Institute of Mathematical Statistics, Hayward, California, pp 11-25.
53. Schwefel HP (1995) Evolution and Optimum Seeking Sixth-Generation Computer Technology, Wiley, New York NY.
54. Tompkins DAD, Hoos HH (2004) UBCSAT: An implementation and experimentation environment for SLS algorithms for SAT & MAX-SAT. In: Proc. of SAT-04.
55. Venables WN, Ripley BD (2002) Modern Applied Statistics with S-PLUS, 4th edn. Springer, Berlin, Heidelberg, New York.
56. Williams BJ, Santner TJ, Notz WI (2000) Sequential design of computer experiments to minimize integrated response functions. Statistica Sinica 10:1133-1152.