Performance evaluation of automatically tuned continuous optimizers on different benchmark sets

Applied Soft Computing Journal

Volumn 27, Issue , 2015, Pages 490-503

  Liao, Tianjun ^a   Molina, Daniel ^b   Stützle, Thomas ^c  

^a BEIJING INSTITUTE OF SYSTEM ENGINEERING   (China)

^b UNIVERSITY OF CÁDIZ   (Spain)

^c UNIVERSITÉ LIBRE DE BRUXELLES   (Belgium)

Author keywords

Automatic parameter tuning; Benchmark sets; Continuous optimization; Evolutionary computation; Swarm intelligence

Indexed keywords

ARTIFICIAL INTELLIGENCE; BENCHMARKING; CALCULATIONS; OPTIMIZATION; PARAMETER ESTIMATION; SOFT COMPUTING;

ALGORITHM PERFORMANCE; AUTOMATIC PARAMETER TUNING; BENCHMARKING ALGORITHM; CONTINUOUS OPTIMIZATION; CONTINUOUS OPTIMIZATION PROBLEMS; LARGE-SCALE OPTIMIZATION; REAL-PARAMETER OPTIMIZATION; SWARM INTELLIGENCE;

EVOLUTIONARY ALGORITHMS;

EID: 84919727129     PISSN: 15684946     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.asoc.2014.11.006     Document Type: Article

References (56)

1. A. Auger, N. Hansen, A restart CMA evolution strategy with increasing population size, in: Proceedings of the 2005 Congress on Evolutionary Computation (CEC 2005), IEEE Press, Piscataway, NJ, 2005, pp. 1769-1776.
2. A. Auger, N. Hansen, J.M. Perez Zerpa, R. Ros, M. Schoenauer, Experimental comparisons of derivative free optimization algorithms, in: J. Vahrenhold (Ed.), Proceedings of the International Symposium on Experimental Algorithms, SEA'09, Lecture Notes in Computer Science, Springer, Heidelberg, Germany, 2009, pp. 3-15.
3. D. Aydin, T. Liao, M.A. Montes de Oca, T. Stützle, Improving performance via population growth and local search: the case of the artificial bee colony algorithm. Technical Report TR/IRIDIA/2011-015, IRIDIA, Université Libre de Bruxelles, Belgium, 2011.
4. T. Back, D.B. Fogel, Z. Michalewicz, Handbook of Evolutionary Computation, IOP Publishing, Bristol, UK, 1997.
5. H. Bersini, M. Dorigo, S. Langerman, G. Seront, L.M. Gambardella, Results of the first international contest on evolutionary optimisation, in: T. Bäck, T. Fukuda, Z. Michalewicz (Eds.), Proceedings of the 1996 IEEE International Conference on Evolutionary Computation (ICEC'96), IEEE Press, Piscataway, NJ, 1996, pp. 611-615.
6. M. Birattari, Tuning metaheuristics: a machine learning perspective, in: Volume 197 of Studies in Computational Intelligence, Springer, Berlin/Heidelberg, Germany, 2009.
7. M. Birattari, Z. Yuan, P. Balaprakash, T. Stützle, F-race and iterated F-race: an overview, in: T. Bartz-Beielstein, M. Chiarandini, L. Paquete, M. Preuss (Eds.), Experimental Methods for the Analysis of Optimization Algorithms, Springer, Berlin, Germany, 2010, pp. 311-336.
8. B. Bischl, O. Mersmann, H. Trautmann, M. Preuss, Algorithm selection based on exploratory landscape analysis and cost-sensitive learning, in: T. Soule, J.H. Moore (Eds.), Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2012, ACM Press, New York, NY, 2012, pp. 313-320.
9. A.R. Conn, K. Scheinberg, L.N. Vicente, Introduction to derivative-free optimization, in: MPS-SIAM Series on Optimization, Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 2009.
10. W.J. Conover, Practical Nonparametric Statistics, third ed., John Wiley & Sons, New York, NY, 1999.
11. M. Dorigo, Optimization, Learning and Natural Algorithms (in Italian), PhD thesis, Dipartimento di Elettronica, Politecnico di Milano, Italy, 1992.
12. M. Dorigo, T. Stützle, Ant Colony Optimization, MIT Press, Cambridge, MA, 2004.
13. A.E. Eiben, Z. Michalewicz, M. Schoenauer, J.E. Smith, Parameter control in evolutionary algorithms, in: F. Lobo, C.F. Lima, Z. Michalewicz (Eds.), Parameter Setting in Evolutionary Algorithms, Springer, Berlin, Germany, 2007, pp. 19-46.
14. M. El-Abd, Performance assessment of foraging algorithms vs evolutionary algorithms, Inform. Sci. 182 (1) (2012) 243-263.
15. N.I.M. Gould, D. Orban, P.L. Toint, CUTEr and SifDec: a constrained and unconstrained testing environment, revisited, ACM Trans. Math. Softw. 29 (2003) 373-394.
16. N. Hansen, A. Ostermeier, Completely derandomized self-adaptation in evolution strategies, Evol. Comput. 9 (2) (2001) 159-195.
17. N. Hansen, A. Auger, S. Finck, R. Ros, Real-parameter black-box optimization benchmarking 2009: Experimental setup. Technical Report RR-6828, INRIA, France, 2009.
18. N. Hansen, A. Auger, R. Ros, S. Finck, P. Pošík, Comparing results of 31 algorithms from the black-box optimization benchmarking BBOB-2009, in: Proceedings of the Conference on Genetic and Evolutionary Computation, GECCO'10 (Companion), ACM, New York, NY, USA, 2010, pp. 1689-1696.
19. N. Hansen, R. Ros, N. Mauny, M. Schoenauer, A. Auger, Impacts of invariance in search: when CMA-ES and PSO face ill-conditioned and non-separable problems, Appl. Soft Comput. 11 (8) (2011) 5755-5769.
20. F. Herrera, M. Lozano, J. Verdegay, Tackling real-coded genetic algorithms: operators and tools for behavioural analysis, Artif. Intell. Rev. 12 (1998) 265-319.
21. F. Herrera, M. Lozano, D. Molina, Test suite for the special issue of soft computing on scalability of evolutionary algorithms and other metaheuristics for large scale continuous optimization problems, 2010 http://sci2s.ugr.es/eamhco/
22. H.H. Hoos, Programming by optimization, Commun. ACM 55 (2) (2012) 70-80.
23. F. Hutter, H.H. Hoos, K. Leyton-Brown, T. Stützle, ParamILS: an automatic algorithm configuration framework, J. Artif. Intell. Res. 36 (October) (2009) 267-306.
24. F. Hutter, H.H. Hoos, K. Leyton-Brown, Sequential model-based optimization for general algorithm configuration, in: C.A. Coello Coello (Ed.), Learning and Intelligent Optimization, 5th International Conference, LION 5, vol. 6683 of Lecture Notes in Computer Science, Springer, Heidelberg, Germany, 2011, pp. 507-523.
25. D. Karaboga, B. Basturk, A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm, J. Global Optim. 39 (2007) 459-471, 0925-5001.
26. J. Kennedy, R. Eberhart, Particle swarm optimization, in: Proceedings of IEEE International Conference on Neural Networks, vol. 4, IEEE Press, Piscataway, NJ, USA, 1995, pp. 1942-1948.
27. J. Kennedy, R.C. Eberhart, Y. Shi, Swarm Intelligence, Morgan Kaufmann Publishers, San Francisco, CA, 2001.
28. N. Krasnogor, J. Smith, A tutorial for competent memetic algorithms: model, taxonomy, and design issues, IEEE Trans. Evol. Comput. 9 (5) (2005) 474-488.
29. A. LaTorre, A framework for hybrid dynamic evolutionary algorithms: multiple offspring sampling (MOS), PhD thesis, Universidad Politécnica de Madrid, Spain, 2009.
30. A. LaTorre, S. Muelas, J.-M. Peña, A MOS-based dynamic memetic differential evolution algorithm for continuous optimization: a scalability test, Soft Comput. 15 (11) (2011) 2187-2199.
31. G. Leguizamón, C. Coello, An alternative ACOR algorithm for continuous optimization problems, in: M. Dorigo, M. Birattari, G.A. Di Caro, R. Doursat, A.P. Engelbrecht, D. Floreano, L.M. Gambardella, R. Groß, E. Sahin, T. Stützle, H. Sayama (Eds.), Proceedings of the Seventh International Conference on Swarm Intelligence, ANTS'10, vol. 6234 of Lecture Notes in Computer Science, Springer, Heidelberg, Germany, 2010, pp. 48-59.
32. T. Liao, Population-based Heuristic Algorithms for Continuous and Mixed Discrete-Continuous Optimization Problem, PhD thesis, Université Libre de Bruxelles, Brussels, Belgium, 2013.
33. T. Liao, M.A. Montes de Oca, D. Aydin, T. Stützle, M. Dorigo, An incremental ant colony algorithm with local search for continuous optimization, in: N. Krasnogor, P.L. Lanzi (Eds.), Genetic and Evolutionary Computation Conference, GECCO 2011, Proceedings, Dublin, Ireland, July 12-16, ACM Press, New York, NY, 2011, pp. 125-132.
34. T. Liao, D. Molina, T. Stützle, Performance evaluation of automatically tuned continuous optimizers on different benchmark sets: Supplementary material, 2012 http://iridia.ulb.ac.be/supp/IridiaSupp2014-001/
35. T. Liao, D. Aydin, T. Stützle, Artificial bee colonies for continuous optimization: experimental analysis and improvements, Swarm Intell. 7 (4) (2013) 327-356.
36. T. Liao, M.A. Montes de Oca, T. Stützle, Computational results for an automatically tuned CMA-ES with increasing population size on the CEC'05 benchmark set, Soft Comput. 17 (6) (2013) 1031-1046.
37. T. Liao, T. Stützle, M.A. Montes de Oca, M. Dorigo, A unified ant colony optimization algorithm for continuous optimization. Technical Report TR/IRIDIA/2013-002, IRIDIA, Université Libre de Bruxelles, Belgium, 2013.
38. T. Liao, T. Stützle, M.A. Montes de Oca, M. Dorigo, A unified ant colony optimization algorithm for continuous optimization, Eur. J. Operat. Res. 234 (3) (2014) 597-609.
39. M. López-Ibáñez, J. Dubois-Lacoste, T. Stützle, M. Birattari, The irace package, iterated race for automatic algorithm configuration. Technical Report TR/IRIDIA/2011-004, IRIDIA, Université Libre de Bruxelles, Belgium, 2011, http://iridia.ulb.ac.be/IridiaTrSeries/IridiaTr2011-004.pdf.
40. M. Lozano, D. Molina, F. Herrera, Editorial scalability of evolutionary algorithms and other metaheuristics for large-scale continuous optimization problems, Soft Comput. 15 (2011) 2085-2087.
41. O. Mersmann, B. Bischl, H. Trautmann, M. Preuss, C. Weihs, Rudolph G., Exploratory landscape analysis, in: N. Krasnogor, P.L. Lanzi (Eds.), Genetic and Evolutionary Computation Conference, GECCO 2011, Proceedings, Dublin, Ireland, July 12-16, ACM Press, New York, NY, 2011, pp. 829-836.
42. D. Molina, M. Lozano, C. García-Martínez, F. Herrera, Memetic algorithms for continuous optimisation based on local search chains, Evol. Comput. 18 (1) (2010) 27-63.
43. M. Montes de Oca, D. Aydn, T. Stützle, An incremental particle swarm for large-scale continuous optimization problems: an example of tuning-in-the-loop (re)design of optimization algorithms, Soft Comput. 15 (2011) 2233-2255.
44. P. Moscato, Memetic algorithms: a short introduction, in: New Ideas in Optimization, McGraw Hill, London, UK, 1999, pp. 219-234.
45. M.A. Muñoz, M. Kirley, S.K. Halgamuge, A meta-learning prediction model of algorithm performance for continuous optimization problems, in: C.A.C. Coello, V. Cutello, K. Deb, S. Forrest, G. Nicosia, M. Pavone (Eds.), Parallel Problem Solving from Nature XII, vol. 7491 of Lecture Notes in Computer Science, Heidelberg, Germany, 2012, pp. 226-235.
46. V. Nannen, A.E. Eiben, Relevance estimation and value calibration of evolutionary algorithm parameters, in: M.M. Veloso (Ed.), Proceedings of the Twentieth International Joint Conference on Artificial Intelligence (IJCAI-07), AAAI Press, Menlo Park, CA, 2007, pp. 975-980.
47. M. Powell, An efficient method for finding the minimum of a function of several variables without calculating derivatives, Comput. J. 7 (2) (1964) 155, 0010-4620.
48. A. Qin, V. Huang, P. Suganthan, Differential evolution algorithm with strategy adaptation for global numerical optimization, IEEE Trans. Evol. Comput. 13 (2) (2009) 398-417.
49. L.M. Rios, N.V. Sahinidis, Derivative-free optimization: a review of algorithms and comparison of software implementations, J. Global Optim. 56 (2013) 1247-1293.
50. S.K. Smit, A.E. Eiben, Parameter tuning of evolutionary algorithms: generalist vs. specialist, in: C.D. Chio, S. Cagnoni, C. Cotta, M. Ebner, A. Ekárt, A. Esparcia-Alcázar, C.K. Goh, J.J.M. Guervós, F. Neri, M. Preuss, J. Togelius, G.N. Yannakakis (Eds.), EvoApplications (1), vol. 6024 of Lecture Notes in Computer Science, Springer, Heidelberg, Germany, 2010, pp. 542-551.
51. K. Socha, M. Dorigo, Ant colony optimization for continuous domains, Eur. J. Oper. Res. 185 (3) (2008) 1155-1173.
52. R. Storn, K. Price, Differential evolution - a simple and efficient heuristic for global optimization over continuous spaces, J. Global Optim. 11 (4) (1997) 341-359.
53. P. Suganthan, N. Hansen, J. Liang, K. Deb, Y. Chen, A. Auger, S. Tiwari, Problem definitions and evaluation criteria for the CEC 2005 special session on realparameter optimization. Technical Report 2005005, Nanyang Technological University, 2005.
54. E.-G. Talbi, A taxonomy of hybrid metaheuristics, J. Heuristics 8 (5) (2002) 541-564.
55. L. Tseng, C. Chen, Multiple trajectory search for large scale global optimization, in: Proceeding of IEEE Congress on Evolutionary Computation, CEC, IEEE Press, Piscataway, NJ, 2008, pp. 3052-3059.
56. L.D. Whitley, S. Rana, J. Dzubera, K.E. Mathias, Evaluating evolutionary algorithms, Artif. Intell. 85 (1996) 245-296.