Using statistical tools to determine the significance and relative importance of the main parameters of an evolutionary algorithm

Intelligent Data Analysis

Volumn 17, Issue 5, 2013, Pages 771-789

  Arenas, M G ^a   Rico, N ^a   Mora, A M ^a   Castillo, P A ^a   Merelo, J J ^a  

^a UNIVERSITY OF GRANADA   (Spain)

Author keywords

ANOVA; EA; optimization algorithms; parameter tuning; search method parameters; Statistical analysis; statistical significance

Indexed keywords

EA; OPTIMIZATION ALGORITHMS; PARAMETER-TUNING; SEARCH METHOD PARAMETERS; STATISTICAL SIGNIFICANCE;

ALGORITHMS; ANALYSIS OF VARIANCE (ANOVA); STATISTICAL MECHANICS; STATISTICAL METHODS;

PARAMETER ESTIMATION;

EID: 84885149196     PISSN: 1088467X     EISSN: 15714128     Source Type: Journal    
DOI: 10.3233/IDA-130606     Document Type: Article

References (72)

1. A.E. Eiben and J.E. Smith, Introduction to Evolutionary Computing. Springer, ISBN 3-540-40184-9, 2003.
2. X.S. Yang, Nature-Inspired Metaheuristic Algorithms, 2nd Edition, Luniver Press., 2010.
3. S. Kirkpatrick, C.D. Gerlatt and M.P. Vecchi, Optimization by Simulated Annealing, Science 220 (1983), 671-680.
4. J. De Vicente, J. Lanchares and R. Hermida, Placement by Thermodynamic Simulated Annealing, Physics Letters A 317(5:6) (2003), 415-423.
5. P. Van-Hentenryck and L. Michel, Constraint-Based Local Search, The MIT Press. ISBN: 978-0-262-22077-4, 2005.
6. H.H. Hoos and T. Stutzle, Stochastic Local Search. Foundations and Applications, Morgan Kaufmann/Elsevier, 2004.
7. N.K. Bambha, S.S. Bhattacharyya, J. Teich and E. Zitzler, Systematic integration of parameterized local search into evolutionary algorithms, IEEE Transactions on Evolutionary Computation 8(2) (2004), 137-155. ISSN: 1089-778X.
8. A.E. Eiben, R. Hinterding and Z. Michalewicz, Parameter Control in Evolutionary Algorithms, IEEE Transactions on Evolutionary Computation 3(2) (1999), 124-141, ISSN: 1089-778X, DOI:10.1109/4235.771166.
9. C.L. Karr and E.Wilson, A self-tuning evolutionary algorithm applied to an inverse partial differential equation, Applied Intelligence 19(3) (November 2003), 147-155.
10. S.K. Smit and A.E. Eiben, Beating the world champion Evolutionary Algorithm via REVAC Tuning, In Proc. of the WCCI2010 IEEE World Congress on Computational Intelligence, issn 978-1-4244-8126-2. IEEE Press. Barcelona, Spain, 2010, pp. 1295-1302.
11. R.A. Fisher, Theory of Statistical Estimation, Proceedings of the Cambridge Philosophical Society 22 (1925), 700-725.
12. P. Dickinson and B. Chow, Some properties of the Tukey test to Duckworth's specification/by Peter Dickinson, Bryant Chow, Office of Institutional Research, University of Southwestern Louisiana, Lafayette, Louisiana, USA, 1971.
13. K.A. De Jong, An analysis of the behavior of a class of genetic adaptive systems, Ph.D. thesis, University of Michigan, Ann Arbor, 1975.
14. S. Das, A. Chowdhury and A. Abraham, A bacterial evolutionary algorithm for automatic data clustering, In Proceedings of the Eleventh conference on Congress on Evolutionary Computation, CEC'09, Piscataway, NJ, USA, 2009. IEEE Press, pp. 2403-2410.
15. A.E. Eiben, Principled Approaches to tuning EA parameters. Tutorials-IEEE Congress on Evolutionary Computation (CEC2009). Available at http://www.few.vu.nl/gusz/papers/eiben-cec-2009-tutorial-corrected.pdf, 2009.
16. K. Liang, X. Yao and C.S. Newton, Adapting Self-Adaptive Parameters in Evolutionary Algorithms, Applied Intelligence 15 (01 November 2001), 171-180.
17. T. Bäck, D. Fogel and Z. Michalewicz, Handbook of Evolutionary Computation, Institute of Physics Publishing Ltd, Bristol and Oxford University Press, NY, 1997.
18. R. Hinterding, Z. Michalewicz and A.E. Eiben, Adaptation in Evolutionary Computation: A Survey. Proceedings of the 4th IEEE Conference on Evolutionary Computation, pp. 65-69. IEEE Press, 1997.
19. G.R. Harik and F.G. Lobo, A parameter-less genetic algorithm, in: Proceedings of the Genetic and Evolutionary Computation Conference, (volume 1), W. Banzhaf, J. Daida, A.E. Eiben, M.H. Garzon, V. Honavar, M. Jakiela and R.E. Smith, eds, Orlando, Florida, USA, 13-17 1999, pp. 258-265, Morgan Kaufmann.
20. A.E. Eiben, Z. Michalewicz, M. Schoenauer and J.E. Smith, Parameter control in evolutionary algorithms, Parameter Setting in Evolutionary Algorithms, Studies in Computational Intelligence 54 (2007), 19-46. ISBN 978-3-540-69431-1, ISSN 1860-949X. Springer Berlin/Heidelberg.
21. S.K. Smit and A.E. Eiben, Comparing parameter tuning methods for evolutionary algorithms, in: CEC'09: Proc. of the Eleventh conference on Congress on Evolutionary Computation, P. Haddow et al., eds, Piscataway, NJ, USA, 2009. pp. 399-406, IEEE Press.
22. Y. Gong and A. Fukunaga, Distributed Island-Model Genetic Algorithms Using Heterogeneous Parameter Settings, Proceedings of the 2011 IEEE Congress on Evolutionary Computation (CEC2011), ISBN 978-1-4244-7834-7. New Orleans, LA, USA., 2011, pp. 820-827.
23. L. Davis, Handbook of genetic algorithms, Van Nostrand Reinhold, NY, 1991.
24. I. Jagielska, C. Matthews and T. Whitfort, An investigation into the application of neural networks, fuzzy logic, genetic algorithms, and rough sets to automated knowledge acquisition problems, Neurocomputing 24(1-3) (1999), 37-54.
25. J.J. Grefenstette, Optimization of control parameters for genetic algorithms, IEEE Trans. Systems, Man, and Cybernetics 16(1) (1986), 122-128.
26. D. Kim and C. Kim, Forecasting time series with genetic fuzzy predictor ensemble, IEEE Transactions on Fuzzy Systems 5(4) (1997), 523-535.
27. D.E. Goldberg, K. Deb and D. Theirens, Toward a better understanding of mixing in genetic algorithms, Proc. of the 4th Int. Conf. on Genetic Algorithms, R.K. Belew and L.B. Booker, eds, Morgan Kaufmann, 1991, pp. 190-195.
28. J.D. Schaffer and A. Morishima, An adaptive crossover distribution mechanism for genetic algorithms, Proceedings of the 2nd International Conference on Genetic Algorithms and Their Applications, J.J. Grefenstette, ed., Lawrence Erlbaum Associates, 1987, pp. 36-40.
29. T. Jansen and D. Weyland, Analysis of evolutionary algorithms for the longest common subsequence problem, in: GECCO '07: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, New York, NY, USA, ACM, 2007, pp. 939-946.
30. T. Jansen and D. Weyland. Analysis of Evolutionary Algorithms for the Longest Common Subsequence Problem, Algorithmica 57(1) (2010), 170-186.
31. D. Weyland, Simulated annealing, its parameter settings and the longest common subsequence problem, in: GECCO '08: Proceedings of the 10th annual conference on Genetic and evolutionary computation, New York, NY, USA, ACM, 2008, pp. 803-810.
32. F.G. Lobo, C.F. Lima and Z.Michalewicz, Parameter Setting in Evolutionary Algorithms, Springer Publishing Company, Incorporated, 2007.
33. V. Nannen and A.E. Eiben, Relevance estimation and value calibration of evolutionary algorithm parameters, In Proc. of the International Joint Conference on Artificial Intelligence (IJCAI2007), AAAI Press, 2007, pp. 975-980.
34. S.K. Smit and A.E. Eiben, Comparing parameter tuning methods for evolutionary algorithms, In Proc. of the 2009 IEEE Congress on Evolutionary Computation, A. Tyrrell, ed., Thondheim, IEEE Press, 2009, pp. 399-406.
35. T. Bartz, K.E. Parsopoulos and M.N. Vrahatis, Analysis of Particle Swarm Optimization Using Computational Statistics, In Chalkis, editor, Proc. of the International Conference of Numerical Analysis and Applied Mathematics (ICAAM2004), 2004, pp. 34-37.
36. T. Bartz, C.W.G. Lasarczyk and M. Preuss, Sequential parameter optimization, In Proc. of the 2005 IEEE Congress on Evolutionary Computation, (Vol. 1), Edinburgh, UK, 2005, pp. 773-780.
37. M. Birattari, T. Stutzle, L. Paquete and K. Varrentrapp, A racing algorithm for configuring metaheuristics, In Proc. of the Genetic and Evolutionary Computation Conference NY USA. Morgan Kaufmann, 2002, pp. 11-18.
38. F. Hutter, H.H. Hoos and T. Stutzle, Automatic algorithm configuration based on local search, In Proc. of the Twenty-Second Conference on Artificial Intelligence (AAAI 2007), AAAI Press, 2007, pp. 1152-1157.
39. E. Montero, M.C. Riff and B. Neveu, New requirements for off-line parameter calibration algorithms, In Proc. of the WCCI2010 IEEE World Congress on Computational Intelligence, issn 978-1-4244-8126-2. IEEE Press. Barcelona, Spain, 2010, pp. 3466-3473.
40. S.W. Leung, S.Y. Yuen and C.K. Chow, Parameter Control by the Entire Search History: Case Study of History-driven Evolutionary Algorithm, In Proc. of the WCCI2010 IEEE World Congress on Computational Intelligence, issn 978-1-4244-8126-2. IEEE Press. Barcelona, Spain, 2010, pp. 3688-3695.
41. S.K. Smit and A.E. Eiben, Parameter tuning of evolutionary algorithms: Generalist vs. specialist, in: Applications of Evolutionary Computation, Cecilia Di Chio et al., ed., volume 6024 of LNCS, Springer, 2010, pp. 542-551.
42. J. He, A.H. Tan and C.L. Tan, OnMachine Learning Methods for Chinese Document Categorization, Applied Intelligence 18 (01 May 2003), 311-322.
43. J. Maudes, J.J. Rodriguez, C. Garcia-Osorio and C. Pardo, Random projections for linear SVM ensembles, Applied Intelligence 34 (01 June 2011), 347-359.
44. D. Shilane, J. Martikainen, S. Dudoit and S.J. Ovaska, A general framework for statistical performance comparison of evolutionary computation algorithms, Information Sciences 178(14) (2008), 2870-2879.
45. S. Garcia, D. Molina, M. Lozano and F. Herrera, A Study on the Use of Non-Parametric Tests for Analyzing the Evolutionary Algorithms' Behaviour: A Case Study on the CEC'2005 Special Session on Real Parameter Optimization, Journal of Heuristics 15 (2009), 617-644.
46. A.K. Smilde, J.J. Jansen, H.C.J. Hoefsloot, R.-J.A.N. Lamers, J.V.D.Greef andM.E. Timmerman, ANOVA-simultaneous component analysis (ASCA): A new tool for analyzing designed metabolomics data, Bioinformatics 21(13) (July 2005), 3043-3048.
47. M.-D. Ma, C.-C. Chang, S.-S. Jang and D.S.-H. Wong, Mixed product run-to-run process control-An ANOVA model with ARIMA disturbance approach, Journal of Process Control 19(4) (2009), 604-614.
48. A. Azadeh, S.F. Ghaderi and S. Sohrabkhani, Forecasting electrical consumption by integration of Neural Network, time series and ANOVA, Applied Mathematics and Computation 186(2) (2007), 1753-1761.
49. M.T. Brenneman, Y.T. Morton and Q. Zhou, An ANOVA-based GPS multipath detection algorithm using multi-channel software receivers, in: IEEE Position, Location and Navigation Symposium, IEEE Press, May 2008, pp. 1316-1323.
50. I. Guyon and A. Elisseeff, An introduction to variable and feature selection, J Mach Learn Res 3 (March 2003), 1157-1182.
51. Y. Saeys, I. Inza and P. Larrañaga, A review of feature selection techniques in bioinformatics, Bioinformatics 23(19) (September 2007), 2507-2517.
52. K.M. Pierce, J.L. Hope, K.J. Johnson, B.W. Wright and R.E. Synovec, Classification of gasoline data obtained by gas chromatography using a piecewise alignment algorithm combined with feature selection and principal component analysis, Journal of Chromatography A 1096(1-2) (2005), 101-110. Chemical Separations and Chemometrics.
53. V. Michel, C. Damon and B. Thirion, Mutual information-based feature selection enhances fMRI brain activity classification, in: IEEE International Symposium on Biomedical Imaging: From Nano to Macro, 2008. ISBI 2008. 5th, IEEE Press., May 2008, pp. 592-595.
54. A. Bharathi and A.M. Natarajan, Minimal Feature Selection using SVM based on ANOVA. Journal of Theoretical and Applied Information Technology, 2005, pp. 162-168.
55. I. Rojas, J. González, H. Pomares, J.J. Merelo, P.A. Castillo and G. Romero, Statistical Analisys of the main Parameters Involved in the Desing of a Genetic Algorithm, in IEEE Trans. on Systems, Man and Cibernetics-Part C 32(10) (2002), 31-37, ISSN:1094-6977, IEEE Press, november.
56. P.A. Castillo, J.J. Merelo, G. Romero, A. Prieto and I. Rojas, Statistical Analysis of the Parameters of a Neuro-Genetic Algorithm, in IEEE Trans. on Neural Networks 13(6) (2002), 1374-1394, ISSN:1045-9227, IEEE Press, november.
57. M.G. Arenas, P.A. Castillo, A.M. Mora, J.J. Merelo, J.L. Laredo and P. Garcia, Statistical Analysis of Parameter Setting in Real-Coded Evolutionary Algorithms, Lecture Notes in Computer Science 6239 (2010), 452-461, R. Schaefer et al., eds, Springer, Heidelberg, isbn 978-3-642-15870-4.
58. F. Herrera, M. Lozano and A.M. Sanchez, A taxonomy for the crossover operator for real-coded genetic algorithms: An experimental study, International Journal of Intelligent Systems 18 (2003), 309-338.
59. E.G. Talbi, Metaheuristics. From design to implementation, University of Lille CNRS INRIA. Published by John Wiley and Sons, Inc., Hoboken, New Jersey. USA. ISBN 978-0-470-27858-1, 2009.
60. T. Back, D.B. Fogel and Z. Michalewicz, eds, Evolutionary Computation 1: Basic Algorithms and Operators. Taylor and Francis; 1st edition, 1 edition, 2000.
61. Z. Michalewicz and D.B. Fogel, How to Solve It: Modern Heuristics, Springer-Verlag, 2 edition, 2004.
62. Z. Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs, Third, Extended Edition, Springer-Verlag, 1996.
63. A.O. Griewank, Generalized descent for global optimization, Journal of Optimization Theory and Applications 34(1) (1981), 11-39.
64. A. Törn and A. Zilinskas, Global Optimization, Vol. 350 of Springer Lecture Notes in Computer Science (Springer, Berlin), 1989, pp. 1-24.
65. D.H. Ackley, A connectionist machine for genetic hillclimbing, Kluwer, Boston, 1987.
66. T. Bäck, Evolutionary algorithms in theory and practice, Oxford University Press, 1996.
67. D. Whitley, D. Garrett and J. paul Watson, Quad search and hybrid genetic algorithms, in: In Genetic and Evolutionary Computation-GECCO 2003, Springer, 2003, pp. 1469-1480.
68. H. Cho, F. Olivera and S.D. Guikema, A derivation of the number of minima of the Griewank function, Applied Mathematics and Computation 204(2) (2008), 694-701.
69. D.H. Wolpert and W.G. Macready, No Free Lunch Theorems for Optimization, IEEE Transactions on Evolutionary Computation 1(1) (1997), 67-82.
70. Y. Benjamini, Opening the box of a boxplot, The American Statistician 42(4) (1988), 257-262.
71. D.E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1989.
72. M. Mitchell, An Introduction to Genetic Algorithms (Complex Adaptive Systems), The MIT Press, 1998.