Replication and comparison of computational experiments in applied evolutionary computing: Common pitfalls and guidelines to avoid them

Applied Soft Computing Journal

Volumn 19, Issue , 2014, Pages 161-170

  Črepinšek, Matej ^a   Liu, Shih Hsi ^b   Mernik, Marjan ^a  

^a UNIVERSITY OF MARIBOR   (Slovenia)

^b CALIFORNIA STATE UNIVERSITY   (United States)

Author keywords

Algorithms comparison; Economic load dispatch; Evolutionary algorithms; Experiments replication

Indexed keywords

ELECTRIC LOAD DISPATCHING; ELECTRIC POWER PLANT LOADS; STOCHASTIC SYSTEMS;

COMPUTATIONAL EXPERIMENT; ECONOMIC LOAD DISPATCH; ECONOMIC LOAD DISPATCH PROBLEM; EVOLUTIONARY ALGORITHMS (EAS); EVOLUTIONARY COMPUTING; PERFORMANCE MEASURE; STATISTICAL SIGNIFICANCE; STOCHASTIC PROPERTIES;

EVOLUTIONARY ALGORITHMS;

EID: 84896336887     PISSN: 15684946     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.asoc.2014.02.009     Document Type: Article

References (45)

1. S. Akhtar, K. Tai, and T. Ray A socio-behavioural simulation model for engineering design optimization Engineering Optimization 34 4 2002 341 354
2. M. Ali, M. Pant, and A. Abraham Improving differential algorithm by synergizing different improvement mechanisms ACM Transactions on Autonomous and Adaptive Systems 7 2 2012 20:1-20:32
3. M.G. Arenas, N. Rico, A.M. Mora, P.A. Castillo, and J.J. Merelo Using statistical tools to determine the significance and relative importance of the main parameters of an evolutionary algorithm Intelligent Data Analysis 17 5 2013 771 789
4. T. Bäck, D.B. Fogel, and Z. Michalewicz Handbook of Evolutionary Computations 1996 Oxford University Press, Taylor & Francis New York, NY, USA
5. R.S. Barr, B.L. Golden, J.P. Kelly, M.G.C. Resende, and W.R. Stewart Jr. Designing and reporting on computational experiments with heuristic methods Journal of Metaheuristics 1 1995 9 32
6. R.S. Barr, B.L. Golden, J.P. Kelly, W.R. Stewart, and M.G.C. Resende Guidelines for designing and reporting on computational experiments with heuristic methods. Technical Report 2001 Department of Computer Science and Engineering, Southern Methodist University Dallas, TX
7. T. Bartz-Beielstein Experimental Research in Evolutionary Computation - The New Experimentalism 2006 Springer Berlin, Germany
8. A. Bhattacharya, and P.K. Chattopadhyay Solving economic emission load dispatch problems using hybrid differential evolution Applied Soft Computing 11 2 2011 2526 2537
9. C. Blum, and A. Roli Metaheuristics in combinatorial optimization: overview and conceptual comparison ACM Computing Surveys 35 3 2003 268 308
10. C. Blum, J. Puchinger, G.A. Raidl, and A. Roli Hybrid metaheuristics in combinatorial optimization: a survey Applied Soft Computing 11 6 2011 4135 4151
11. M. Clerc Randomness matters. Technical Report 2012 http://clerc.maurice. free.fr/pso/
12. M. Črepinšek, M. Mernik, and S.H. Liu Analysis of exploration and exploitation in evolutionary algorithms by ancestry trees International Journal of Innovative Computing and Applications 3 1 2011 11 19
13. M. Črepinšek, S.H. Liu, and L. Mernik A note on teaching-learning-based optimization algorithm Information Sciences 212 2012 79 93
14. M. Črepinšek, S.-H. Liu, and M. Mernik Exploration and exploitation in evolutionary algorithms: a survey ACM Computing Surveys 45 2013 35:1-35:33
15. J. Derrac, S. Garcia, D. Molina, and F. Herrera A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms Swarm and Evolutionary Computation 1 1 2011 3 18
16. A.E. Eiben, R. Hinterding, and Z. Michalewicz Parameter control in evolutionary algorithms IEEE Transactions on Evolutionary Computation 3 2 1999 124 141
17. A.E. Eiben, and M. Jelasity A critical note on experimental research methodology in EC Proceedings of the 2002 Congress on Evolutionary Computation (CEC'2002) 58 2002 2 587
18. A.E. Eiben, and J.E. Smith Introduction to Evolutionary Computing 2008 Springer Berlin, Germany
19. S. Fomel, and J. Claerbout Guest editors' introduction: reproducible research Computing in Science Engineering 11 1 2009 5 7
20. S. Garcia, D. Molina, M. Lozano, and F. Herrera A study on the use of non-parametric tests for analysing the evolutionary algorithms' behaviour: a case study on the CEC'2005 special session on real parameter optimization Journal of Metaheuristics 15 6 2009 617 644
21. S. Garcia, A. Fernandez, J. Luengo, and F. Herrera Advances nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power Information Sciences 180 10 2010 2044 2064
22. D. Goldberg Genetic Algorithms in Search Optimization, and Machine Learning 1989 Addison-Wesley Boston, MA, USA
23. N. Hansen, A. Auger, S. Finck, and R. Ros Real-Parameter Black-Box Optimization Benchmarking 2010: Experimental Setup. INRIA research report RR-7215 2010
24. S. Hawking On the Shoulders of Giants: The Great Works of Physics and Astronomy 2003 Running Press Philadelphia, PA, USA
25. D. Karaboga, and B. Akay A modified Artificial Bee Colony (ABC) algorithm for constrained optimization problems Applied Soft Computing 11 3 2011 3021 3031
26. P. Korošec, J. Šilc, and B. Filipič The differential ant-stigmergy algorithm Information Sciences 192 1 2012 82 96
27. A.G. Krishna, and A.M. Rao Optimization of machining parameters for milling operations using a scatter search approach International Journal of Advanced Manufacturing Technology 31 3-4 2006 219 224
28. X. Li, K. Tang, M.N. Omidvar, Z. Yang, and K. Qin Benchmark Functions for the CEC'2013 Special Session and Competition on Large-Scale Global Optimization, Technical Report 2013
29. S.H. Ling, H.H.C. Iu, K.Y. Chan, H.K. Lam, B.C.W. Yeung, and F.H.F. Leung Hybrid particle swarm optimization with wavelet mutation and its industrial applications IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics 38 3 2008 743 763
30. S.-H. Liu, M. Mernik, D. Hrnčič, and M. Črepinšek A parameter control method of evolutionary algorithms using exploration and exploitation measures with a practical application for fitting Sovova's mass transfer model Applied Soft Computing 13 9 2013 3792 3805
31. H. Lu, P. Sriyanyong, Y.H. Song, and T. Dillon Experimental Study of a new hybrid PSO with mutation for economic dispatch with non-smooth cost function Electrical Power and Energy Systems 32 9 2010 921 935
32. D.C. Mattfeld, and C. Bierwirth An efficient genetic algorithm for job shop scheduling with tardiness objectives European Journal of Operational Research 155 3 2004 616 630
33. K. Meng, H.G. Wang, Z.Y. Dong, and K.P. Wong Quantum-inspired particle swarm optimization for valve-point economic load dispatch IEEE Transactions on Power Systems 25 1 2010 215 222
34. Z. Michalewicz Genetic Algorithms + Data Structures = Evolution Programs third ed. 1996 Springer-Verlag Berlin, Germany
35. L.M. Pickard, B.A. Kitchenham, and P.W. Jones Combining empirical results in software engineering Information and Software Technology 40 14 1998 811 821
36. M. Preuss Reporting on Experiments in Evolutionary Computation. Technical Report No. CI-221/07 2007 University of Dortmund Germany
37. D. Shilane, J. Martikainen, S. Dudoit, and S. Ovaska A general framework for statistical performance comparisons of evolutionary computation algorithms Information Sciences 178 14 2008 2870 2879
38. J. Srinavas, R. Giri, and S.-H. Yang Optimization of multi-pas turning using particle swarm intelligence International Journal of Advanced Manufacturing Technology 40 1-2 2009 56 66
39. V. Stodden Trust your science? Open your data and code Amstat News 2 2011 1 22
40. P.N. Suganthan, N. Hansen, J.J. Liang, K. Deb, Y.-P. Chen, A. Auger, and S. Tiwari Problem Definitions and Evaluation Criteria for the CEC 2005: Special Session on Real-Parameter Optimization. Technical Report 2005 Nanyang Technological University Singapore
41. K. Tang, X. Li, P.N. Suganthan, Z. Yang, and T. Weise Benchmark Functions for the CEC'2010 Special Session and Competition on Large Scale Global Optimization. Technical Report 2010
42. G. Waghmare Comments on A note on teaching-learning-based optimisation algorithm Information Sciences 229 2013 159 169
43. D.C. Walters, and G.B. Sheble Genetic algorithm solution of economic dispatch with valve point loading IEEE Transactions on Power Systems 8 3 1993 1325 1332
44. G.E. Welch, and S.G. Gabbe Review of statistics usage in the american journal of obstetrics and gynecology American Journal of Obstetrics and Gynecology 175 5 1996 1138 1141
45. X. Yuan, L. Wang, Y. Yuan, Y. Zhang, and B. Yang A modified differential evolution approach for dynamic economic dispatch with valve-point effects Energy Conversion and Management 49 12 2008 3447 3453