Practical performance models of algorithms in evolutionary program induction and other domains

Artificial Intelligence

Volumn 174, Issue 15, 2010, Pages 1254-1276

  Graff, Mario ^a   Poli, Riccardo ^a  

^a UNIVERSITY OF ESSEX   (United Kingdom)

Author keywords

Algorithm selection problem; Algorithm taxonomies; Evolution algorithms; Performance prediction; Program induction

Indexed keywords

ACCURATE PREDICTION; ALGORITHM SELECTION; APPROXIMATE MODEL; ARTIFICIAL NEURAL NETWORK; AUTOMATIC CONSTRUCTION; BIN PACKING PROBLEM; EVOLUTION ALGORITHMS; EVOLUTIONARY COMPUTATION TECHNIQUES; GENE EXPRESSION PROGRAMMING; HILL CLIMBING; INDUCTION ALGORITHMS; INDUCTION SYSTEM; OPTIMISATIONS; PARAMETERS SETTING; PERFORMANCE MODEL; PERFORMANCE PREDICTION; PRACTICAL MODEL; PROGRAM INDUCTION; SYMBOLIC REGRESSION; TRAINING ALGORITHMS;

BOOLEAN FUNCTIONS; DISTILLATION; GENE EXPRESSION; GENETIC PROGRAMMING; NEURAL NETWORKS; PARAMETER ESTIMATION; PROBLEM SOLVING; STOCHASTIC MODELS; STOCHASTIC SYSTEMS; TAXONOMIES;

EVOLUTIONARY ALGORITHMS;

EID: 77955849498     PISSN: 00043702     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.artint.2010.07.005     Document Type: Article

References (101)

1. J.H. Holland Adaptation in Natural and Artificial Systems 1975 University of Michigan Press Ann Arbor, USA
2. D.E. Goldberg Genetic Algorithms in Search, Optimization and Machine Learning 1989 Addison-Wesley
3. Z. Michalewicz Genetic Algorithms + Data Structures = Evolution Programs 2nd edition 1994 Springer Berlin
4. M. Mitchell An Introduction to Genetic Algorithms 1996 MIT Press Cambridge, MA
5. T. Bäck, D.B. Fogel, Z. Michalewicz, Evolutionary Computation 1: Basic Algorithms and Operators 2000 Institute of Physics Publishing
6. J.R. Koza Genetic Programming: On the Programming of Computers by Natural Selection 1992 MIT Press Cambridge, MA, USA
7. D.B. Fogel, Evolutionary Computation. The Fossil Record. Selected Readings on the History of Evolutionary Computation 1998 IEEE Press
8. A.E. Nix, and M.D. Vose Modeling genetic algorithms with Markov chains Annals of Mathematics and Artificial Intelligence 5 1992 79 88
9. M.D. Vose The Simple Genetic Algorithm: Foundations and Theory 1999 MIT Press Cambridge, MA
10. T.E. Davis, and J.C. Principe A Markov chain framework for the simple genetic algorithm Evolutionary Computation 1 3 1993 269 288
11. G. Rudolph Convergence analysis of canonical genetic algorithm IEEE Transactions on Neural Networks 5 1 1994 96 101
12. C.R. Stephens, and H. Waelbroeck Schemata evolution and building blocks Evolutionary Computation 7 2 1999 109 124
13. J. He, and X. Yao Drift analysis and average time complexity of evolutionary algorithms Artificial Intelligence 127 1 2001 57 85
14. R. Poli Exact schema theory for genetic programming and variable-length genetic algorithms with one-point crossover Genetic Programming and Evolvable Machines 2 2 2001 123 163
15. W.B. Langdon, and R. Poli Foundations of Genetic Programming 2002 Springer
16. J. He, and X. Yao Towards an analytic framework for analysing the computation time of evolutionary algorithms Artificial Intelligence 145 1-2 2003 59 97
17. R. Poli, and N.F. McPhee General schema theory for genetic programming with subtree-swapping crossover: Part II Evolutionary Computation 11 2 2003 169 206
18. R. Poli, N.F. McPhee, and J.E. Rowe Exact schema theory and Markov chain models for genetic programming and variable-length genetic algorithms with homologous crossover Genetic Programming and Evolvable Machines 5 1 2004 31 70
19. R. Poli, W.B. Langdon, and N.F. McPhee A field guide to genetic programming published via http://lulu.com and freely available at http://www.gp-field-guide.org.uk 2008 (with contributions by J.R. Koza)
20. J.F. Miller, and P. Thomson Cartesian genetic programming R. Poli, W. Banzhaf, W.B. Langdon, J.F. Miller, P. Nordin, T.C. Fogarty, Proceedings of the Third European Conference on Genetic Programming (EuroGP-2000) LNCS vol. 1802 2000 Springer-Verlag Edinburgh 121 132
21. M. O'Neill, and C. Ryan Grammatical evolution IEEE Transactions on Evolutionary Computation 5 4 2001 349 358
22. C. Ferreira Gene expression programming: A new adaptive algorithm for solving problems Complex Systems 13 2 2001 87 129
23. R. Poli, and C.R. Stephens The building block basis for genetic programming and variable-length genetic algorithms International Journal of Computational Intelligence Research 1 2 2005 183 197 invited paper
24. R. Poli, Y. Borenstein, and T. Jansen Editorial for the special issue on: Evolutionary algorithms - bridging theory and practice Evolutionary Computation 15 4 2007 iii-v
25. J.R. Rice The algorithm selection problem Advances in Computers 15 1976 65 118
26. K. Leyton-Brown, E. Nudelman, and Y. Shoham Empirical hardness models: Methodology and a case study on combinatorial auctions J. ACM 56 4 2009 1 52
27. K. Leyton-Brown, E. Nudelman, and Y. Shoham Empirical hardness models for combinatorial auctions P. Cramton, Y. Shoham, R. Steinberg, Combinatorial Auctions 2006 MIT Press 479 504 (Ch. 19)
28. F. Hutter, Y. Hamadi, H.H. Hoos, and K. Leyton-Brown Performance prediction and automated tuning of randomized and parametric algorithms F. Benhamou, CP Lecture Notes in Computer Science vol. 4204 2006 Springer 213 228
29. E. Nudelman, K. Leyton-Brown, H.H. Hoos, A. Devkar, Y. Shoham, Understanding random SAT: Beyond the clauses-to-variables ratio, in: Wallace [100], pp. 438-452.
30. K. Leyton-Brown, E. Nudelman, G. Andrew, J. McFadden, and Y. Shoham Boosting as a metaphor for algorithm design F. Rossi, CP Lecture Notes in Computer Science vol. 2833 2003 Springer 899 903
31. K. Leyton-Brown, E. Nudelman, G. Andrew, J. McFadden, and Y. Shoham A portfolio approach to algorithm selection G. Gottlob, T. Walsh, IJCAI 2003 Morgan Kaufmann 1542 1543
32. K. Leyton-Brown, E. Nudelman, and Y. Shoham Learning the empirical hardness of optimization problems: The case of combinatorial auctions P.V. Hentenryck, CP Lecture Notes in Computer Science vol. 2470 2002 Springer 556 572
33. L. Xu, F. Hutter, H.H. Hoos, and K. Leyton-Brown SATzilla: portfolio-based algorithm selection for SAT Journal of Artificial Intelligence Research 32 2008 565 606
34. L. Xu, F. Hutter, H.H. Hoos, K. Leyton-Brown, SATzilla-07: The design and analysis of an algorithm portfolio for SAT, in: Bessiere [101], pp. 712-727.
35. L. Xu, H.H. Hoos, K. Leyton-Brown, Hierarchical hardness models for SAT, in: Bessiere [101], pp. 696-711.
36. K. Deb, and D.E. Goldberg Analyzing deception in trap functions L.D. Whitley, Foundations of Genetic Algorithms Workshop, vol. 2 1993 Morgan Kaufmann 93 108
37. M. Mitchell, S. Forrest, and J.H. Holland The royal road for genetic algorithms: fitness landscapes and ga performance F.J. Varela, P. Bourgine, Toward a Practice of Autonomous Systems: Proceedings of the First European Conference on Artificial Life 1992 The MIT Press 245 254
38. S.J. Wright, The roles of mutation, inbreeding, crossbreeding and selection in evolution, in: D.F. Jones (Ed.), Proceedings of the Sixth International Congress on Genetics, vol. 1, 1932, pp. 356-366.
39. J. Horn, and D.E. Goldberg Genetic algorithm difficulty and the modality of the fitness landscapes L.D. Whitley, M.D. Vose, Foundations of Genetic Algorithms Workshop, vol. 3 1995 Morgan Kaufmann 243 269
40. S.A. Kauffman, and S. Johnsen Coevolution to the edge of chaos: Coupled fitness landscapes, poised states, and coevolutionary avalanches Journal of Theoretical Biology 149 4 1991 467 505
41. T. Jones, and S. Forrest Fitness distance correlation as a measure of problem difficulty for genetic algorithms L.J. Eshelman, ICGA 1995 Morgan Kaufmann 184 192
42. M. Clergue, P. Collard, M. Tomassini, and L. Vanneschi Fitness distance correlation and problem difficulty for genetic programming W.B. Langdon, E. Cantú-Paz, K.E. Mathias, R. Roy, D. Davis, R. Poli, K. Balakrishnan, V. Honavar, G. Rudolph, J. Wegener, L. Bull, M.A. Potter, A.C. Schultz, J.F. Miller, E.K. Burke, N. Jonoska, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2002) July 2002 Morgan Kaufmann New York, USA 724 732
43. L. Vanneschi, M. Tomassini, P. Collard, and M. Clergue Fitness distance correlation in structural mutation genetic programming C. Ryan, T. Soule, M. Keijzer, E.P.K. Tsang, R. Poli, E. Costa, EuroGP Lecture Notes in Computer Science vol. 2610 2003 Springer 455 464
44. L. Vanneschi, M. Tomassini, M. Clergue, and P. Collard Difficulty of unimodal and multimodal landscapes in genetic programming E. Cantú-Paz, J.A. Foster, K. Deb, L. Davis, R. Roy, U.-M. O'Reilly, H.-G. Beyer, R.K. Standish, G. Kendall, S.W. Wilson, M. Harman, J. Wegener, D. Dasgupta, M.A. Potter, A.C. Schultz, K.A. Dowsland, N. Jonoska, J.F. Miller, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2003) Lecture Notes in Computer Science vol. 2723 July 2003 Springer Chicago, IL, USA 1788 1799
45. M. Tomassini, L. Vanneschi, P. Collard, and M. Clergue A study of fitness distance correlation as a difficulty measure in genetic programming Evolutionary Computation 13 2 2005 213 239
46. L. Vanneschi, M. Clergue, P. Collard, M. Tomassini, and S. Vérel Fitness clouds and problem hardness in genetic programming K. Deb, R. Poli, W. Banzhaf, H.-G. Beyer, E.K. Burke, P.J. Darwen, D. Dasgupta, D. Floreano, J.A. Foster, M. Harman, O. Holland, P.L. Lanzi, L. Spector, A. Tettamanzi, D. Thierens, A.M. Tyrrell, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2004) Lecture Notes in Computer Science vol. 3103 June 2004 Springer Seattle, WA, USA 690 701
47. L. Vanneschi, M. Tomassini, P. Collard, and M. Clergue A survey of problem difficulty in genetic programming S. Bandini, S. Manzoni, Advances in Artificial Intelligence: Proceedings of the Ninth Congress of the Italian Association for Artificial Intelligence (AI*IA-2005) Lecture Notes in Computer Science vol. 3673 September 2005 Springer Milan, Italy 66 77
48. L. Vanneschi, M. Tomassini, P. Collard, and S. Vérel Negative slope coefficient: A measure to characterize genetic programming fitness landscapes P. Collet, M. Tomassini, M. Ebner, S. Gustafson, A. Ekárt, EuroGP Lecture Notes in Computer Science vol. 3905 2006 Springer 178 189
49. R. Poli, L. Vanneschi, Fitness-proportional negative slope coefficient as a hardness measure for genetic algorithms, in: Lipson [98], pp. 1335-1342.
50. S. Droste, T. Jansen, and I. Wegener A rigorous complexity analysis of the (1 + 1 ) evolutionary algorithm for separable functions with Boolean inputs Evolutionary Computation 6 2 1998 185 196
51. S. Droste, T. Jansen, and I. Wegener On the analysis of the (1 + 1 ) evolutionary algorithm Theoretical Computer Science 276 1-2 2002 51 81
52. I. Wegener On the expected runtime and the success probability of evolutionary algorithms U. Brandes, D. Wagner, WG Lecture Notes in Computer Science vol. 1928 2000 Springer 1 10
53. T. Jansen, K.A.D. Jong, and I. Wegener On the choice of the offspring population size in evolutionary algorithms Evolutionary Computation 13 4 2005 413 440
54. C. Witt Runtime analysis of the (mu + 1) ea on simple pseudo-boolean functions Evolutionary Computation 14 1 2006 65 86
55. T. Jansen, and I. Wegener The analysis of evolutionary algorithms - a proof that crossover really can help Algorithmica 34 1 2002 47 66
56. T. Jansen, and I. Wegener Real royal road functions - where crossover provably is essential Discrete Applied Mathematics 149 1-3 2005 111 125
57. O. Giel, and I. Wegener Evolutionary algorithms and the maximum matching problem H. Alt, M. Habib, STACS Lecture Notes in Computer Science vol. 2607 2003 Springer 415 426
58. F. Neumann, and I. Wegener Randomized local search, evolutionary algorithms, and the minimum spanning tree problem Theoretical Computer Science 378 1 2007 32 40
59. J. Scharnow, K. Tinnefeld, and I. Wegener The analysis of evolutionary algorithms on sorting and shortest paths problems Journal of Mathematical Modelling and Algorithms 3 4 2004 349 366
60. S. Baswana, S. Biswas, B. Doerr, T. Friedrich, P.P. Kurur, and F. Neumann Computing single source shortest paths using single-objective fitness Foundations of Genetic Algorithms Workshop, vol. 10 2009 ACM New York, NY, USA 59 66
61. T. Storch How randomized search heuristics find maximum cliques in planar graphs M. Cattolico, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2006) 2006 ACM New York, NY, USA 567 574
62. J. Reichel, M. Skutella, Evolutionary algorithms and matroid optimization problems, in: Lipson [98], pp. 947-954.
63. D.H. Wolpert, and W.G. Macready No free lunch theorems for optimization IEEE Transactions on Evolutionary Computation 1 1 1997 67 82
64. T.M. English Optimization is easy and learning is hard in the typical function A. Zalzala, C. Fonseca, J.-H. Kim, A. Smith, Proceedings of the Congress on Evolutionary Computation (CEC-2000) July 2000 IEEE Press 924 931
65. T.M. English Practical implications of new results in conservation of optimizer performance M. Schoenauer, K. Deb, G. Rudolph, X. Yao, E. Lutton, J.J.M. Guervós, H.-P. Schwefel, Proceedings of the Sixth International Conference on Parallel Problem Solving from Nature (PPSN VI) Lecture Notes in Computer Science vol. 1917 September 2000 Springer Paris, France 69 78
66. C. Schumacher, M.D. Vose, and L.D. Whitley The no free lunch and problem description length L. Spector, E.D. Goodman, A. Wu, W.B. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M.H. Garzon, E. Burke, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001) 2001 Morgan Kaufmann San Francisco, CA, USA 565 570
67. C. Igel, and M. Toussaint On classes of functions for which no free lunch results hold Information Processing Letters 86 6 2003 317 321
68. R. Poli, M. Graff, and N.F. McPhee Free lunches for function and program induction Foundations of Genetic Algorithms Workshop, vol. 10 2009 ACM New York, NY, USA 183 194
69. R. Poli, and M. Graff There is a free lunch for hyper-heuristics, genetic programming and computer scientists L. Vanneschi, S. Gustafson, A. Moraglio, I.D. Falco, M. Ebner, EUROGP-2009 Lecture Notes in Computer Science vol. 5481 2009 Springer 195 207
70. R. Poli, and M. Graff Free lunches for neural network search F. Rothlauf, GECCO 2009 ACM 1291 1298
71. T. Bäck Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms 1996 Oxford University Press New York
72. G. Rudolph Convergence of evolutionary algorithms in general search spaces Proceedings of IEEE Conference on International Conference on Evolutionary Computation (ICEC-1996) May 1996 IEEE Press Nagoya, Japan 50 54
73. R. Poli, W.B. Langdon, M. Clerc, and C.R. Stephens Continuous optimisation theory made easy? Finite-element models of evolutionary strategies, genetic algorithms and particle swarm optimizers C.R. Stephens, M. Toussaint, L.D. Whitley, P.F. Stadler, Foundations of Genetic Algorithms Workshop, vol. 9 Lecture Notes in Computer Science vol. 4436 2007 Springer Mexico City, Mexico 165 193
74. Y. Borenstein, and R. Poli Information landscapes H.-G. Beyer, U.-M. O'Reilly, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2005) June 2005 ACM Washington DC, USA 1515 1522
75. J. Hadamard Sur les problèmes aux dérivées partielles et leur signification physique Princeton University Bulletin 1902 49 52
76. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani Least angle regression Annals of Statistics 32 2 2004 407 499
77. E. Alpaydin Introduction to Machine Learning 2004 MIT Press Cambridge, MA, USA
78. R. Poli, J. Woodward, and E.K. Burke A histogram-matching approach to the evolution of bin-packing strategies IEEE Congress on Evolutionary Computation 2007 IEEE 3500 3507
79. R. Poli TinyGP see Genetic and Evolutionary Computation Conference (GECCO-2004) competition at http://cswww.essex.ac.uk/staff/sml/gecco/TinyGP.html June 2004
80. U.-M. O'Reilly, and F. Oppacher Program search with a hierarchical variable length representation: Genetic programming, simulated annealing and hill climbing Y. Davidor, H.-P. Schwefel, R. Manner, Proceedings of the Third International Conference on Parallel Problem Solving from Nature (PPSN VI) Lecture Notes in Computer Science vol. 866 1994 Springer-Verlag Jerusalem 397 406
81. C. Igel, and M. Hüsken Empirical evaluation of the improved Rprop learning algorithms Neurocomputing 50 2003 105 123
82. K.A. Smith-Miles Cross-disciplinary perspectives on meta-learning for algorithm selection ACM Comput. Surv. 41 1 2008 1 25
83. J.A. Boyan, and A.W. Moore Learning evaluation functions for global optimization and boolean satisfiability Proceeding of the Fifteenth National Conference on Artificial Intelligence and Tenth Innovative Applications of Artificial Intelligence Conference (AAAI-1998, IAAI-1998) 1998 AAAI Press Madison, WI, USA 3 10
84. O. Telelis, and P. Stamatopoulos Combinatorial optimization through statistical instance-based learning Proceedings of the 13th IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2001) 2001 IEEE Computer Society Dallas, TX, USA 203
85. M.G. Lagoudakis, and M.L. Littman Learning to select branching rules in the dpll procedure for satisfiability Electronic Notes in Discrete Mathematics 9 2001 344 359
86. E. Horvitz, Y. Ruan, C.P. Gomes, H.A. Kautz, B. Selman, and D.M. Chickering A Bayesian approach to tackling hard computational problems J.S. Breese, D. Koller, UAI 2001 Morgan Kaufmann 235 244
87. M.G. Lagoudakis, M.L. Littman, Algorithm selection using reinforcement learning, in: Langley [99], pp. 511-518.
88. R. Vuduc, J. Demmel, and J. Bilmes Statistical models for automatic performance tuning V.N. Alexandrov, J. Dongarra, B.A. Juliano, R.S. Renner, C.J.K. Tan, International Conference on Computational Science, vol. 1 Lecture Notes in Computer Science vol. 2073 2001 Springer 117 126
89. N. Thomas, G. Tanase, O. Tkachyshyn, J. Perdue, N.M. Amato, and L. Rauchwerger A framework for adaptive algorithm selection in STAPL K. Pingali, K.A. Yelick, A.S. Grimshaw, PPOPP 2005 ACM 277 288
90. E.A. Brewer High-level optimization via automated statistical modeling Proceedings of the Fifth ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPOPP-1995) 1995 ACM Press Santa Barbara, CA, USA 80 91
91. B. Singer, M.M. Veloso, Learning to predict performance from formula modeling and training data, in: Langley [99], pp. 887-894.
92. C. Gebruers, B. Hnich, D.G. Bridge, and E.C. Freuder Using CBR to select solution strategies in constraint programming H. Muñoz-Avila, F. Ricci, ICCBR Lecture Notes in Computer Science vol. 3620 2005 Springer 222 236
93. C. Gebruers, A. Guerri, B. Hnich, and M. Milano Making choices using structure at the instance level within a case based reasoning framework J.-C. Régin, M. Rueher, CPAIOR Lecture Notes in Computer Science vol. 3011 2004 Springer 380 386
94. C. Gebruers, A. Guerri, Machine learning for portfolio selection using structure at the instance level, in: Wallace [100], p. 794.
95. T. Carchrae, and J.C. Beck Applying machine learning to low-knowledge control of optimization algorithms Computational Intelligence 21 4 2005 372 387
96. L. Lobjois, and M. Lemaître Branch and bound algorithm selection by performance prediction Proceeding of the Fifteenth National Conference on Artificial Intelligence and Tenth Innovative Applications of Artificial Intelligence Conference (AAAI-1998, IAAI-1998) 1998 AAAI Press Madison, WI, USA 353 358
97. S. Johnson Hierarchical clustering schemes Psychometrika 32 3 1967 241 254
98. H. Lipson, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2007) July 2007 ACM London, England, UK
99. P. Langley, Proceedings of the Seventeenth International Conference on Machine Learning (ICML 2000) Stanford University, Stanford, CA, USA, June 29-July 2, 2000 2000 Morgan Kaufmann
100. M. Wallace, Principles and Practice of Constraint Programming - CP 2004, 10th International Conference, CP 2004, Toronto, Canada, September 27-October 1, 2004, Proceedings Lecture Notes in Computer Science vol. 3258 2004 Springer
101. C. Bessiere, Principles and Practice of Constraint Programming - CP 2007, 13th International Conference, CP 2007, Providence, RI, USA, September 23-27, 2007, Proceedings Lecture Notes in Computer Science vol. 4741 2007 Springer