Efficient covariance matrix update for variable metric evolution strategies

Machine Learning

Volumn 75, Issue 2, 2009, Pages 167-197

  Suttorp, Thorsten ^a   Hansen, Nikolaus ^b   Igel, Christian ^a  

^a RUHR UNIVERSITY BOCHUM   (Germany)

^b Parc Orsay Université   (France)

Author keywords

Covariance matrix adaptation; Evolution strategy; Evolutionary algorithm; Multi objective optimization; Reinforcement learning; Stochastic optimization; Variable metric algorithm

Indexed keywords

COMPUTATIONAL COMPLEXITY; DECOMPOSITION; EDUCATION; EVOLUTIONARY ALGORITHMS; HEURISTIC METHODS; LEARNING ALGORITHMS; MACHINE TOOLS; MULTIOBJECTIVE OPTIMIZATION; NORMAL DISTRIBUTION; RANDOM PROCESSES; REINFORCEMENT; REINFORCEMENT LEARNING; ROBOT LEARNING;

ALGEBRAIC COMPUTATIONS; APPLICATION AREAS; CANDIDATE SOLUTIONS; CONTINUOUS DOMAINS; COVARIANCE MATRIX ADAPTATION; COVARIANCE MATRIX ADAPTATION EVOLUTION STRATEGIES; DIRECT SEARCH ALGORITHMS; EVOLUTION STRATEGY; ILL-CONDITIONED; MACHINE LEARNING PROBLEMS; MACHINE-LEARNING; MODEL SELECTIONS; MULTI OBJECTIVES; OBJECTIVE FUNCTIONS; RANDOMIZED SEARCH STRATEGIES; REWARD FUNCTIONS; SEARCH SPACE DIMENSIONS; STOCHASTIC OPTIMIZATION; VARIABLE METRIC ALGORITHM; VARIABLE METRIC METHODS;

COVARIANCE MATRIX;

EID: 62149099100     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-009-5102-1     Document Type: Article

References (75)

1. Arnold, D. V. (2002). Noisy optimization with evolution strategies. Dordrecht: Kluwer Academic.
2. D. V. Arnold 2006 Weighted multirecombination evolution strategies Theoretical Computer Science 361 1 18 37
3. D. V. Arnold H. G. Beyer 2003 A comparison of evolution strategies with other direct search methods in the presence of noise Computational Optimization and Applications 24 1 135 159
4. A. Auger 2005 Convergence results for the (1, λ)-SA-ES using the theory of φ-irreducible Markov chains Theoretical Computer Science 334 1 35 69
5. Auger, A., & Hansen, N. (2005a). Performance evaluation of an advanced local search evolutionary algorithm. In Proceedings of the IEEE congress on evolutionary computation (CEC 2005) (pp. 1777-1784). New York: IEEE Press.
6. Auger, A., & Hansen, N. (2005b). A restart CMA evolution strategy with increasing population size. In Proceedings of the IEEE congress on evolutionary computation (CEC 2005) (pp. 1769-1776). New York: IEEE Press.
7. Beume, N., & Rudolph, G. (2006). Faster S-metric calculation by considering dominated hypervolume as Klee's measure problem. In IASTED international conference on computational intelligence (pp. 231-236). Calgary: ACTA Press.
8. N. Beume B. Naujoks M. Emmerich 2007 SMS-EMOA: Multiobjective selection based on dominated hypervolume European Journal of Operational Research 181 3 1653 1669
9. Beyer, H. G. (2001). The theory of evolution strategies. New York: Springer.
10. H. G. Beyer 2007 Evolution strategies Scholarpedia 2 8 1965
11. H. G. Beyer H. P. Schwefel 2002 Evolution strategies: A comprehensive introduction Natural Computing 1 1 3 52
12. Deb, K. (2001). Multi-objective optimization using evolutionary algorithms. New York: Wiley.
13. K. Deb S. Agrawal A. Pratap T. Meyarivan 2002 A fast and elitist multiobjective genetic algorithm: NSGA-II IEEE Transactions on Evolutionary Computation 6 2 182 197
14. Emmerich, M., Beume, N., & Naujoks, B. (2005). An EMO algorithm using the hypervolume measure as selection criterion. In C. A. Coello Coello, E. Zitzler, & A. Hernandez Aguirre (Eds.), LNCS: Vol. 3410. Third international conference on evolutionary multi-criterion optimization (EMO 2005) (pp. 62-76). New York: Springer.
15. Fogel, D. B. (1995). Evolutionary computation: Toward a new philosophy of machine intelligence. New York: IEEE Press.
16. Fonseca, C. M., Paquete, L., & López-Ibáñez, M. (2006). An improved dimension-sweep algorithm for the hypervolume indicator. In: IEEE congress on evolutionary computation (pp. 1157-1163).
17. F. Friedrichs C. Igel 2005 Evolutionary tuning of multiple SVM parameters Neurocomputing 64 C 107 117
18. Glasmachers, T., & Igel, C. (2008). Uncertainty handling in model selection for support vector machines. In G. Rudolph (Ed.), LNCS: Vol. 5199. Parallel problem solving from nature (PPSN X) (pp. 185-194). New York: Springer.
19. Gomez, F., Schmidhuber, J., & Miikkulainen, R. (2006). Efficient non-linear control through neuroevolution. LNCS: Vol. 4212. In Proceedings of the European conference on machine learning (ECML 2006) (pp. 654-662). New York: Springer.
20. Grewal, M. S., & Andrews, A. P. (1993). Kalman filtering: Theory and practice. Englewood Cliffs: Prentice-Hall.
21. W. W. Hager 1989 Updating the inverse of a matrix SIAM Review 31 2 221 239
22. Hansen, N. (1998). Verallgemeinerte individuelle Schrittweitenregelung in der Evolutionsstrategie. Eine Untersuchung zur entstochastisierten, koordinatensystemunabhängigen Adaptation der Mutationsverteilung. Berlin: Mensch und Buch Verlag. ISBN 3-933346-29-0.
23. Hansen, N. (2006). The CMA evolution strategy: a comparing review. In J. Lozano, P. Larranaga, I. Inza, & E. Bengoetxea (Eds.), Towards a new evolutionary computation. Advances on estimation of distribution algorithms (pp. 75-102). New York: Springer.
24. Hansen, N. (2008). http://www.bionik.tu-berlin.de/user/niko/ cmaapplications.pdf.
25. Hansen, N., & Kern, S. (2004). Evaluating the CMA evolution strategy on multimodal test functions. In X. Yao, E. Burke, J. A. Lozano, J. Smith, J. J. Merelo-Guervós, J. A. Bullinaria, J. Rowe, P. Tiňo, A. Kabán, & H. P. Schwefel (Eds.), LNCS: Vol. 3242. Parallel problem solving from nature (PPSN VIII) (pp. 282-291). New York: Springer.
26. Hansen, N., & Ostermeier, A. (1996). Adapting arbitrary normal mutation distributions in evolution strategies: The covariance matrix adaptation. In Proceedings of the 1996 IEEE conference on evolutionary computation (ICEC '96) (pp. 312-317).
27. N. Hansen A. Ostermeier 2001 Completely derandomized self-adaptation in evolution strategies Evolutionary Computation 9 2 159 195
28. Hansen, N., Ostermeier, A., & Gawelczyk, A. (1995). On the adaptation of arbitrary normal mutation distributions in evolution strategies: The generating set adaptation. In L. Eshelman (Ed.), Proceedings of the sixth international conference on genetic algorithms (pp. 57-64). San Francisco: Morgan Kaufmann.
29. N. Hansen S. D. Müller P. Koumoutsakos 2003 Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES) Evolutionary Computation 11 1 1 18
30. Hansen, N., Niederberger, S. P. N., Guzzella, L., & Koumoutsakos, P. (2008). A method for handling uncertainty in evolutionary optimization with an application to feedback control of combustion. IEEE Transactions on Evolutionary Computation. doi: 10.1109/TEVC.2008.924423.
31. Heidrich-Meisner, V., & Igel, C. (2008a). Evolution strategies for direct policy search. In G. Rudolph (Ed.), LNCS: Vol. 5199. Parallel problem solving from nature (PPSN X) (pp. 428-437). New York: Springer.
32. Heidrich-Meisner, V., & Igel, C. (2008b). Similarities and differences between policy gradient methods and evolution strategies. In: M. Verleysen (Ed.), 16th European symposium on artificial neural networks (ESANN 2008), Evere, Belgien: d-side publications (pp. 427-432).
33. Heidrich-Meisner, V., & Igel, C. (2008c). Variable metric reinforcement learning methods applied to the noisy mountain car problem. In S. Girgin et al. (Eds.), LNAI: Vol. 5323. European workshop on reinforcement learning (EWRL 2008) (pp. 136-150). New York: Springer.
34. Igel, C. (2003). Neuroevolution for reinforcement learning using evolution strategies. In R. Sarker, R. Reynolds, H. Abbass, K. C. Tan, B. McKay, D. Essam, & T. Gedeon (Eds.), Vol. 4. Congress on evolutionary computation (CEC 2003) (pp. 2588-2595). New York: IEEE Press.
35. Igel, C. (2005). Multi-objective model selection for support vector machines. In C. A. Coello Coello, E. Zitzler, & A. Hernandez Aguirre (Eds.), LNAI: Vol. 3410. Third international conference on evolutionary multi-criterion optimization (EMO 2005) (pp. 534-546). New York: Springer.
36. C. Igel M. Toussaint 2003 On classes of functions for which No Free Lunch results hold Information Processing Letters 86 6 317 321
37. C. Igel W. Erlhagen D. Jancke 2001 Optimization of dynamic neural fields Neurocomputing 36 1-4 225 233
38. Igel, C., Suttorp, T., & Hansen, N. (2006). A computational efficient covariance matrix update and a (1+1)-CMA for evolution strategies. In Proceedings of the genetic and evolutionary computation conference (GECCO 2006) (pp. 453-460). New York: ACM Press.
39. C. Igel N. Hansen S. Roth 2007 Covariance matrix adaptation for multi-objective optimization Evolutionary Computation 15 1 1 28
40. Igel, C., Suttorp, T., & Hansen, N. (2007b). Steady-state selection and efficient covariance matrix update in the multi-objective CMA-ES. In S. Obayashi, K. Deb, C. Poloni, T. Hiroyasu, & T. Murata (Eds.), LNCS: Vol. 4403. Fourth international conference on evolutionary multi-criterion optimization (EMO 2007) (pp. 171-185). New York: Springer.
41. C. Igel T. Glasmachers V. Heidrich-Meisner 2008 Shark Journal of Machine Learning Research 9 993 996
42. Iorio, A., & Li, X. (2005). Solving rotated multi-objective optimization problems using differential evolution. In G. I. Webb & X. Yu (Eds.), LNCS: Vol. 3339. Proceeding of the 17th joint Australian conference on artificial intelligence (pp. 861-872). New York: Springer.
43. J. Jägersküpper 2006 How the (1+1) ES using isotropic mutations minimizes positive definite quadratic forms Theoretical Computer Science 36 1 38 56
44. Jägersküpper, J. (2007). Lower bounds for hit-and-run direct search. In Proceedings of the fourth international symposium on stochastic algorithms: foundations and applications (SAGA) (pp. 118-129). New York: Springer.
45. J. Jägersküpper 2008 Lower bounds for randomized direct search with isotropic sampling Operations Research Letter 36 3 327 332
46. Jin, Y. (Ed.). (2006). Multi-objective machine learning. New York: Springer.
47. Jones, T., & Forrest, S. (1995). Fitness distance correlation as a measure of problem difficulty for genetic algorithms. In L. Eshelman (Ed.), Proceedings of the 6th international conference on genetic algorithms (pp. 184-192). San Francisco: Morgan Kaufmann.
48. Kassahun, Y., & Sommer, G. (2005). Efficient reinforcement learning through evolutionary acquisition of neural topologies. In M. Verleysen (Ed.), 13th European symposium on artificial neural networks (ESANN 2005) d-side (pp. 259-266).
49. S. Kern S. Müller N. Hansen D. Büche J. Ocenasek P. Koumoutsakos 2004 Learning probability distributions in continuous evolutionary algorithms-A comparative review Natural Computing 3 77 112
50. i ] be computed in less than O(nlog∈n) steps? American Mathematical Monthly 84 284 285
51. Knight, J. N., & Lunacek, M. (2007). Reducing the space-time complexity of the CMA-ES. In Proceedings of the genetic and evolutionary computation conference (GECCO 2007) (pp. 658-665). New York: ACM Press.
52. M. Mandischer 2002 A comparison of evolution strategies and backpropagation for neural network training Neurocomputing 42 1-4 87 117
53. B. Mersch T. Glasmachers P. Meinicke C. Igel 2007 Evolutionary optimization of sequence kernels for detection of bacterial gene starts International Journal of Neural Systems 17 5 369 381 special issue on selected papers presented at the international conference on artificial neural networks (ICANN 2006)
54. Mierswa, I. (2006). Evolutionary learning with kernels: a generic solution for large margin problems. In Proceedings of the 8th annual conference on genetic and evolutionary computation (GECCO 2006) (pp. 1553-1560).
55. Mierswa, I. (2007). Controlling overfitting with multi-objective support vector machines. In Proceedings of the 9th annual conference on genetic and evolutionary computation (GECCO 2007) (pp. 1830-1837).
56. Miettinen, K. (1999). Kluwer's international series in operations research & management science: Vol. 12: Nonlinear multiobjective optimization. Dordrecht: Kluwer Academic.
57. A. Ostermeier 1992 An evolution strategy with momentum adaptation of the random number distribution Parallel Problem Solving from Nature 2 197 206
58. M. H. Overmars C. K. Yap 1991 New upper bounds in Klee's measure problem SIAM Journal on Computing 20 6 1034 1045
59. A. Pellecchia C. Igel J. Edelbrunner G. Schöner 2005 Making driver modeling attractive IEEE Intelligent Systems 20 2 8 12
60. Poland, J., & Zell, A. (2001). Main vector adaptation: A CMA variant with linear time and space complexity. In L. Spector, E. D. Goodman, A. Wu, W. B. Langdon, H. M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. H. Garzon, & E. Burke (Eds.), Proceedings of the genetic and evolutionary computation conference (GECCO 2001) (pp. 1050-1055). San Francisco: Morgan Kaufmann.
61. Rechenberg, I. (1973). Evolutionsstrategie: Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. Stuttgart: Frommann-Holzboog.
62. Ros, R., & Hansen, N. (2008). A simple modification in CMA-ES achieving linear time and space complexity. In G. Rudolph (Ed.), LNCS: Vol. 5199. Parallel problem solving from nature (PPSN X) (pp. 296-305). New York: Springer.
63. Rudolph, G. (1992). On correlated mutations in evolution strategies. In R. Männer & B. Manderick (Eds.), Parallel problem solving from nature 2 (PPSN II) (pp. 105-114). Amsterdam: Elsevier.
64. T. P. Runarsson S. Sigurdsson 2004 Asynchronous parallel evolutionary model selection for support vector machines Neural Information Processing-Letters and Reviews 3 3 59 68
65. S. Schneider C. Igel C. Klaes H. Dinse J. Wiemer 2004 Evolutionary adaptation of nonlinear dynamical systems in computational neuroscience Journal of Genetic Programming and Evolvable Machines 5 2 215 227
66. M. Schumer K. Steiglitz 1968 Adaptive step size random search IEEE Transactions on Automatic Control 13 3 270 276
67. Schwefel, H. P. (1995). Sixth-generation computer technology series: Evolution and optimum seeking. New York: Wiley.
68. N. T. Siebel G. Sommer 2007 Evolutionary reinforcement learning of artificial neural networks International Journal of Hybrid Intelligent Systems 4 3 171 183
69. Sutton, R., & Barto, A. (1998). Reinforcement learning: An Introduction. Cambridge: MIT Press.
70. Suttorp, T., & Igel, C. (2006). Multi-objective optimization of support vector machines. In Y. Jin (Ed.), Studies in computational intelligence: Vol. 16. Multi-objective machine learning (pp. 199-220). New York: Springer.
71. Voß, T., Beume, N., Rudolph, G., & Igel, C. (2008). Scalarization versus indicator-based selection in multi-objective CMA evolution strategies. In IEEE congress on evolutionary computation 2008 (CEC 2008) (pp. 3041-3048). New York: IEEE Press.
72. Whitley, D., Lunacek, M., & Sokolov, A. (2006). Comparing the niches of CMA-ES, CHC and pattern search using diverse benchmarks. LNCS: Vol. 4193. In Parallel problem solving from nature (PPSN IX) (pp. 988-997). New York: Springer.
73. S. Winter B. Brendel I. Pechlivanis K. Schmieder C. Igel 2008 Registration of CT and intraoperative 3D ultrasound images of the spine using evolutionary and gradient-based methods IEEE Transactions on Evolutionary Computation 12 3 284 296
74. Zitzler, E., & Thiele, L. (1998). Multiobjective optimization using evolutionary algorithms-a comparative case study. In A. E. Eiben, T. Bäck, M. Schoenauer, & H. P. Schwefel (Eds.), Parallel problem solving from nature (PPSN V) (pp. 292-301). New York: Springer.
75. E. Zitzler L. Thiele M. Laumanns C. M. Fonseca V. Grunert da Fonseca 2003 Performance assessment of multiobjective optimizers: An analysis and review IEEE Transactions on Evolutionary Computation 7 2 117 132