Natural evolution strategies

Journal of Machine Learning Research

Volumn 15, Issue , 2014, Pages 949-980

  Wierstra, Daan ^a   Schaul, Tom ^a   Glasmachers, Tobias ^b   Sun, Yi ^c   Peters, Jan ^d   Schmidhuber, Jürgen ^e  

^a DEEPMIND   (United Kingdom)

^b RUHR UNIVERSITY BOCHUM   (Germany)

^c GOOGLE INC   (United States)

^d DARMSTADT UNIVERSITY OF TECHNOLOGY   (Germany)

^e UNIVERSITY OF LUGANO   (Switzerland)

Author keywords

Black box optimization; Evolution strategies; Natural gradient; Sampling; Stochastic search

Indexed keywords

ALGORITHMS; NORMAL DISTRIBUTION; SAMPLING;

BLACK-BOX OPTIMIZATION; COMPETITIVE PERFORMANCE; EVOLUTION STRATEGIES; HIGH DIMENSIONAL SPACES; MULTI-VARIATE NORMAL DISTRIBUTIONS; NATURAL EVOLUTION; NATURAL GRADIENT; STOCHASTIC SEARCH;

BENCHMARKING;

EID: 84899819580     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (63)

1. Y. Akimoto, Y. Nagata, I. Ono, and S. Kobayashi. Bidirectional relation between CMA evolution strategies and natural evolution strategies. In Parallel Problem Solving from Nature (PPSN), 2010.
2. S. Amari. Natural gradient works eficiently in learning. Neural Computation, 10(2):251-276, 1998. (Pubitemid 128463152)
3. S. Amari and S. C. Douglas. Why natural gradient? In Proceedings of the 1998 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '98), volume 2, pages 1213-1216, 1998.
4. A. Auger. Convergence results for the (1, )-SA-ES using the theory of -irreducible Markov chains. Theoretical Computer Science, 334(1-3):35-69, 2005. (Pubitemid 40467294)
5. A. Berny. Selection and reinforcement learning for combinatorial optimization. In Parallel Problem Solving from Nature PPSN VI, volume 1917, pages 601-610. Springer Berlin /Heidelberg, 2000.
6. A. Berny. Statistical machine learning and combinatorial optimization, pages 287-306. Springer-Verlag, London, UK, 2001.
7. H.-G. Beyer. The Theory of Evolution Strategies. Springer-Verlag, New York, USA, 2001.
8. H.-G. Beyer and H.-P. Schwefel. Evolution strategies: a comprehensive introduction. Natural Computing, 1:3-52, 2002.
9. P. A. N. Bosman and D. Thierens. Expanding from discrete to continuous estimation of distribution algorithms: the IDEA. In Proceedings of the 6th International Conference on Parallel Problem Solving from Nature, pages 767-776, London, UK, 2000. Springer- Verlag.
10. P. A. N. Bosman, J. Grahl, and D. Thierens. Adapted maximum-likelihood Gaussian models for numerical optimization with continuous EDAs. Technical report, 2007.
11. F. Friedrichs and C. Igel. Evolutionary tuning of multiple svm parameters. Neurocomputing, 64:107-117, 2005. (Pubitemid 40425314)
12. T. Glasmachers, T. Schaul, and J. Schmidhuber. A natural evolution strategy for multiobjective optimization. In Parallel Problem Solving from Nature (PPSN), 2010a.
13. T. Glasmachers, T. Schaul, Y. Sun, D. Wierstra, and J. Schmidhuber. Exponential natural evolution strategies. In Genetic and Evolutionary Computation Conference (GECCO), Portland, USA, 2010b.
14. D. E. Goldberg. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1st edition, 1989.ISBN 0201157675.
15. F. Gomez, J. Schmidhuber, and R. Miikkulainen. Accelerated neural evolution through cooperatively coevolved synapses. Journal of Machine Learning Research, 2008.
16. N. Hansen. Benchmarking a BI-population CMA-ES on the BBOB-2009 function testbed. In GECCO (Companion), pages 2389-2396, 2009a.
17. N. Hansen. Benchmarking a BI-population CMA-ES on the BBOB-2009 noisy testbed. In GECCO (Companion), pages 2397-2402, 2009b.
18. N. Hansen and A. Auger. Real-parameter black-box optimization benchmarking 2010: Experimental setup. Technical report, INRIA, 2010.
19. N. Hansen and A. Ostermeier. Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation, 9(2):159-195, 2001.
20. N. Hansen, A. S. P. Niederberger, L. Guzzella, and P. Koumoutsakos. A method for handling uncertainty in evolutionary optimization with an application to feedback control of combustion. Transactions on Evolutionary Computation, 13:180-197, 2009.
21. N. Hansen, S. Finck, R. Ros, and A. Auger. Real-parameter black-box optimization benchmarking 2010: Noiseless function definitions. Technical report, INRIA, 2010a.
22. N. Hansen, S. Finck, R. Ros, and A. Auger. Real-parameter black-box optimization benchmarking 2010: Noisy functions definitions. Technical report, INRIA, 2010b.
23. M. Hasenjäger, B. Sendhó, T. Sonoda, and T. Arima. Three dimensional evolutionary aerodynamic design optimization with CMA-ES. In Proceedings of the 2005 Conference on Genetic and Evolutionary Computation, pages 2173-2180, New York, NY, USA, 2005. ACM. (Pubitemid 43226582)
24. J. H. Holland. Adaptation in Natural and Artificial Systems. The University of Michigan Press, Ann Arbor, 1975.
25. C. Igel and M. Hüsken. Empirical evaluation of the improved Rprop learning algorithm. Neurocomputing, 50:2003, 2003.
26. J. Jägersküpper. Analysis of a simple evolutionary algorithm for minimization in Euclidean spaces. Theoretical Computer Science, 379(3):329-347, 2007. (Pubitemid 46754584)
27. G. A. Jastrebski and D. V. Arnold. Improving evolution strategies through active covariance matrix adaptation. In IEEE Congress on Evolutionary Computation, 2006.
28. M. Jebalia, A. Auger, M. Schoenauer, F. James, and M. Postel. Identification of the isotherm function in chromatography using CMA-ES. In IEEE Congress on Evolutionary Computation, pages 4289-4296, 2007.
29. M. Jebalia, A. Auger, and N. Hansen. Log-linear convergence and divergence of the scaleinvariant (1+1)-ES in noisy environments. Algorithmica, pages 1-36, 2010.
30. J. Kennedy and R. Eberhart. Swarm Intelligence. Morgan Kaufmann, San Francisco, CA, 2001.
31. S. Kirkpatrick, C. D. Gelatt, Jr, and M. P. Vecchi. Optimization by simulated annealing. Science, 220:671-680, 1983.
32. J. Klockgether and H. P. Schwefel. Two-phase nozzle and hollow core jet experiments. In Proc. 11th Symp. Engineering Aspects of Magnetohydrodynamics, pages 141-148, 1970.
33. S. Kullback and R. A. Leibler. On information and suffciency. The Annals of Mathematical Statistics, 22(1):79-86, 1951. ISSN 00034851.
34. P. Larra-naga. Estimation of Distribution Algorithms. A New Tool for Evolutionary Computation. Kluwer Academic Publishers 2002.
35. H. Mühlenbein and G. Paass. From recombination of genes to the estimation of distributions I. binary parameters. In Proceedings of the 4th International Conference on Parallel Problem Solving from Nature, PPSN IV, pages 178-187, London, UK, 1996. Springer- Verlag. (Pubitemid 126128306)
36. S. D. Muller, J. Marchetto, S. Airaghi, and P. Koumoutsakos. Optimization based on bacterial chemotaxis. IEEE Transactions on Evolutionary Computation, 6:6-16, 2002.
37. J. A. Nelder and R. Mead. A simplex method for function minimization. The Computer Journal, 7(4):308-313, 1965.
38. A. Ostermeier, A. Gawelczyk, and N. Hansen. Step-size adaption based on non-local use of selection information. In The Third Conference on Parallel Problem Solving from Nature, pages 189-198, London, UK, 1994. Springer-Verlag.
39. M. Pelikan, D. Goldberg, and F. Lobo. A survey of optimization by building and using probabilistic models. In American Control Conference, volume 5, pages 3289 -3293 vol.5, 2000. (Pubitemid 32027961)
40. M. Pelikan, K. Sastry, and E. C. Paz. Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications (Studies in Computational Intelligence). Springer-Verlag New York, Inc., 2006.
41. J. Peters. Machine Learning of Motor Skills for Robotics. PhD thesis, Department of Computer Science, University of Southern California, 2007.
42. I. Rechenberg and M. Eigen. Evolutionsstrategie: Optimierung Technischer Systeme nach Prinzipien der Biologischen Evolution. Frommann-Holzboog Stuttgart, 1973.
43. M. Riedmiller and H. Braun. A direct adaptive method for faster backpropagation learning: The RPROP algorithm. In IEEE International Conference on Neural Networks, pages 586-591. IEEE Press, 1993.
44. R. Ros and N. Hansen. A simple modification in CMA-ES achieving linear time and space complexity. In R. et al., editor, Parallel Problem Solving from Nature, PPSN X, pages 296-305. Springer, 2008.
45. R. Y. Rubinstein and D. P. Kroese. The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation and Machine Learning (Information Science and Statistics). Springer, 2004.
46. T. Schaul. Benchmarking separable natural evolution strategies on the noiseless and noisy black-box optimization testbeds. In Black-box Optimization Benchmarking Workshop, Genetic and Evolutionary Computation Conference, Philadelphia, PA, 2012a.
47. T. Schaul. Benchmarking exponential natural evolution strategies on the noiseless and noisy black-box optimization testbeds. In Black-box Optimization Benchmarking Workshop, Genetic and Evolutionary Computation Conference, Philadelphia, PA, 2012b.
48. T. Schaul. Benchmarking natural evolution strategies with adaptation sampling on the noiseless and noisy black-box optimization testbeds. In Black-box Optimization Benchmarking Workshop, Genetic and Evolutionary Computation Conference, Philadelphia, PA, 2012c.
49. T. Schaul. Comparing natural evolution strategies to BIPOP-CMA-ES on noiseless and noisy black-box optimization testbeds. In Black-box Optimization Benchmarking Workshop, Genetic and Evolutionary Computation Conference, Philadelphia, PA, 2012d.
50. T. Schaul. Investigating the impact of adaptation sampling in natural evolution strategies on black-box optimization testbeds. In Black-box Optimization Benchmarking Workshop, Genetic and Evolutionary Computation Conference, Philadelphia, PA, 2012e.
51. T. Schaul. Natural evolution strategies converge on sphere functions. In Genetic and Evolutionary Computation Conference, Philadelphia, PA, 2012f.
52. T. Schaul and J. Schmidhuber. Metalearning. Scholarpedia, 5(6):4650, 2010.
53. T. Schaul, J. Bayer, D. Wierstra, Y. Sun, M. Felder, F. Sehnke, T. Rückstié, and J. Schmidhuber. PyBrain. Journal of Machine Learning Research, 11:743-746, 2010.
54. T. Schaul, T. Glasmachers, and J. Schmidhuber. High dimensions and heavy tails for natural evolution strategies. In Genetic and Evolutionary Computation Conference, 2011.
55. H.-P. Schwefel. Numerische optimierung von computer-modellen mittels der evolutionsstrategie, 1977.
56. J. Shepherd, D. McDowell, and K. Jacob. Modeling morphology evolution and mechanical behavior during thermo-mechanical processing of semi-crystalline polymers. Journal of the Mechanics and Physics of Solids, 54(3):467-489, 2006. (Pubitemid 43031538)
57. O. M. Shir and T. Bäck. The second harmonic generation case-study as a gateway for ES to quantum control problems. In Proceedings of the 9th annual conference on Genetic and evolutionary computation, GECCO '07, pages 713-721, New York, NY, USA, 2007. ACM. (Pubitemid 47291606)
58. R. Storn and K. Price. Differential evolution - a simple and eficient heuristic for global optimization over continuous spaces. J. of Global Optimization, 11:341-359, December 1997. ISSN 0925-5001. (Pubitemid 127502202)
59. Y. Sun, D. Wierstra, T. Schaul, and J. Schmidhuber. Stochastic search using the natural gradient. In International Conference on Machine Learning (ICML), 2009a.
60. Y. Sun, D. Wierstra, T. Schaul, and J. Schmidhuber. Eficient natural evolution strategies. In Genetic and Evolutionary Computation Conference (GECCO), 2009b.
61. A. Wieland. Evolving neural network controllers for unstable systems. In Proceedings of the International Joint Conference on Neural Networks (Seattle, WA), pages 667-673, 1991.
62. D. Wierstra, T. Schaul, J. Peters, and J. Schmidhuber. Natural evolution strategies. In Proceedings of the Congress on Evolutionary Computation (CEC08), Hongkong. IEEE Press, 2008.
63. S. Winter, B. Brendel, and C. Igel. Registration of bone structures in 3D ultrasound and CT data: Comparison of different optimization strategies. International Congress Series, 1281:242-247, 2005. (Pubitemid 43693570)