Abandoning objectives: Evolution through the search for novelty alone

Evolutionary Computation

Volumn 19, Issue 2, 2011, Pages 189-222

  Lehman, Joel ^a   Stanley, Kenneth O ^a  

^a University of Central Florida   (United States)

Author keywords

Deception; Evolutionary algorithms; Neuroevolution; Novelty search; Open ended evolution

Indexed keywords

ARTIFICIAL LIFE; BIPED WALKING; DEAD-ENDS; DECEPTION; EVOLUTIONARY COMPUTATIONS; FITNESS FUNCTIONS; INHERENT LIMITATIONS; NATURAL EVOLUTION; NEUROEVOLUTION; NOVELTY SEARCH; OBJECTIVE FUNCTIONS; OPEN-ENDED EVOLUTION; REAL-WORLD PROBLEM; SEARCH SPACES; SIMPLE BEHAVIORS;

EVOLUTIONARY ALGORITHMS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; EVOLUTION;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; BIOLOGICAL EVOLUTION;

EID: 79957482984     PISSN: 10636560     EISSN: 15309304     Source Type: Journal    
DOI: 10.1162/EVCO_a_00025     Document Type: Article

References (106)

1. Aaltonen, T., Adelman, J., Akimoto, T., Álvarez González, B., Amerio, S., Amidei, D., et al. (2009). Measurement of the top quark mass with dilepton events selected using neuroevolution at CDF. Physical Review Letters D, 79 (7): 072005.
2. Adami, C., Ofria, C., and Collier, T. C. (2000). Evolution of biological complexity. Proceedings of the National Academy of Sciences, USA, 97:4463-4468.
3. Allen, B., and Faloutsos, P. (2009). Complex networks of simple neurons for bipedal locomotion. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 4457-4462.
4. Arthur, W. (1994). On the evolution of complexity. In G. Cownan, D. Pines, and D. Meltzer (Eds.), Complexity: Metaphors, Models and Reality (pp. 65-81). Reading, MA: Addison-Wesley.
5. Barnett, L. (2001). Netcrawling-Optimal evolutionary search with neutral networks. In Proceedings of the 2001 IEEE International Conference on Evolutionary Computation, pp. 30-37.
6. Bedau, M. (1998). Four puzzles about life. Artificial Life, 4:125-140.
7. Bedau, M. A., and Packard, N. H. (1991). Measurement of evolutionary activity, teleology, and life. In C. G. Langton, C. Taylor, J. D. Farmer, and S. Rasmussen (Eds.), Proceedings of Artificial Life II, pp. 431-461.
8. Bedau, M. A., Snyder, E., and Packard, N. H. (1998). A classification of longterm evolutionary dynamics. In C. Adami, R. Belew, H. Kitano, and C. Taylor (Eds.), Proceedings of Artificial Life VI, pp. 228-237.
9. Benbrahim, H. (1996). Biped dynamic walking using reinforcement learning. PhD thesis, University of New Hampshire, Durham, New Hampshire.
10. Brockhoff, D., Friedrich, T., Hebbinghaus, N., Klein, C., Neumann, F., and Zitzler, E. (2007). Do additional objectives make a problem harder? In GECCO '07: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, pp. 765-772.
11. Cartlidge, J., and Bullock, S. (2004a). Combating coevolutionary disengagement by reducing parasite virulence. Evolutionary Computation, 12(2):193-222.
12. Cartlidge, J., and Bullock, S. (2004b). Unpicking tartan CIAO plots: Understanding irregular coevolutionary cycling. Adaptive Behavior, 12(2):69-92.
13. Channon, A. (2001). Passing the Alife test: Activity statistics classify evolution in Geb as unbounded. In Proceedings of the European Conference on Artificial Life (ECAL-2001), 41-47.
14. Channon, A. D., and Damper, R. I. (2000). Towards the evolutionary emergence of increasingly complex advantageous behaviours. International Journal of Systems Science, 31(7):843-860.
15. Chellapilla, K., and Fogel, D. B. (1999). Evolving neural networks to play checkers without relying on expert knowledge. IEEE Transactions on Neural Networks, 10(6):1382-1391.
16. Cliff, D., andMiller, G. (1995). Tracking the red queen:Measurements of adaptive progress in coevolutionary simulations. In F. Morán, A. Moreno, J. J. Merelo, & P. Chaćon (Eds.), Advances in artificial life: Proceedings of the third European conference on artificial life (pp. 200-218). Berlin: Springer-Verlag.
17. Davidor, Y. (1990). Epistasis variance: A viewpoint on GA-hardness. In Proceedings of the First Workshop on Foundations of Genetic Algorithms, pp. 23-35.
18. De Jong, E. D. (2004). The incremental Pareto-coevolution archive. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2004), 525-536.
19. De Jong, E. D., and Pollack, J. B. (2003). Multi-objective methods for tree size control. Genetic Programming and Evolvable Machines, 4(3):211-233.
20. De Jong, K. A. (2006). Evolutionary computation: A unified approach. Cambridge, MA: MIT Press.
21. Deb, K. (1999). Multi-objective genetic algorithms: Problem difficulties and construction of test problems. Evolutionary Computation, 7:205-230.
22. Elman, J. L. (1991). Incremental learning, or the importance of starting small. Technical Report 9101, CRL, La Jolla, CA.
23. Endler, J. A., and Greenwood, J. J. D. (1988). Frequency-dependent predation, crypsis and aposematic coloration [and discussion]. Philosophical Transactions of the Royal Society of London. B, Biological Sciences, 319(1196):505-523.
24. Ficici, S., and Pollack, J. B. (1998). Challenges in coevolutionary learning: Arms-race dynamics, open-endedness, andmediocre stable states. In Proceedings of the Sixth International Conference on Artificial Life, pp. 238-247.
25. Fogel, L., Owens, A., and Walsh, M. (1966). Artificial intelligence through simulated evolution. New York: JohnWiley.
26. Glover, F. (1989). Tabu search, part I. ORSA Journal on Computing, 1(3):190-206.
27. Goldberg, D. E. (1987). Simple genetic algorithms and the minimal deceptive problem. In L. D. Davis (Ed.), Genetic Algorithms and Simulated Annealing, Research Notes in Artificial Intelligence (Chap. 6, pp. 71-88).
28. Goldberg, D. E., Deb, K., and Korb, B. (1989). Messy genetic algorithms: Motivation, analysis, and first results. Complex Systems, (3):493-530.
29. Goldberg, D. E., and Richardson, J. (1987). Genetic algorithms with sharing for multimodal function optimization. In Proceedings of the Second International Conference on Genetic Algorithms, pp. 41-49.
30. Gomez, F. (2009). Sustaining diversity using behavioral information distance. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2009), pp. 113-120.
31. Gomez, F., and Miikkulainen, R. (1997). Incremental evolution of complex general behavior. Adaptive Behavior, 5:317-342.
32. Gould, S. J. (1996). Full house: The spread of excellence from Plato to Darwin. NewYork:Harmony Books.
33. Grefenstette, J. J. (1992). Deception considered harmful. In Foundations of Genetic Algorithms 2 (pp. 75-91). San Mateo, CA: Morgan Kaufmann.
34. Greiner, D., Emerador, J. M., Winter, G., and Galvan, B. (2007). Improving computational mechanics optimum design using helper objectives: An application in frame bar structures. In Proceedings of the 4th International Conference on Evolutionary Multi-Criterion Optimization, 575-589.
35. Handl, J., Lovell, S. C., and Knowles, J. (2008a). Investigations into the effect of multiobjectivization in protein structure prediction. In Proceedings of the 10th International Conference on Parallel Problem Solving from Nature, pp. 702-711.
36. Handl, J., Lovell, S. C., and Knowles, J. (2008b). Multiobjectivization by decomposition of scalar cost functions. In Proceedings of the 10th International Conference on Parallel Problem Solving from Nature, pp. 31-40.
37. Harvey, I. (1993). The artificial evolution of adaptive behavior. PhD thesis, School of Cognitive and Computing Sciences, University of Sussex, England.
38. Harvey, I., and Thompson, A. (1996). Through the labyrinth evolution finds a way: A silicon ridge. In Proceedings of the First International Conference on Evolvable Systems: From Biology to Hardware, pp. 406-422.
39. Hein, D., Hild, M., and Berger, R. (2007). Evolution of biped walking using neural oscillators and physical simulation. In RoboCup 2007: Proceedings of the International Symposium, Lecture Notes in Artificial Intelligence (vol. 5001, pp. 433-440). Berlin: Springer.
40. Hillis, W. D. (1991). Co-evolving parasites improve simulated evolution as an optimization procedure. In J. D. Farmer, C. Langton, S. Rasmussen, and C. Taylor (Eds.), Artificial Life II.
41. Holland, J. H. (1975). Adaptation in natural and artificial systems: An introductory analysis with applications to biology, control and artificial intelligence. Ann Arbor, MI: University ofMichigan Press.
42. Hordijk, W. (1995). A measure of landscapes. Evolutionary Computation, 4:335-360.
43. Hornby, G. S. (2006). ALPS: The age-layered population structure for reducing the problem of premature convergence. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2006), pp. 815-822.
44. Hu, J., Goodman, E., Seo, K., Fan, Z., and Rosenberg, R. (2005). The hierarchical fair competition (HFC) framework for sustainable evolutionary algorithms. Evolutionary Computation, 13(2):241-277.
45. Hughes, K. A., Du, L., Rodd, F. H., and Reznick, D. N. (1999). Familiarity leads to female mate preference for novel males in the guppy, poecilia reticulata. Animal Behavior, 58(4):907-916.
46. Hutter, M., and Legg, S. (2006). Fitness uniform optimization. IEEE Transactions on Evolutionary Computation, 10:568-589.
47. Huyen, M. (1995). Exploring phenotype space through neutral evolution. Working Papers 95-10-100, Santa Fe Institute.
48. Ishiguro, A., Fujii, A., and Hotz, P. E. (2003). Neuromodulated control of bipedal locomotion using a polymorphic CPG circuit. Adaptive Behavior, 11(1):7-17.
49. Jones, T., and Forrest, S. (1995). Fitness distance correlation as ameasure of problem difficulty for genetic algorithms. In Proceedings of the Sixth International Conference on Genetic Algorithms, pp. 184-192.
50. Kampis, G., and Gulyás, L. (2008). Full body: The importance of the phenotype in evolution. Artificial Life, 14(3):375-386.
51. Kaplan, F., and Hafner, V. (2006). Information-theoretic framework for unsupervised activity classification. Advanced Robotics, 20(10):1087-1103.
52. Katíc, D., and Vukobratovíc, M. (2003). Survey of intelligent control techniques for humanoid robots. Journal of Intelligent Robotics Systems, 37(2):117-141.
53. Kauffman, S. A. (1993). The origins of order: Self-organization and selection in evolution. NewYork: Oxford University Press.
54. Kimura, M. (1983). The neutral theory of molecular evolution. Cambridge, UK: CambridgeUniversity Press.
55. Kirkpatrick, S., Gelatt, C. D., Jr., and Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220:671-680.
56. Knowles, J. D., Watson, R. A., and Corne, D. (2001). Reducing local optima in single-objective problems by multi-objectivization. InEMO'01: Proceedings of the First International Conference on Evolutionary Multi-Criterion Optimization, pp. 269-283.
57. Lehman, J., and Stanley, K. O. (2008). Exploiting open-endedness to solve problems through the search for novelty. In Proceedings of the Eleventh International Conference on Artificial Life (ALIFE XI), pp. 329-336.
58. Lehman, J., and Stanley, K. O. (2010a). Efficiently evolving programs through the search for novelty. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2010). ACM, pp. 837-844.
59. Lehman, J., and Stanley, K. O. (2010b). Revising the evolutionary computation abstraction: Minimal criteria novelty search. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2010). ACM, pp. 103-110.
60. Liepins, G. E., and Vose, M. D. (1990). Deceptiveness and genetic algorithm dynamics. In G. J. E. Rawlins (Ed.), FOGA (Foundations of Genetic Algorithms), pp. 36-50.
61. Lynch, M. (2007a). The evolution of genetic networks by non-adaptive processes. Nature Reviews Genetics, 8:803-813.
62. Lynch, M. (2007b). The frailty of adaptive hypotheses for the origins of organismal complexity. In Proceedings of the National Academy of Science, USA, 104: 8597-8604.
63. Mahfoud, S.W. (1995). Niching methods for genetic algorithms. PhD thesis, University of Illinois, Champaign, IL.
64. Maley, C. C. (1999). Four steps toward open-ended evolution. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-1999), pp. 1336-1343.
65. Manderick, B., deWeger, M. K., and Spiessens, P. (1991). The genetic algorithm and the structure of the fitness landscape. In R. K. Belew and L. B. Booker (Eds.), ICGA (International Conference on Genetic Algorithms), pp. 143-150.
66. Martin, A. P. (1999). Increasing genomic complexity by gene duplication and the origin of vertebrates. The American Naturalist, 154(2):111-128.
67. McHale, G., and Husbands, P. (2004). Gasnets and other evolvable neural networks applied to bipedal locomotion. In From Animals to Animats 8, Proceedings of the Eighth International Conference on the Simulation of Adaptive Behavior, 163-172.
68. McShea, D.W. (1991). Complexity and evolution:What everybody knows. Biology and Philosophy, 6(3):303-324.
69. Miconi, T. (2007). Evolution and complexity: The double-edged sword. Artificial Life: Special Issue on the Evolution of Complexity, 14(3): 325-344.
70. Mitchell, M., Forrest, S., and Holland, J. H. (1992). The royal road for genetic algorithms: Fitness landscapes and GA performance. In F. J. Varela and P. Bourgine (Eds.), Proceedings of the First European Conference on Artificial Life, 245-254.
71. Mitchell, T. M. (1997). Machine learning. New York: McGraw-Hill.
72. Mouret, J.-B. (2009). Novelty-based multiobjectivization. In Proceedings of theWorkshop on Exploring New Horizons in Evolutionary Design of Robots, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.
73. Nehaniv, C. L., and Rhodes, J. L. (1999). On the manner inwhich biological complexitymay grow. In Mathematical and Computational Biology. Lectures on Mathematics in the Life Sciences, vol. 26 (pp. 93-102). Washington, DC: American Mathematical Society.
74. Ok, S., and Kim, D. (2005). Evolution of the CPG with sensory feedback for bipedal locomotion. In Advances in Natural Computation (pp. 714-726). Berlin: Springer.
75. Oudeyer, P., Kaplan, F., and Hafner, V. (2007). Intrinsic motivation systems for autonomous mental development. IEEE Transactions on Evolutionary Computation, 11(2):265-286.
76. Oudeyer, P., Kaplan, F., Hafner, V., and Whyte, A. (2005). The playground experiment: Taskindependent development of a curious robot. In Proceedings of the AAAI Spring Symposium on Developmental Robotics, pp. 42-47.
77. Paul, C. (2003). Bilateral decoupling in the neural control of biped locomotion. In Proceedings of the 2nd International Symposium on Adaptive Motion of Animals and Machines.
78. Pelikan, M., Pelikan, M., Goldberg, D. E., and Goldberg, D. E. (2001). Escaping hierarchical traps with competent genetic algorithms. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), pp. 511-518.
79. Pollack, J., Blair, A. D., and Land, M. (1996). Coevolution of a backgammon player. In Proceedings of Artificial Life V, pp. 92-98.
80. Ray, T. (1992). Evolution, ecology and optimization of digital organisms. Technical Report Working paper 92-08-042, Santa Fe Institute.
81. Reil, T., and Husbands, P. (2002). Evolution of central pattern generators for bipedal walking in a real-time physics environment. IEEE Transactions on Evolutionary Computation, 6(2):159-168.
82. Rice, H.G. (1953). Classes of recursively enumerable sets and their decision problems. Transactions of the American Mathematical Society, 74(2):358-366.
83. Risi, S., Vanderbleek, S. D., Hughes, C.E., and Stanley, K. O. (2009). Hownovelty search escapes the deceptive trap of learning to learn. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2009), 153-160.
84. Rothlauf, F., and Goldberg, D. E. (2002). Representations for genetic and evolutionary algorithms. Berlin: Physica-Verlag.
85. Schmidhuber, J. (2003). Exploring the predictable. In A. Ghosh and S. Tsutsui (Eds.), Advances in Evolutionary Computing: Theory and Applications (pp. 579-612). Berlin: Springer.
86. Schmidhuber, J. (2006). Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts. Connection Science, 18(2):173-187.
87. Sigmund, K. (1993). Games of life: Explorations in ecology, evolution and behaviour. NewYork:Oxford University Press.
88. Sims, K. (1994). Evolving 3D morphology and behavior by competition. In R. A. Brooks and P. Maes (Eds.), Proceedings of the Fourth International Workshop on the Synthesis and Simulation of Living Systems (Artificial Life IV), pp. 28-39.
89. Soltoggio, A., and Jones, B. (2009). Novelty of behaviour as a basis for the neuro-evolution of operant reward learning. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2009), 169-176.
90. Standish, R. (2003). Open-ended artificial evolution. International Journal of Computational Intelligence and Applications, 3:167.
91. Stanley, K. O., Bryant, B. D., andMiikkulainen, R. (2005). Real-time neuroevolution in the NERO video game. IEEE Transactions on Evolutionary Computation Special Issue on Evolutionary Computation and Games, 9(6):653-668.
92. Stanley, K. O., and Miikkulainen, R. (2002). Evolving neural networks through augmenting topologies. Evolutionary Computation, 10:99-127.
93. Stanley, K. O., and Miikkulainen, R. (2003). A taxonomy for artificial embryogeny. Artificial Life, 9(2):93-130.
94. Stanley, K. O., and Miikkulainen, R. (2004). Competitive coevolution through evolutionary complexification. Journal of Artificial Intelligence Research, 21:63-100.
95. Stewart, T. C. (2001). Extrema selection:Accelerated evolution on neutral networks. In Proceedings of the 2001 IEEE International Conference on Evolutionary Computation, 25-29.
96. Taylor, T., and Hallam, J. (1997). Studying evolution with self-replicating computer programs. In Fourth European Conference on Artificial Life, pp. 550-559.
97. Van de Panne, M., and Lamouret, A. (1995). Guided optimization for balanced locomotion. In D. Terzopoulos and D. Thalmann (Eds.), Sixth Eurographics Workshop on Animation and Simulation, pp. 165-177.
98. Veldhuizen, D.A.V., and Lamont, G. B. (2000). Multiobjective evolutionary algorithms: Analyzing the state-of-the-art. Evolutionary Computation, 8(2):125-147.
99. Watson, J. D., Hopkins, N. H., Roberts, J. W., Steitz, J. A., and Weiner, A. M. (1987). Molecular biology of the gene, 4th ed. Menlo Park, CA: Benjamin Cummings.
100. Watson, R., and Pollack, J. (2001). Coevolutionary dynamics in a minimal substrate. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2001), 702-709.
101. Weinberger, E. (1990). Correlated and uncorrelated fitness landscapes and how to tell the difference. Biological Cybernetics, 63(5):325-336.
102. Whiteson, S., and Stone, P. (2006). Evolutionary function approximation for reinforcement learning. Journal of Machine Learning Research, 7:877-917.
103. Whitley, L. D. (1991). Fundamental principles of deception in genetic search. In Foundations of genetic algorithms (pp. 221-241). San Mateo, CA: Morgan Kaufmann.
104. Yaeger, L. (1994). Computational genetics, physiology, metabolism, neural systems, learning, vision and behavior or polyworld: Life in a new context. In C. G. Langton (Ed.), Proceedings of Artificial Life III, Volume XVII, pp. 263-298.
105. Yao, X. (1993). An empirical study of genetic operators in genetic algorithms. Microprocessing and Microprogramming, 38(1-5):707-714.
106. Zaera, N., Cliff, D., and Bruten, J. (1996). (Not) evolving collective behaviours in synthetic fish. In From Animals to Animats 4: Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior, 635-644.