Neuroevolution: From architectures to learning

Evolutionary Intelligence

Volumn 1, Issue 1, 2008, Pages 47-62

  Floreano, Dario ^a   Dürr, Peter ^a   Mattiussi, Claudio ^a  

^a EPFL   (Switzerland)

Author keywords

Evolution; Learning; Neural networks

Indexed keywords

ARTIFICIAL NEURAL NETWORKS; EVOLUTIONARY BIOLOGY; EVOLUTIONARY SEARCH; LEARNING ARCHITECTURES; LEARNING NEURAL NETWORKS; NEUROEVOLUTION; NEURON MODEL; PATTERN CLASSIFICATION; REAL-WORLD PROBLEM; ROBOT CONTROLS;

BIOLOGY; EVOLUTIONARY ALGORITHMS; LEARNING ALGORITHMS;

NEURAL NETWORKS;

EID: 55349116293     PISSN: 18645909     EISSN: 18645917     Source Type: Journal    
DOI: 10.1007/s12065-007-0002-4     Document Type: Review

References (91)

1. Ackley DH, Littman ML (1992) Interactions between learning and evolution. In: Langton C, Farmer J, Rasmussen S, Taylor C (eds) Artificial Life II: Proceedings volume of Santa Fe conference, vol XI. Addison Wesley, Redwood City, pp 487-510.
2. Bailey CH, Giustetto M, Huang Y.-Y, Hawkins RD, Kandel ER (2000) Is heterosynaptic modulation essential for stabilizing Hebbian plasticity and memory? Nat Rev Neurosci 1(1): 11-20.
3. Baldwin JM (1896) A new factor in evolution. Am Nat 30: 441-451.
4. Ackley DH, Littman ML (1992) Interactions between learning and evolution. In: Langton C, Farmer J, Rasmussen S, Taylor C (eds) Artificial Life II: Proceedings volume of Santa Fe conference, vol XI. Addison Wesley, Redwood City, pp 487-510.
5. Barto AG (1995) Adaptive critic and the basal ganglia. In: Houk JC, Davis JL, Beiser DG (eds) Models of information processing in the basal ganglia. MIT Press, Cambridge, pp 215-232.
6. Baxter J (1992) The evolution of learning algorithms for artificial neural networks. In: Green D, Bossomaier T (eds) Complex Systems. IOS Press.
7. Beer RD, Gallagher JC (1992) Evolving dynamical neural networks for adaptive behavior. Adapt Behav 1: 91-122.
8. Belew RK, McInerney J, Schraudolph NN (1992) Evolving networks: using the genetic algorithm with connectionistic learning. In: Langton CG, Taylor C, Farmer JD, Rasmussen S (eds) Proceedings of the 2nd Conference on Artificial Life. Addison-Wesley, Reading, pp 511-548.
9. Blynel J, Floreano D (2003) Exploring the T-maze: evolving learning-like robot behaviors using CTRNNs. In: Raidl Ge AE (ed) 2nd European workshop on evolutionary robotics (EvoRob'2003).
10. Bongard J (2002) Evolving modular genetic regulatory networks. In: Proceedings of the 2002 congress on evolutionary computation 2002, CEC '02, vol 2, pp 1872-1877.
11. Chalmers DJ (1990) The evolution of learning: an experiment in genetic connectionism. In: Touretzky DS, Elman JL, Sejnowski T, Hinton GE (eds) Proceedings of the 1990 connectionist models summer school. Morgan Kaufmann, San Mateo, pp 81-90.
12. Chandra A, Yao X (2006) Ensemble learning using multi-objective evolutionary algorithms. J Math Model Algorithms 5(4): 417-445.
13. Chellapilla K, Fogel D (2001) Evolving an expert checkers playing program without using humanexpertise. IEEE Trans Evol Comput 5(4): 422-428.
14. Dasdan A, Oflazer K (1993) Genetic synthesis of unsupervised learning algorithms. In: Proceedings of the 2nd Turkish symposium on artificial intelligence and ANNs. Department of Computer Engineering and Information Science, Bilkent University, Ankara.
15. DiPaolo E (2003) Evolving spike-timing-dependent plasticity for single-trial learning in robots. Phil Trans R Soc Lond A 361: 2299-2319.
16. Dürr P, Mattiussi C, Floreano D (2006) Neuroevolution with Analog Genetic Encoding. In: Parallel problem solving from nature-PPSN iX, vol 9. Springer, Berlin, pp 671-680.
17. Federici D (2005) Evolving developing spiking neural networks. In: Proceedings of CEC 2005 IEEE congress on evolutionary computation.
18. Fellous J-M, Linster C (1998) Computational models of neuromodulation. Neural Comput 10(4): 771-805.
19. Floreano D, Mattiussi C (2001) Evolution of spiking neural controllers for autonomous vision-based robots. In: Gomi T (ed) Evolutionary robotics. From intelligent robotics to artificial life. Springer, Tokyo.
20. Floreano D, Mondada F (1996) Evolution of plastic neurocontrollers for situated agents. In: Maes P, Matarić M, Meyer J, Pollack J, Roitblat H, Wilson S (eds) From animals to animats IV: proceedings of the 4th international conference on simulation of adaptive behavior. MIT Press-Bradford Books, Cambridge, pp 402-410.
21. Floreano D, Urzelai J (2000) Evolutionary robots with online self-organization and behavioral fitness. Neural Netw 13: 431-443.
22. Floreano D, Urzelai J (2001) Evolution of plastic control networks. Autonom Robots 11(3): 311-317.
23. Fontanari JF, Meir R (1991) Evolving a learning algorithm for the binary perceptron. Network 2: 353-359.
24. Funahashi K, Nakamura Y (1993) Approximation of dynamical systems by continuous time recurrent neural networks. Neural Netw 6(6): 801-806.
25. Geard NL, Wiles J (2003) Structure and dynamics of a gene network model incorporating small RNAs. In: Proceedings of 2003 congress on evolutionary computation, pp 199-206.
26. Gerstner W (1999) Spiking neurons. In: Maass W, Bishop CM (eds) Pulsed neural networks. MIT Press-Bradford Books, Cambridge.
27. Gomez F, Miikkulainen R (1997) Incremental evolution of complex general behavior. Adapt Behav 5(3-4): 317-342.
28. Gruau F (1995) Automatic definition of modular neural networks. Adapt Behav 3(2): 151-183.
29. Gruau, F, Whitley, D, and Pyeatt, L (1996) A comparison between cellular encoding and direct encoding for genetic neural networks. In: Koza JR, Goldberg DE, Fogel DB, Riolo RL (eds) Genetic programming 1996: proceedings of the first annual conference. MIT Press, Stanford University, pp 81-89.
30. Hansen N, Ostermeier A (2001) Completely derandomized self-adaptation in evolution strategies. Evol Comput 9(2): 159-195.
31. Haykin, S (1999) Neural networks. a comprehensive foundation, 2nd edn. Prentice Hall, Upper Saddle River.
32. Hebb DO (1949) The organisation of behavior. Wiley, New York.
33. Hinton GE, Nowlan SJ (1987) How learning can guide evolution. Complex Syst 1: 495-502.
34. Hodgkin AL, Huxley AF (1952) A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol (Lond) 108: 500-544.
35. Hurst J, Bull L (2006) A neural learning classifier system with self-adaptive constructivism for mobile robot control. Artif Life 12 (3): 353-380.
36. Husbands P, Harvey I, Cliff D, Miller G (1994) The use of genetic algorithms for the development of sensorimotor control systems. In: Gaussier P, Nicoud J-D (eds) From perceptin to action. IEEE Press, Los Alamitos.
37. Husbands P, Smith T, Jakobi N, O'Shea M (1998) Better living through chemistry: evolving gasnets for robot control. Connect Sci 10: 185-210.
38. Igel, C (2003) Neuroevolution for reinforcement learning using evolution strategies. In: Sarker R, et al (eds) Congress on evolutionary computation, vol 4. IEEE Press, New York, pp 2588-2595.
39. Katz PS (1999) What are we talking about? Modes of neuronal communication. In: Katz P (eds) Beyond neurotransmission: neuromodulation and its importance for information processing, chap 1. Oxford University Press, Oxford, pp 1-28.
40. Kitano H (1990) Designing neural networks by genetic algorithms using graph generation system. Complex Syst J 4: 461-476.
41. Korkin M, Nawa NE, de Garis H (1998) A 'spike interval information coding' representation for ATR's CAM-brain machine (CBM) In: Proceedings of the 2nd international conference on evolvable systems: from biology to hardware (ICES'98). Springer, Heidelberg.
42. Koza JR (1994) Genetic programming II: automatic discovery of reusable programs. MIT Press, Cambridge.
43. Magg S, Philippides A (2006) Gasnets and CTRNNs: a comparison in terms of evolvability. In: From animals to animats 9: proceedings of the 9th international conference on simulation of adaptive behavior. Springer, Heidelberg, pp 461-472.
44. Mattiussi C, Floreano D (2004) Evolution of analog networks using local string alignment on highly reorganizable genomes. In: Zebulum RS et al (eds) NASA/DoD conference on evolvable hardware (EH'2004), pp 30-37.
45. Mattiussi C, Dürr P, Floreano D (2007a) Center of mass encoding: a self-adaptive representation with adjustable redundancy for real-valued parameters. In: GECCO 2007. ACM Press, New York, pp 1304-1311.
46. Mattiussi C, Marbach D, Dürr P, Floreano D (2007b) The age of analog networks. AI Magazine (in press).
47. Mayley G (1996) Landscapes, learning costs and genetic assimilation. Evol Comput 4(3): 213-234.
48. McHale G, Husbands P (2004) Gasnets and other evolvable neural networks applied to bipedal locomotion. In: Schaal S (ed) Proceedings from animals to animats 8: proceedings of the 8th international conference on simulation of adaptive behaviour (SAB'2004). MIT Press, Cambridge, pp 163-172.
49. Mizutani E, Dreyfus SE (1998) Totally model-free reinforcement learning by actor-critic elman networks in non-markovian domains. In: Proceedings of the IEEE world congress on computational intelligence. IEEE Press, New York.
50. Montague P, Dayan P, Sejnowski T (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16(5): 1936-1947.
51. Montana D, Davis L (1989) Training feed forward neural networks using genetic algorithms. In: Proceedings of the 11th international joint conference on artificial intelligence. Morgan Kaufmann, San Mateo, pp 529-538.
52. Moriarty DE, Miikkulainen R (1996) Efficient reinforcement learning through symbiotic evolution. Machine Learn 22: 11-32.
53. Moscato P (1989) On evolution, search, optimization, genetic algorithms and martial arts: towards memetic algorithms. In: Technical report C3P 826, Pasadena.
54. Niv Y, Joel D, Meilijson I, Ruppin E (2002) Evolution of reinforcement learning in uncertain environments: A simple explanation for complex foraging behaviors. Adapt Behav 10(1): 5-24.
55. Nolfi S, Floreano D (1999) Learning and evolution. Auton Robots 7(1): 89-113.
56. Nolfi S, Parisi D (1996) Learning to adapt to changing environments in evolving neural networks. Adapt Behav 5(1): 75-98.
57. Nolfi S, Miglino O, Parisi D (1994) Phenotypic plasticity in evolving neural networks. In: Gaussier P, Nicoud J-D (eds) From perception to action. IEEE Press, Los Alamitos.
58. Pfeifer R, Scheier C (1999) Understanding Intelligence. MIT Press, Cambridge.
59. Purves D (1994) Neural activity in the growth of the brain. Cambridge University Press, Cambridge.
60. Quartz S, Sejnowski TJ (1997) The neural basis of cognitive development: a constructivist manifesto. Behav Brain Sci 4: 537-555.
61. Radcliffe NJ (1991) Form an analysis and random respectful recombination. In: Belew RK, Booker LB (eds) Proceedings of the 4th international conference on genetic algorithms. Morgan Kaufmann, San Mateo.
62. Rechenberg I (1973) Evolutionsstrategie-Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. Fommann-Holzboog, Stuttgart.
63. Reil T (1999) Dynamics of gene expression in an artificial genome-implications for biological and artificial ontogeny. In: Proceedings of the 5th European conference on artificial life, pp 457-466.
64. Reil T (2003) On growth, form and computers. In: Artificial genomes as models of gene regulation. Academic Press, London, pp 256-277.
65. Reisinger J, Miikkulainen R (2007) Acquiring evolvability through adaptive representations. In: Proceedings of genetic and evolutionary computation conference (GECCO 2007).
66. Reisinger J, Bahçeci E, Karpov I, Miikkulainen R (2007) Coevolving strategies for general game playing. In: Proceedings of the IEEE symposium on computational intelligence and games (CIG-2007).
67. Rieke F, Warland D, van Steveninck R, Bialek W (1997) Spikes. Exploring the neural code. MIT Press, Cambridge.
68. Rumelhart DE, Hinton GE, Williams RJ (1986a) Learning representations by back-propagation of errors. Nature 323: 533-536.
69. Rumelhart DE, McClelland J, the PDP Research Group (1986b) Parallel distributed processing: explorations in the microstructure of cognition. Foundations, vol 1. MIT Press-Bradford Books, Cambridge.
70. Saggie K, Keinan A, Ruppin E (2004) Spikes that count: rethinking spikiness in neurally embedded systems. Neurocomputing 58-60: 303-311.
71. Sasaki T, Tokoro M (1997) Adaptation toward changing environments: Why Darwinian in nature?. In: Husbands P, Harvey I (eds) Proceedings of the 4th European conference on artificial life. MIT Press, Cambridge.
72. Schaffer JD, Whitley D, Eshelman LJ (1992) Combinations of genetic algorithms and neural networks: a survey of the state of the art. In: Whitley D, Schaffer JD (eds) Proceedings of an international workshop on the combinations of genetic algorithms and neural networks (COGANN-92). IEEE Press, New York.
73. Schraudolph NN, Belew RK (1992) Dynamic parameter encoding for genetic algorithms. Machine Learn 9: 9-21.
74. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275(5306): 1593-1599.
75. Shapiro J (2005) A 21st century view of evolution: genome system architecture, repetitive DNA, and natural genetic engineering. Gene 345(1): 91-100.
76. Siddiqi A, Lucas S (1998) A comparison of matrix rewriting versus direct encoding for evolving neural networks. In: Proceedings of the 1998 IEEE international conference on evolutionary computation. Piscataway, NJ, pp 392-397.
77. Singer W, Gray CM (1995) Visual feature integration and the temporal correlation hypothesis. Annu Rev Neurosci 18: 555-586.
78. Soltoggio A, Duerr P, Mattiussi C, Floreano D (2007) Evolving neuromodulatory topologies for reinforcement learning-like problems. In: Angeline P, Michaelewicz M, Schonauer G, Yao X, Zalzala Z (eds) Proceedings of the 2007 congress on evolutionary computation. IEEE Press, New York.
79. Stanley K, Miikkulainen R (2002) Evolving neural networks through augmenting topologies. Evol Comput 10(2): 99-127.
80. Stanley KO, Miikkulainen R (2004) Competitive coevolution through evolutionary complexification. J Artif Intell Res 21: 63-100.
81. Stanley K, Kohl N, Sherony R, Miikkulainen R (2005a) Neuroevolution of an automobile crash warning system. In: Proceedings of genetic and evolutionary computation conference (GECCO 2005).
82. Stanley KO, Cornelius R, Miikkulainen R, D'Silva T, Gold A (2005b) Real-time learning in the nero video game. In: Proceedings of the artificial intelligence and interactive digital entertainment conference (AIIDE 2005) demo papers.
83. Sutton RS (1988) Learning to predict by the method of temporal difference. Machine Learn 3: 9-44.
84. Sutton RS, Barto AG (1998) Reinforcement learning. an introduction. MIT Press, Cambridge.
85. Trianni V, Ampatzis C, Christensen A, Tuci E, Dorigo M, Nolfi S (2007) From solitary to collective behaviours: decision making and cooperation. In: Advances in artificial life, proceedings of ECAL 2007. Lecture Notes in Artificial Intelligence, vol LNAI 4648. Springer, Berlin, pp 575-584.
86. Tuci E, Quinn M, Harvey I (2002) An evolutionary ecological approach to the study of learning behavior using a robot-based model. Adapt Behav 10(3-4): 201-221.
87. Urzelai J, Floreano D (2001) Evolution of adaptive synapses: robots with fast adaptive behavior in new environments. Evol Comput 9: 495-524.
88. Whitley D, Starkweather T, Bogart C (1990) Genetic algorithms and neural networks: optimizing connections and connectivity. Parallel Comput 14: 347-361.
89. Widrow B, Hoff ME (1960) Adaptive switching circuits. In: Proceedings of the 1960 IRE WESCON convention, vol IV, New York. IRE. Reprinted in Anderson and Rosenfeld, 1988, pp 96-104.
90. Yamauchi BM, Beer RD (1994) Sequential behavior and learning in evolved dynamical neural networks. Adapt Behav 2(3): 219-246.
91. Yao X (1999) Evolving artificial neural networks. Proc IEEE 87(9): 1423-1447.