Evolutionary design of neural network architectures using a descriptive encoding language

IEEE Transactions on Evolutionary Computation

Volumn 10, Issue 6, 2006, Pages 676-688

  Jung, Jae Yoon ^a   Reggia, James A ^a  

^a Institute of Advanced Computer Science   (United States)

Author keywords

Descriptive encoding; Evolutionary neural networks; Modular neural network

Indexed keywords

COMPUTATIONAL COMPLEXITY; COMPUTATIONAL METHODS; ENCODING (SYMBOLS); EVOLUTIONARY ALGORITHMS; PROBLEM SOLVING;

DESCRIPTIVE ENCODING; EVOLUTIONARY NEURAL NETWORKS; MODULAR NEURAL NETWORKS;

NEURAL NETWORKS;

EID: 33947361151     PISSN: 1089778X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TEVC.2006.872346     Document Type: Article

References (49)

1. K. Mehrotra, C. Mohan, and S. Ranka, Elements of Artificial Neural Neworks. Cambridge, MA: MIT Press, 1997, pp. 116-135.
2. A. Blum and R. Rivest, "Training a 3-node neural network is NP-complete," Neural Netw., vol. 5, no. 1, pp. 117-128, 1992.
3. G. Miller, P. Todd, and S. Hegde, "Designing neural networks using genetic algorithms," in Proc. 3rd Int. Conf. Genetic Algorithms, 1989, pp. 379-384.
4. W. Banzhf, P. Nordin, R. Keller, and F. Francone, Genetic Programming an Introduction. San Mateo, CA: Morgan Kaufmann, 1997.
5. P. Bentley and D. Corne, Eds., Creative Evolutionary Systems. San Diego, CA: Academic Press, 2002.
6. J. Koza, F. Bennett, D. Andre, and M. Keane, Genetic Programming III: Darwinian Invention and Problem Solving. San Mateo, CA: Morgan Kaufmann, 1999.
7. K. Balakrishnan and V. Honavar, "Evolving neuro-controllers and sensors for artificial agents," in Advances in the Evolutionary Synthesis of Intelligent Agents, M. Patel, V. Honavar, and K. Balakrishnan, Eds. Cambridge, MA: MIT Press, 2001, pp. 109-152.
8. E. Ruppin, "Evolutionary autonomous agents: A neuroscience perspective," Nature Rev. Neurosci., vol. 3, no. 2, pp. 132-141, 2002.
9. N. Saravanan and D. Fogel, "Evolving neural control systems," IEEE Expert, vol. 10, pp. 23-27, 1995.
10. X. Yao, "Evolving artificial neural networks," Proc. IEEE, vol. 87, no. 9, pp. 1423-1447, 1999.
11. H. Kitano, "Neurogenetic learning: An integrated method of designing and training neural networks using genetic algorithms," Physica D, vol. 75, pp. 225-238, 1994.
12. F. Gruau, "Automatic definition of modular neural networks," Adaptive Behavior, vol. 3, pp. 151-183, 1995.
13. F. Gruau, D. Whitley, and L. Pyeatt, "A comparison between cellular encoding and direct encoding for genetic neural networks," in Proc. 1st Annu. Conf. Genetic Programming, 1996, pp. 81-89.
14. S. Luke and L. Spector, "Evolving graphs and networks with edge encoding: Preliminary report," in Proc. Genetic Programming Conf. (Late Breaking Papers), 1996, pp. 117-124.
15. T. Hussain and R. Browse, "Network generating attribute grammar encoding," in Proc. Int. Joint Conf. Neural Netw., 1998, vol. 1, pp. 431-436.
16. V. Mountcastle, The Cerebral Cortex. Cambridge, MA: Harvard Univ. Press, 1998.
17. G. Schlosser and G. Wagner, Eds., Modularity in Development and Evolution. Chicago, IL: Univ. Chicago Press, 2004.
18. G. Wagner, "Adaptation and the modular design of organizms," in Advances in Artificial Life, F. Moran, A. Moreno, J. Merelo, and P. Chacon, Eds. Berlin, Germany: Springer-Verlag, 1995, pp. 317-328.
19. T. Caelli, L. Guan, and W. Wen, "Modularity in neural computing," Proc. IEEE, vol. 87, pp. 1497-1518, 1999.
20. R. Calabretta, S. Nolfi, D. Parisi, and G. Wagner, "Duplication of modules facilitates the evolution of functional specialization," Artif. Life, vol. 6, pp. 69-84, 2000.
21. L. Franco and S. Cannas, "Generalization properties of modular networks," IEEE Trans. Neural Netw., vol. 12, pp. 1306-1313, 2001.
22. S. Cho and K. Shimohara, "Evolutionary learning of modular neural networks with genetic programming," Appl. Intell., vol. 9, pp. 191-200, 1998.
23. B. Happel and J. Murre, "Design and evolution of modular neural network architectures," Neural Netw., vol. 7, pp. 985-1004, 1994.
24. S. Harp, T. Samad, and A. Guha, "Toward the genetic synthesis of neural networks," in Proc. 3rd Int. Conf. Genetic Algorithms, 1989, pp. 379-384.
25. M. O'Neill and C. Ryan, "Grammatical evolution," IEEE Trans. Evol. Comput., vol. 5, no. 4, pp. 349-358, Aug. 2001.
26. M. O'Neill and C. Ryan, Grammatical Evolution: Evolutionary Automatic Programming in an Arbitrary Language (Genetic Programming). Berlin, Germany: Springer-Verlag, 2003.
27. C. Jacob and J. Rehder, "Evolution of neural net architectures by a hierarchical grammar-based genetic system," in Proc. Artif. Neural Netw. Genetic Algorithms, 1993, pp. 72-79.
28. C. Jacob, "Genetic L-system programming," in PPSN III - Parallel Problem Solving from Nature. Berlin, Germany: Sprinaer-Verlas, 1994, vol. 866, pp. 334-343.
29. G. Hornby, "Shortcomings with tree-structured edge encodings for neural networks," in Proc. Genetic and Evol. Comput. Conf., 2004, pp. 495-506.
30. A. Ferdinando, R. Calabretta, and D. Parisi, "Evolving modular architectures for neural networks," in Connectionist Models of Learning. Development and Evolution, R. French and J. Sougne, Eds. Berlin, Germany: Springer-Verlag, 2001, pp. 253-262.
31. C. Coello, "Evolutionary multi-objective optimization: A critical review," in Evolutionary Optimization, R. Sarkar, M. Mohammadian, and X. Yao, Eds. Norwell, MA: Kluwer, 2002, pp. 117-146.
32. E. Kandel, J. Schwartz, and T. Jessel, Principles of Neural Science. New York: Appleton and Lange, 1991.
33. S. Dimond and D. Blizard, Eds., Evolution and Lateralization of the Brain. New York: New York Academy of Sciences, 1977.
34. M. Brown, R. Keynes, and A. Lumsden, The Developing Brain. Oxford, U.K.: Oxford Univ. Press, 2001.
35. H. Killackey, "Evolution of the human brain: A neuroanatomical perspective," in The Cognitive Neurosciences, M. Gazzaniga, Ed. Cambridge, MA: MIT Press, 1996, pp. 1243-1253.
36. J. Tooby and L. Cosmides, "Toward mapping the evolved functional oreanization of mind and brain," in The New Cognitive Neurosciences, MTCazzinga,Ed. Cambridge, MA: MIT Press, 2000, pp. 1167-1178.
37. M. Riedmiller and H. Braun, "A direct adaptive method for faster back-propagation learning: The RPROP algorithm," in Proc. IEEE Int. Conf. Neural Netw., 1993, vol. 1, pp. 586-591.
38. S. Haykin, Neural Nenvorks, A Comprehensive Foundation. Englewood Cliffs, NJ: Prentice-Hall, 1999.
39. Y. Shkuro and J. Reggia, "Cost minimization during simulated evolution of paired neural networks leads to asymmetries and specialization," Cognitive Syst. Res., vol. 4, pp. 365-383, 2003.
40. G. Cybenko, "Approximation by superpositions of a sigmoidal function," Math. Control, Signals, Syst., vol. 2, no. 4, pp. 303-314, 1989.
41. M. Radio, J. Reggia, and R. Berndt, "Learning word pronunciations using a recurrent neural network," in Proc. Int. Joint Conf. Neural Netw., 2001, vol. 1, pp. 11-15.
42. C. Giles, C. Miller, D. Chen, G. Sun, H. Chen, and Y. Lee, "Extracting and learning an unknown grammar with recurrent neural networks," in Advances in Neural Information Systems 4, J. Moody, S. Hanson, and R. P. Lippmann, Eds. San Mateo, CA: Morgan Kaufmann, 1992.
43. J. Elman, "Finding structure in time," Cognitive Science, vol. 14, pp. 179-211, 1990.
44. M. Jordan, "Attractor dynamics and parallelism in a connectionist sequential machine," in Proc. 9th Annu. Conf. Cognitive Sci. Soc., 1986, pp. 531-546.
45. J. Pérez-Ortiz, J. Calera-Rubio, and M. Forcada, "A comparison between recurrent neural architectures for real-time nonlinear prediction of speech signals," in Proc. IEEE Neural Nefworks for Signal Processing Workshop, D. J. Miller, T. Adali, J. Larsen, M. Van Hulle, and S. Douglas, Eds., 2001, pp. 73-81.
46. T. Sejnowski and C. Rosenberg, "Parallel networks that learn to pronounce English text," Complex Syst., vol. 1, pp. 145-168, 1987.
47. E. Zitzler and L. Thiele, "Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach," IEEE Trans. Evol. Comput., vol. 3, no. 4, pp. 257-271, 1999.
48. J.-Y. Jung and J. Reggia, "A descriptive encoding language for evolving modular neural networks," in Proc. Genetic and Evol. Comput. Conf., 2004, pp. 519-530.
49. M. Potter and K. de Jong, "Cooperative coevolution: An architecture for evolving coadapted subcomponents," Evol. Comput., vol. 8, no. 1, pp. 1-29, 2000.