An Evolutionary Algorithm that Constructs Recurrent Neural Networks

IEEE Transactions on Neural Networks

Volumn 5, Issue 1, 1994, Pages 54-65

  Angeline, Peter J ^a   Saunders, Gregory M ^a   Pollack, Jordan B ^a  

^a OHIO STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; COMPUTATIONAL COMPLEXITY; COMPUTER ARCHITECTURE; COMPUTER PROGRAMMING; CONSTRAINT THEORY; DATA ACQUISITION; ELECTRIC NETWORK TOPOLOGY; ONLINE SEARCHING;

EVOLUTIONARY COMPUTATION; EVOLUTIONARY PROGRAMMING; GENETIC ALGORITHM; NETWORK ACQUISITION; RECURRENT NEURAL NETWORK; SEARCH METHOD; SOFTWARE PACKAGE GNARL;

NEURAL NETWORKS;

EID: 0028202641     PISSN: 10459227     EISSN: 19410093     Source Type: Journal    
DOI: 10.1109/72.265960     Document Type: Article

References (48)

1. A. G. Barto, “Connectionist learning for control,” in Neural Networks for Control, W. T. Miller III, R. S. Sutton, and P. J. Werbos, Eds. Cambridge, MA: MIT Press, 1990, pp. 5–58.
2. T. Ash, “Dynamic node creation in backpropagation networks,” Connection Science, vol. 1, no. 4, pp. 365–375, 1989.
3. M. Frean, “The upstart algorithm: A method for constructing and training feed-forward neural networks,” Technical Report Preprint 89/469, Edinburgh Physics Dept, 1990.
4. S. J. Hanson, “Meiosis networks,” in Advances in Neural Information Processing Systems 2, D. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1990, pp. 533–541.
5. S. E. Fahlman and C. Lebiere, “The cascade-correlation architecture,” in Advances in Neural Information Processing Systems 2, D. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1990, 524–532.
6. S. Fahlman, “The recurrent cascade-correlation architecture,” In Advances in Neural Information Processing System 3, R. Lippmann, J. Moody, and D. Touretzky, Eds. San Mateo, CA: Morgan Kaufmann, 1991, pp. 190–196.
7. D. Chen, C. Giles, G. Sun, H. Chen, Y. Less, and M. Goudreau, “Constructive learning of recurrent neural networks” IEEE International Conference on Neural Networks, vol. 3, 1993, pp. 1196–1201.
8. M. R. Azimi-Sadjadi, S. Sheedvash and F. O. Trujillo, “Recursive dynamic node creation in multilayer neural networks,” IEEE Transactions on Neural Networks, vol. 4, no. 2, pp. 242–256, 1993.
9. M. Mozer and P. Smolensky, “Skeletonization: A technique for trimming the fat from a network via relevance assessment,” in Advances in Neural Information Processing Systems 1, D. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1989, pp. 107–115.
10. Y. L. Cun, J. Denker, and S. Solla, “Optimal brain damage,” in Advances in Neural Information Processing Systems 2, D. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1990.
11. B. Hassibi and D. G. Stork, “Second order derivatives for network pruning: Optimal brain surgeon,” in Advances in Neural Information Processing Systems 5, S. J. Hanson, J. D. Cowan, and C. L. Giles, Eds. San Mateo, CA: Morgan Kaufmann, 1993, pp. 164–171.
12. C. W. Omlin and C. L. Giles, “Pruning recurrent neural networks for improved generalization performance,” Tech. Report No. 93–6, Computer Science Department, Rensselaer Polytechnic Institute, April 1993.
13. L. J. Fogel, A. J. Owens, and M. J. Walsh, Artificial Intelligence through Simulated Evolution, New York: John Wiley & Sons, 1966.
14. D. B. Fogel, “Evolving artificial intelligence,” Ph.D. thesis, University of California, San Diego, 1992.
15. J. H. Holland, Adaptation in Natural and Artificial Systems, Ann Arbor, MI: The University of Michigan Press, 1975.
16. D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Reading, MA: Addison-Wesley Publishing Company, Inc., 1989.
17. D. B. Fogel, “An introduction to simulated evolutionary optimization,” IEEE Transactions on Neural Networks: Special Issue on Evolutionary Programming, D. B. Fogel and L. J. Fogel, Eds. In press.
18. A. P. Wieland, “Evolving neural network controllers for unstable systems,” in IEEE International Joint Conference on Neural Networks, Seattle, WA: IEEE Press, 1990, pp. II-667-II-673.
19. D. Montana and L. Davis, “Training feedforward neural networks using genetic algorithms,” in Proceedings of the Eleventh International Joint Conference on Artificial Intelligence, San Mateo, CA: Morgan Kaufmann, 1989, pp. 762–767.
20. D. Whitley, T. Starkweather, and C. Bogart, “Genetic algorithms and neural networks: Optimizing connections and connectivity,” Parallel Computing, vol. 14, pp. 347–361, 1990.
21. R. D. Beer and J. C. Gallagher, “Evolving dynamical neural networks for adaptive behavior,” Adaptive Behavior, vol. 1, no. 1, pp. 91–122, 1992.
22. G. F. Miller, P. M. Todd, and S. U. Hegde, “Designing neural networks using genetic algorithms,” in Proceedings of the Third International Conference on Genetic Algorithms, J. D. Schaffer, Ed. San Mateo, CA: Morgan Kaufmann, 1989, pp. 379–384.
23. R. K. Belew, J. McInemey, and N. N. Schraudolf, “Evolving networks: Using the genetic algorithm with connectionist learning,” in Artificial Life II: Proceedings of the Workshop on Artificial Life, C. G. Langton, C. Taylor, J. D. Farmer, and S. Rasmussen, Eds. Reading, MA: Addison-Wesley, 1992, pp. 511–547.
24. J. Torreele, “Temporal processing with recurrent networks: An evolutionary approach,” in Fourth International Conference on Genetic Algorithms, R. K. Belew and L. B. Booker, Eds. San Mateo, CA: Morgan Kaufmann, 1991, 555–561.
25. M. A. Potter, “A genetic cascade-correlation learning algorithm,” in Proceedings of COGANN-92 International Workshop on Combinations of Genetic Algorithms and Neural Networks, L. D. Whitley and J. D. Schaffer, Eds. Los Alamitos, CA: IEEE Computer Society Press, 1992.
26. N. Karunanithi, R. Das, and D. Whitley, “Genetic cascade learning for neural networks,” in Proceedings of COGANN-92 International Workshop on Combinations of Genetic Algorithms and Neural Networks, L. D. Whitley and J. D. Schaffer, Eds. Los Alamitos, CA: IEEE Computer Society Press, 1992.
27. D. E. Goldberg, “Genetic algorithms and Walsh functions: Part 2, Deception and its analysis,” Complex Systems vol. 3, pp. 153–171, 1989.
28. D. E. Goldberg, “Genetic algorithms and Walsh functions: Part 1, A gentle introduction,” Complex Systems, vol. 3, pp. 129–152, 1989.
29. J. D. Schaffer, L. D. Whitley, and L. J. Eshelman, “Combinations of genetic algorithms and neural networks: A survey of the state of the art,” in Proceedings of COGANN-92 International Workshop on Combinations of Genetic Algorithms and Neural Networks, L. D. Whitley and J. D. Schaffer, Eds. Los Alamitos, CA: IEEE Computer Society Press, 1992.
30. G. E. Hinton, J. L. McClelland, and D. E. Rumelhart, “Distributed representations,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 1: Foundations, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge, MA: MIT Press, 1986, pp. 77–109.
31. T. J. Sejnowski and C. R. Rosenberg, “Parallel networks that learn to pronounce english text,” Complex Systems, vol. 1, pp. 145–168, 1987.
32. J. Koza and J. Rice, “Genetic generation of both the weights and architecture for a neural network,” in IEEE International Joint Conference on Neural Networks. Seattle, WA: IEEE Press, 1991, pp. II-397-II-404.
33. R. Collins and D. Jefferson, “An artificial neural network representation for artificial organisms,” in Parallel Problem Solving from Nature, H. P. Schwefel and R. Manner, Eds. Heidelberg: Springer-Verlag, 1991.
34. D. B. Fogel, “A brief history of simulated evolution,” in Proceedings of the First Annual Conference on Evolutionary Programming, D. B. Fogel and W. Atmar, Eds. La Jolla, CA: Evolutionary Programming Society, 1992.
35. D. B. Fogel, L. J. Fogel, and V. W. Porto, “Evolving neural networks,” Biological Cybernetics, vol. 63, pp. 487–493, 1990.
36. J. R. McDonnell and D. Waagen, “Determining neural network connectivity using evolutionary programming,” in Twenty-fifth Asilomar Conferences on Signals, Systems, and Computers, Monterey, CA, 1992.
37. D. B. Fogel, “Using evolutionary programming to create neural networks that are capable of playing Tic-Tac-Toe,” in International Conference on Neural Networks, San Francisco, CA: IEEE Press, 1993, pp. 875–880.
38. S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, “Optimization by simulated annealing,” Science, vol. 220, pp. 671–680, 1983.
39. R. J. Williams, Adaptive State Representation and Estimation Using Recurrent Connectionist Networks. Cambridge, MA: MIT Press, 1990, chap. 4, pp. 97–114.
40. J. B. Pollack, “The induction of dynamical recognizers,” Machine Learning, vol. 7, pp. 227–252, 1991.
41. M. Tomita, “Dynamic construction of finite automata from examples using hill-climbing,” in Proceedings of the Fourth Annual Conference of the Cognitive Science Society, Ann Arbor, MI, 1982, pp. 105–108.
42. R. L. Watrous and G. M. Kuhn, “Induction of finite-state automata using second-order recurrent networks,” in Advances in Neural Information Processing 4, J. E. Moody, S. J. Hanson, and R. P. Lippmann, Eds. San Mateo, CA: Morgan Kaufmann, 1992, pp. 309–316.
43. C. L. Giles, G. Z. Sun, H. H. Chen, Y. C. Lee, and D. Chen, “Higher order recurrent networks & grammatical inference,” in Advances in Neural Information Processing Systems 2, D. S. Touretsky, Ed. San Mateo, CA: Morgan Kaufmann, 1990, pp. 380–387.
44. C. L. Giles, C. B. Miller, D. Chen, G. Z. Sun, H. H. Chen, and Y. C. Lee, “Extracting and learning an unknown grammar with recurrent neural networks,” in Advances in Neural Information Processing 4, J. E. Moody, S. J. Hanson, and R. P. Lippmann, Eds. San Mateo, CA: Morgan Kaufmann, 1992, pp. 317–324.
45. Z. Zeng, R. M. Goodman, and P. Smyth, “Learning finite state machines with self-clustering recurrent networks,” Neural Computation, in press.
46. D. Jefferson, R. Collins, C. Cooper, M. Dyer, M. Flowers, R. Korf, C. Taylor, and A. Wang, “Evolution as a theme in artificial life: The genesys/tracker system,” in Artificial Life II: Proceedings of the Workshop on Artificial Life, C. G. Langton, C. Taylor, J. D. Farmer, and S. Rasmussen, Eds. Reading, MA: Addison-Wesley, 1992, 549–577.
47. J. Koza, “Genetic evolution and co-evolution of computer programs,” in Artificial Life II: Proceedings of the Workshop on Artificial Life, C. G. Langton, C. Taylor, J. D. Farmer, and S. Rasmussen, Eds. Reaading, MA: Addison-Wesley, 1992, pp. 603–629.
48. P. J. Angeline and J. B. Pollack, “Competitive environments evolve better solutions for complex tasks,” in Genetic Algorithms: Proceedings of the Fifth International Conference (GA93), S. Forrest, Ed. San Mateo, CA: Morgan Kaufmann, 1993, pp. 264–270.