A new evolutionary system for evolving artificial neural networks

IEEE Transactions on Neural Networks

Volumn 8, Issue 3, 1997, Pages 694-713

  Yao, Xin ^a,b   Liu, Yong ^b  

^a IEEE   (Australia)

^b UNIVERSITY OF NEW SOUTH WALES   (Australia)

Author keywords

Evolution; Evolution of behaviors; Evolutionary programming; Generalization; Learning; Neural network design; Parsimony

Indexed keywords

EVOLUTIONARY PROGRAMMING (EP); PARSIMONY;

ALGORITHMS; COMPUTER ARCHITECTURE; LEARNING SYSTEMS; MATHEMATICAL OPERATORS; MATHEMATICAL PROGRAMMING;

NEURAL NETWORKS;

EID: 0031143030     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.572107     Document Type: Article

References (62)

1. L. J. Fogel, A. J. Owens, and M. J. Walsh, Artificial Intelligence Through Simulated Evolution. New York: Wiley, 1966.
2. D. B. Fogel, System Identification Through Simulated Evolution: A Machine Learning Approach to Modeling. Needham Heights, MA: Ginn, 1991.
3. D. B. Fogel, Evolutionary Computation: Toward a New Philosophy of Machine Intelligence. New York: IEEE Press, 1995.
4. Y. Liu and X. Yao, "Evolutionary design of artificial neural networks with different nodes," in Proc. 1996 IEEE Int. Conf. Evolutionary Computation (ICEC'96), Nagoya, Japan, 1996, pp. 670-675.
5. S. E. Fahlman and C. Lebiere, "The cascade-correlation learning architecture," in Advances in Neural Information Processing Systems 2, D. S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1990, pp. 524-532.
6. J.-P. Nadal, "Study of a growth algorithm for a feedforward network," Int. J. Neural Syst., vol. 1, pp. 55-59, 1989.
7. N. Burgess, "A constructive algorithm that converges for real-valued input patterns," Int. J. Neural Syst., vol. 5, no. 1, pp. 59-66, 1994.
8. R. Setiono and L. C. K. Hui, "Use of a quasinewton method in a feedforward neural-network construction algorithm," IEEE Trans. Neural Networks, vol. 6, pp. 273-277, 1995.
9. R. Reed, "Pruning algorithms - A survey," IEEE Trans. Neural Networks, vol. 4, pp. 740-747, 1993.
10. P. J. Angeline, G. M. Sauders, and J. B. Pollack, "An evolutionary algorithm that constructs recurrent neural networks," IEEE Trans. Neural Networks, vol. 5, pp. 54-65, 1994.
11. G. F. Miller, P. M. Todd, and S. U. Hegde, "Designing neural networks using genetic algorithms," in Proc. 3rd Int. Conf. Genetic Algorithms Their Applications, J. D. Schaffer, Ed. San Mateo, CA: Morgan Kaufmann, 1989, pp. 379-384.
12. D. B. Fogel, L. J. Fogel, and V. W. Porto, "Evolving neural networks," Biol. Cybern., vol. 63, pp. 487-493, 1990.
13. J. R. McDonnell and D. Waagen, "Evolving recurrent perceptrons for time-series modeling," IEEE Trans. Neural Networks, vol. 5, pp. 24-38, 1994.
14. _, "Neural-network structure design by evolutionary programming," in Proc. 2nd Annu. Conf. Evolutionary Programming, D. B. Fogel and W. Atmar, Eds. La Jolla, CA: Evolutionary Programming Soc., 1993, pp. 79-89.
15. D. B. Fogel, "Using evolutionary programming to create neural networks that are capable of playing tic-tac-toe," in Proc. 1993 Int. Joint Conf. Neural Networks (IJCNN'93). New York, NY: IEEE Press, 1993, pp. 875-880.
16. X. Yao and Y. Liu, "Toward designing artificial neural networks by evolution," in Proc. Int. Symp. Artificial Life and Robotics (AROB), Beppu, Oita, Japan, Feb. 18-20, 1996, pp. 265-268.
17. J. D. Schaffer, D. Whitley, and L. J. Eshelman, "Combinations of genetic algorithms and neural networks: A survey of the state of the art," in Proc. Int. Wkshp. Combinations Genetic Algorithms Neural Networks (COGANN-92), D. Whitley and J. D. Schaffer, Eds. Los Alamitos, CA: IEEE Computer Soc. Press, 1992, pp. 1-37.
18. X. Yao, "Evolution of connectionist networks," in Preprints Int. Symp. AI, Reasoning and Creativity, T. Dartnall, Ed. Queensland, Australia: Griffith Univ., pp. 49-52, 1991.
19. X. Yao, "A review of evolutionary artificial neural networks," Int. J. Intell. Syst., vol. 8, no. 4, pp. 539-567, 1993.
20. _, "Evolutionary artificial neural networks," Int. J. Neural Syst., vol. 4, no. 3, pp. 203-222, 1993.
21. _, "Evolutionary artificial neural networks," in Encyclopedia of Computer Science and Technology, A. Kent and J. G. Williams, Eds. New York: Marcel Dekker, vol. 33, pp. 137-170, 1995.
22. D. Whitley, T. Starkweather, and C. Bogart, "Genetic algorithms and neural networks: Optimizing connections and connectivity," Parallel Computing, vol. 14, pp. 347-361, 1990.
23. D. Whitley and T. Starkweather, "Optimizing small neural networks using a distributed genetic algorithm," in Proc. Int. Joint Conf. Neural Networks. Hillsdale, NJ: Lawrence Erlbaum, vol. I, 1990, pp. 206-209.
24. X. Yao and Y. Shi, "A preliminary study on designing artificial neural networks using co-evolution," in Proc. IEEE Singapore Int. Conf. Intell. Contr. Instrumentation, Singapore, June 1995, pp. 149-154.
25. D. B. Fogel, "Phenotypes, genotypes, and operators in evolutionary computation," in Proc. 1995 IEEE Int. Conf. Evolutionary Computation (ICEC'95). New York: IEEE Press, 1995, pp. 193-198.
26. V. Maniezzo, "Genetic evolution of the topology and weight distribution of neural networks," IEEE Trans. Neural Networks, vol. 5, pp. 39-53, 1994.
27. R. K. Belew, J. McInerney, and N. N. Schraudolph, "Evolving networks: Using genetic algorithm with connectionist learning," Computer Sci. Eng. Dept., Univ. California-San Diego, Tech. Rep. CS90-174 revised, Feb. 1991.
28. P. J. B. Hancock, "Genetic algorithms and permutation problems: A comparison of recombination operators for neural net structure specification," in Proc. Int. Wkshp. Combinations of Genetic Algorithms and Neural Networks (COGANN-92), D. Whitley and J. D. Schaffer, Eds. Los Alamitos, CA: IEEE Computer Soc. Press, 1992, pp. 108-122.
29. R. Smalz and M. Conrad, "Combining evolution with credit apportionment: A new learning algorithm for neural nets," Neural Networks, vol. 7, no. 2, pp. 341-351, 1994.
30. S. V. Odri, D. P. Petrovacki, and G. A. Krstonosic, "Evolutional development of a multilevel neural network," Neural Networks, vol. 6, no. 4, pp. 583-595, 1993.
31. X. Yao and Y. Liu, "Evolving artificial neural networks for medical applications," in Proc. 1995 Australia-Korea Joint Wkshp. Evolutionary Computa., Kaist, Taejon, Korea, Sept. 1995, pp. 1-16.
32. _, "Evolving artificial neural networks through evolutionary programming," in Evolutionary Programming V: Proc. 5th Annu. Conf. Evolutionary Programming, L. Fogel, P. Angeline, and T. Bäck, Eds. Cambridge, MA: MIT Press, 1996, pp. 257-266.
33. _, "Evolutionary artificial neural networks that learn and generalize well," in 1996 IEEE Int. Conf. Neural Networks, Washington, DC, vol. on Plenary, Panel, and Special Sessions. New York: IEEE Press, June 3-6, 1996, pp. 159-164.
34. Y. Liu and X. Yao, "A population-based learning algorithm which learns both architectures and weights of neural networks," Chinese J. Advanced Software Res., vol. 3, no. 1, pp. 54-65, 1996.
35. P. J. Werbos, The Roots of Backpropagation: From Ordered Derivatives to Neural Networks and Political Forecasting. New York: Wiley, 1994.
36. L. Prechelt, "Proben1 - A set of neural network benchmark problems and benchmarking rules," Fakultät für Informatik, Univ. Karlsruhe, Karlsruhe, Germany, Tech. Rep. 21/94, Sept. 1994.
37. X. Yao, "An empirical study of genetic operators in genetic algorithms," Microprocessing Microprogramming, vol. 38, pp. 707-714, 1993.
38. D. H. Ackley and M. S. Littman, "A case for Lamarckian evolution," in Artificial Life III, SFI Studies in the Sciences of Complexity, C. G. Langton, Ed. Reading, MA: Addison-Wesley, vol. XVIII, 1994, pp. 487-509.
39. D. Whitley, S. Gordon, and K. Mathias, "Lamarkian evolution, the Baldwin effect, and function optimization," in Parallel Problem Solving from Nature (PPSN) III, Y. Davidor, H.-P. Schwefel, and R. Männer, Eds. Berlin: Springer-Verlag, 1994, pp. 6-15.
40. D. Whitley and T. Starkweather, "GENITOR II: A distributed genetic algorithm," J. Experimental Theoretical Artificial Intell., vol. 2, pp. 189-214, 1990.
41. G. Syswerda, "A study of reproduction in generational and steady-state genetic algorithms," in Foundations of Genetic Algorithms, G. J. E. Rawlins, Ed. San Mateo, CA: Morgan Kaufmann, 1991, pp. 94-101.
42. G. B. Fogel and D. B. Fogel, "Continuous evolutionary programming: Analysis and experiments," Cybern. Syst., vol. 26, pp. 79-90, 1995.
43. F. Vavak and T. C. Forgarty, "Comparison of steady-state and generational genetic algorithms for use in nonstationary environments," in Proc. 1996 IEEE Int. Conf. Evolutionary Computa. (ICEC'96). New York: IEEE Press, 1996, pp. 192-195.
44. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning internal representations by error propagation," in Parallel Distributed Processing: Explorations in the Micro structures of Cognition, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge, MA: MIT Press, vol. I, 1986, pp. 318-362.
45. P. P. C. Yip and Y.-H. Pao, "Growing neural networks using guided evolutionary simulated annealing," in Proc. 3d Annu. Conf. Evolutionary Prog., A. V. Sebald and L. J. Fogel, Eds. Singapore: World Scientific, 1994, pp. 17-25.
46. Y. Shang and B. Wah, "Global optimization for neural network training," IEEE Computer, vol. 29, no. 3, pp. 45-54, 1996.
47. X. Yao and Y. Liu, "Fast evolutionary programming," in Evolutionary Programming V: Proc. 5th Annu. Conf. Evolutionary Programming, L. Fogel, P. Angeline, and T. Bäck, Eds. Cambridge, MA: MIT Press, 1996, pp. 451-460.
48. W. Finnoff, F. Hergent, and H. G. Zimmermann, "Improving model selection by nonconvergent methods," Neural Networks, vol. 6, pp. 771-783, 1993.
49. X. Yao and Y. Liu, "Ensemble structure of evolutionary artificial neural networks," in Proc. 1996 IEEE Int. Conf. Evolutionary Computa. (ICEC'96) New York: IEEE Press, 1996, pp. 659-664.
50. X. Yao, Y. Liu, and P. Darwen, "How to make best use of evolutionary learning," in Complex Systems: From Local Interactions to Global Phenomena, R. Stocker, H. Jelinek, and B. Durnota, Eds. Amsterdam, The Netherlands: IOS, 1996, pp. 229-242.
51. P. Darwen and X. Yao, "Automatic modularization by speciation," in Proc. 1996 IEEE Int. Conf. Evolutionary Computa. (ICEC'96). New York: IEEE Press, 1996, pp. 88-93.
52. D. Stork and J. Allen, "How to solve the N-bit parity problem with two hidden units," Neural Networks, vol. 5, no. 6, pp. 923-926, 1992.
53. L. Prechelt, "Some notes on neural learning algorithm benchmarking," Neurocomputing, vol. 9, no. 3, pp. 343-347, 1995.
54. D. Michie, D. J. Spiegelhalter, and C. C. Taylor, Machine Learning, Neural and Statistical Classification. London: Ellis Horwood, 1994.
55. A. Roy, S. Govil, and R. Miranda, "An algorithm to generate radial basis function (RBF)-like nets for classification problems," Neural Networks, vol. 8, pp. 179-201, 1995.
56. K. P. Bennett and O. L. Mangasarian, "Robust linear programming discrimination of two linearly inseparable sets," Optimization Methods Software, vol. 1, pp. 23-34, 1992.
57. W. Schiffmann, M. Joost, and R. Werner, "Synthesis and performance analysis of multilayer neural network architectures," Univ. Koblenz, Inst. für Physics, Koblenz, Germany, Tech. Rep. 16/1992, 1992.
58. J. D. Farmer and J. J. Sidorowich, "Predicting chaotic time series," Phys. Rev. Lett., vol. 59, no. 8, pp. 845-847, 1987.
59. M. Mackey and L. Glass, "Oscillation and chaos in physiological control systems," Sci., vol. 197, p. 287, 1977.
60. T. M. Martinetz, S. G. Berkovich, and K. J. Schulten, "'Neuralgas' network for vector quantization and its application to time-series prediction," IEEE Trans. Neural Networks, vol. 4, pp. 558-569, 1993.
61. R. S. Crowder, "Predicting the Mackey-Glass time series with cascade-correlation learning," in Proc. 1990 Connectionist Models Summer School, 1990, pp. 117-123.
62. J. Moody and C. J. Darken, "Fast learning in networks of locally tuned processing units," Neural Computa., vol. 1, pp. 281-294, 1989.