Encoding subcomponents in cooperative co-evolutionary recurrent neural networks

Neurocomputing

Volumn 74, Issue 17, 2011, Pages 3223-3234

  Chandra, Rohitash ^a,b   Frean, Marcus ^a   Zhang, Mengjie ^a   Omlin, Christian W ^c  

^a VICTORIA UNIVERSITY OF WELLINGTON   (New Zealand)

^b FIJI NATIONAL UNIVERSITY   (Fiji)

^c MIDDLE EAST TECHNICAL UNIVERSITY   (Turkey)

Author keywords

Cooperative coevolution; Genetic algorithms; Grammatical inference; Neuro evolution; Recurrent neural networks

Indexed keywords

CO-EVOLUTIONARY; COOPERATIVE CO-EVOLUTION; ENCODING SCHEMES; GRAMMATICAL INFERENCES; HIGH-DIMENSIONAL; NEURAL NETWORK TRAINING; NEURO-EVOLUTION; NON-SEPARABILITY; PROBLEM-DECOMPOSITION METHODS; SEARCH PROBLEM;

BIOLOGY; COMPUTATIONAL GRAMMARS; ENCODING (SYMBOLS); EVOLUTIONARY ALGORITHMS; GENETIC ALGORITHMS;

RECURRENT NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; FUZZY SYSTEM; MATHEMATICAL COMPUTING; NERVE CELL; PRIORITY JOURNAL;

EID: 80052929304     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2011.05.003     Document Type: Article

References (36)

1. Giles C.L., Omlin C., Thornber K.K. Equivalence in knowledge representation: automata, recurrent neural networks, and dynamical fuzzy systems. Proc. IEEE 1999, 87(9):1623-1640.
2. Hornik K., Stinchcombe M., White H. Multilayer feedforward networks are universal approximators. Neural Networks 1989, 2(5):359-366.
3. Scarselli F., Tsoi A.C. Universal approximation using feedforward neural networks: a survey of some existing methods, and some new results. Neural Networks 1998, 11(1):15-37.
4. Schaefer A., Zimmermann H. Recurrent neural networks are universal approximators. Int. J. Neural Syst. 2007, 17(4):253-263.
5. Bengio Y., Simard P., Frasconi P. Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Networks 1994, 5(2):157-166.
6. Frasconi P., Gori M., Tesi A. Successes and failures of backpropagation: a theoretical investigation. Progress in Neural Networks 1993, 205-242. Ablex Publishing.
7. Yao X. Evolving artificial neural networks. Proc. IEEE 1999, 87(9):1423-1448.
8. Potter M.A., De Jong K.A. A cooperative coevolutionary approach to function optimization. PPSN III: Proceedings of the International Conference on Evolutionary Computation. The Third Conference on Parallel Problem Solving from Nature 1994, 249-257. Springer-Verlag, London.
9. Gomez F., Mikkulainen R. Incremental evolution of complex general behavior. Adapt. Behav. 1997, 5(3-4):317-342. 〈http://dx.doi.org/10.1177/105971239700500305〉.
10. F.J. Gomez, Robust Non-Linear Control through Neuroevolution, Technical Report AI-TR-03-303, Ph.D. Thesis, Department of Computer Science, The University of Texas at Austin, 2003.
11. Gomez F., Schmidhuber J., Miikkulainen R. Accelerated neural evolution through cooperatively coevolved synapses. J. Mach. Learn. Res. 2008, 9:937-965.
12. Garcia-Pedrajas N., Hervas-Martinez C., Munoz-Perez J. COVNET: a cooperative coevolutionary model for evolving artificial neural networks. IEEE Trans. Neural Networks 2003, 14(3):575-596.
13. Garcia-Pedrajas N., Hervas-Martinez C., Ortiz-Boyer D. Cooperative coevolution of artificial neural network ensembles for pattern classification. IEEE Trans Evolut. Comput. 2005, 9(3):271-302.
14. Chandra R., Frean M., Zhang M. An encoding scheme for cooperative coevolutionary neural networks. Twenty-Third Australian Joint Conference on Artificial Intelligence, Lecture Notes in Artificial Intelligence 2010, 253-262. Springer-Verlag, Adelaide, Australia.
15. Y. Liu, X. Yao, Q. Zhao, T. Higuchi, Scaling up fast evolutionary programming with cooperative coevolution, in: Proceedings of the 2001 Congress on Evolutionary Computation, 2001, pp. 1101-1108, doi:10.1109/CEC.2001.934314.
16. Salomon R. Re-evaluating genetic algorithm performance under coordinate rotation of benchmark functions. A survey of some theoretical and practical aspects of genetic algorithms. Biosystems 1996, 39(3):263-278.
17. Schmidhuber J., Wierstra D., Gagliolo M., Gomez F. Training recurrent networks by Evolino. Neural Comput. 2007, 19(3):757-779. 〈http://dx.doi.org/10.1162/neco.2007.19.3.757〉.
18. Blanco A., Delgado M., Pegalajar M.C. A real-coded genetic algorithm for training recurrent neural networks. Neural Network 2001, 14(1):93-105. 〈http://dx.doi.org/10.1016/S0893-6080(00)00081-2〉.
19. M. Tomita, Dynamic construction of finite automata from examples using hill-climbing, in: Proceedings of the Fourth Annual Cognitive Science Conference, Ann Arbor, MI, 1982, pp. 105-108.
20. Castaño M.A., Vidal E., Casacuberta F. Finite state automata and connectionist machines: a survey. IWANN, Lecture Notes in Computer Science 1995, vol. 930:433-440. Springer.
21. Elman J.L. Finding structure in time. Cognitive Sci. 1990, 14:179-211.
22. Manolios P., Fanelli R. First-order recurrent neural networks and deterministic finite state automata. Neural Comput. 1994, 6(6):1155-1173. 〈http://dx.doi.org/10.1162/neco.1994.6.6.1155〉.
23. Omlin C.W., Giles C.L. Constructing deterministic finite-state automata in recurrent neural networks. J. ACM 1996, 43(6):937-972. 〈http://doi.acm.org/10.1145/235809.235811〉.
24. Omlin C.W., Thornber K.K., Giles C.L. Fuzzy finite state automata can be deterministically encoded into recurrent neural networks. IEEE Trans. Fuzzy Syst. 1998, 6:76-89.
25. Watrous R.L., Kuhn G.M. Induction of finite-state languages using second-order recurrent networks. Neural Comput. 1992, 4(3):406-414. 〈http://dx.doi.org/10.1162/neco.1992.4.3.406〉.
26. Chandra R., Omlin C.W. Training and extraction of fuzzy finite state automata in recurrent neural networks. Proceedings of International Conference on Computational Intelligence 2006, 274-279.
27. Brookshear J.G. Theory of computation: formal languages, automata, and complexity 1989, Benjamin-Cummings Publishing Co., Inc., Redwood City, CA, USA.
28. Ortiz-Boyer D., HerváMartínez C., García-Pedrajas N. Cixl2: a crossover operator for evolutionary algorithms based on population features. J. Artif. Intell. Res. 2005, 24:1-48.
29. Yang Z., Tang K., Yao X. Large scale evolutionary optimization using cooperative coevolution. Inf. Sci. 2008, 178(15):2985-2999. 〈http://dx.doi.org/10.1016/j.ins.2008.02.017〉.
30. Yang Z., Tang K., Yao X. Multilevel cooperative coevolution for large scale optimization. IEEE Congress on Evolutionary Computation 2008, 1663-1670.
31. Omidvar M., Li X., Yao X. Cooperative co-evolution for large scale optimization through more frequent random grouping. IEEE Congress on Evolutionary Computation (CEC), 2010 2010, 1754-1761.
32. Potter M.A., De Jong K.A. Cooperative coevolution: an architecture for evolving coadapted subcomponents. Evol. Comput. 2000, 8(1):1-29. 〈http://dx.doi.org/10.1162/106365600568086〉.
33. Deb K., Anand A., Joshi D. A computationally efficient evolutionary algorithm for real-parameter optimization. Evol. Comput. 2002, 10(4):371-395.
34. Werbos P.J. Backpropagation through time: what it does and how to do it. Proc. IEEE 1990, 78(10):1550-1560.
35. van den Bergh F., Engelbrecht A. A cooperative approach to particle swarm optimization. IEEE Trans. Evolut. Comput. 2004, 8(3):225-239.
36. Shi Y.-j., Teng H.-f., Li Z.-q. Cooperative co-evolutionary differential evolution for function optimization. Advances in Natural Computation, Lecture Notes in Computer Science 2005, vol. 3611:1080-1088. Springer Berlin/Heidelberg.