Multi-objective cooperative coevolution of artificial neural networks (multi-objective cooperative networks)

Neural Networks

Volumn 15, Issue 10, 2002, Pages 1259-1278

  Garcia Pedrajas N ^a   Hervas Martinez C ^b   Munoz Perez J ^b  

^a UNIVERSITY OF CÓRDOBA   (Spain)

^b UNIVERSITY OF MÁLAGA   (Spain)

Author keywords

Automatic neural network design; Cooperative coevolution; Evolutionary computation; Multi objective optimization; Topology evolution

Indexed keywords

COMPUTATIONAL METHODS; OPTIMIZATION; PERFORMANCE; TOPOLOGY;

COOPERATIVE COEVOLUTION;

NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; CLASSIFICATION; ERROR; EVALUATION; EVOLUTION; MODEL; PRIORITY JOURNAL;

EVOLUTION; NEURAL NETWORKS (COMPUTER);

EID: 0036887498     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(02)00095-3     Document Type: Article

References (80)

1. Anderson T.W. (1984) An introduction to multivariate statistical analysis. (2nd ed). Wiley, New York.
2. Angeline P.J., Saunders G.M., Pollack J.B. An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks. 5:(1):1994;54-65.
3. Baum E.B. Book review on neural network design and the complexity of learning. IEEE Transactions on Neural Networks. 2:1991;181-182.
4. Baum E.B., Haussler D. What net size gives valid generalization? Neural Computation. 1:1989;151-160.
5. Bäck T. Evolutionary algorithms in theory and practice. 1996;Oxford University Press, New York, NY.
6. Bebis G., Georgiopoulos M., Kasparis T. Coupling weight elimination with genetic algorithms to reduce network size and preserve generalization. Neurocomputing. 17:1997;167-194.
7. Belew, R. K., McInerney, J., & Schraudolph, N. N. (1991). Evolving networks: Using genetic algorithms with connectionist learning (Technical Report No. CS90-174). San Diego: Department of Computer Science Engineering, University of California.
8. Blum A.L., Rivest R.L. Training a 3-node neural network is NP-complete. Neural Networks. 5:1992;117-127.
9. Caelli T., Guan L., Wen W. Modularity in neural computing. Proceedings of the IEEE. 87:(9):1999;1497-1518.
10. Caelli T., Squire D., Wild T. Model-based neural network. Neural Networks. 6:(2):1993;613-625.
11. Carlson, D. (1996). Nevprop 3 ©user manual. Reno, NE.
12. Cho S.-B., Shimohara K. Evolutionary learning of modular neural networks with genetic programming. Applied Intelligence. 9:1998;191-200.
13. Cybenko G. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals, and Systems. 2:1989;303-314.
14. Deb, K. (1999). Evolutionary algorithms for multi-criterion optimization in engineering design. Proceedings of Evolutionary Algorithms in Engineering and Computer Science (EUROGEN'99) (pp. 135-161). Jyväskylä, Finland.
15. Deb K., Goldberg D.E. An investigation of niche and species formation in genetic function optimization. Schaffer J.D., Proceedings of the Third International Conference on Genetic Algorithms. 1989;42-50 Morgan Kaufmann, Los Altos, CA.
16. Detrano R., Janosi A., Steinbrunn W., Pfisterer M., Schmid J., Sandhu S., Guppy K., Lee S., Froelicher V. International application of a new probability algorithm for the diagnosis of coronary artery disease. American Journal of Cardiology. 64:1989;304-310.
17. Dillon W.R., Goldstein M. Multivariate analysis: Methods and applications. 1984;Wiley, New York.
18. Ficici S.G., Pollack J.B. Pareto optimality in coevolutionary learning. Kelemen J., Sosík P., Sixth European Conference on Artificial Life, ECAL 2001. 2001;316-325. Prague, Czech Republic.
19. Finnoff W., Hergert F., Zimmermann H.G. Improving model selection by nonconvergent methods. Neural Networks. 6:1993;771-783.
20. Fonseca C.M., Flemming P.J. Genetic algorithms for multi-objective optimization: Formulation, discussion, and generalization. Forrest S., Proceedings of the Fifth International Conference on Genetic Algorithms. 1993;416-423 Morgan Kauffman, Urbana, IL.
21. Fonseca C.M., Flemming P.J. An overview of evolutionary algorithms in multi-objective optimization. Evolutionary Computation. 3:(1):1995;1-16.
22. García-Pedrajas N., Hervás-Martínez C., Muñoz-Pérez J. Symbiont: A cooperative evolutionary model for evolving artificial neural networks for classification. Bouchon-Meunier B., Gutiérrez-Ríos J., Magdalena L., Yager R.R., Technologies for constructing intelligent systems. Vol. 2:2001;341-354 Springer, Berlin.
23. García-Pedrajas, N., Hervás-Martínez, C., & Muzñoz-Pérez, J. (submitted for publication). COVNET: a cooperative coevolutionary model for evolving artificial neural networks. IEEE Transactions on Neural Networks.
24. Giordana A., Neri F. Search-intensive concept induction. Evolutionary Computation. 3:(4):1996;375-416.
25. Girosi F., Jones M., Poggio T. Regularization theory and neural networks architectures. Neural Computation. 7:1995;219-269.
26. Goldberg D.E. Genetic algorithms in search, optimization and machine learning. 1989;Addison-Wesley, Reading, MA.
27. Goldberg, D. E., & Richardson, J. (1987). Genetic algorithms with sharing for multimodal function optimization. Proceedings of the Second International Conference on Genetic Algorithms (pp. 148-154). San Mateo, CA: Morgan Kaufmann.
28. Goutte C., Hansen L.K. Regularization with a pruning prior. Neural Networks. 10:(6):1997;1053-1059.
29. Guan L., Anderson J.A., Sutton J.P. A network of networks processing model for image regularization. IEEE Transactions on Neural Networks. 8:(1):1997;169-174.
30. Hancock P.J.B. Genetic algorithms and permutation problems: A comparison of recombination operators for neural net structure specification. Whitley D., Schaffer J.D., Proceedings of the International Workshop of Combinations of Genetic Algorithms and Neural Networks (COGANN-92). 1992;108-122 IEEE Computer Society Press, Los Alamitos, CA.
31. Hansen, L. K., & Pedersen, M. W. (1994). Controlled growth of cascade-correlation nets. Proceedings of the International Conference on Artificial Neural Networks (Vol. 1, pp. 797-800). Sorrento, Italy.
32. Haykin S. Neural networks - A comprehensive foundation. 1994;Macmilan, New York, NY.
33. Haynes T.D., Sen S. Co-adaptation in a team. International Journal of Computational Intelligence and Organizations (IJCIO). 1:(4):1997.
34. Hillis W.D. Co-evolving parasites improves simulated evolution as an optimization technique. Langton C.G., Taylor C., Farmer J.D., Rasmussen S., Artificial life. Vol. ii:1991;313-384 Addison-Wesley, Reading, MA.
35. Horn J., Goldberg D.E., Deb K. Implicit niching in a learning classifier system: natures's way. Evolutionary Computation. 2:(1):1994;37-66.
36. Hornik K., Stinchombe M. Multilayer feed-forward networks are universal approximators. White H., et al. Artificial neural networks: Approximation and learning theory. 1992;Blackwell Science, Oxford.
37. Jolliffe I.T. Principal components analysis. 1986;Springer, New York, NY.
38. Judd J.S. On the complexity of loading shallow neural networks. Journal of Complexity. 4:1988;177-192.
39. Judd J.S. Neural network design and the complexity of learning. 1990;MIT Press, Cambridge, MA.
40. Juillé, H., & Pollack, J. B. (1996). Coevolving intertwined spirals. Proceedings of the Fifth Annual Conference on Evolutionary Programming (pp. 461-468). Cambridge, MA: MIT Press.
41. Kirkpatrick S. Jr., C D.G., Vecchi M.P. Optimization by simulated annealing. Science. 220:1983;671-680.
42. Koza J.R. Genetic programming II. Automatic discovery of reusable programs. 1994;MIT Press, Cambridge, MA.
43. Lin S.-H., Kung S.Y., Lin L.J. Face recognition/detection by probabilistic decision-based neural networks. IEEE Transactions on Neural Networks. 8:(1):1997;114-132.
44. Liu Y., Yao X. Ensemble learning via negative correlation. Neural Networks. 12:(10):1999;1399-1404.
45. Liu Y., Yao X., Higuchi T. Evolutionary ensembles with negative correlation learning. IEEE Transactions on Evolutionary Computation. 4:(4):2000;380-387.
46. Liu, Y., Yao, X., Zhao, Q., & Higuchi, T. (2001). Evolving a cooperative population of neural networks by minimizing mutual information. Proceedings of the 2001 IEEE Congress on Evolutionary Computation (pp. 384-389). Seoul, Korea.
47. Michalewicz Z. Genetic algorithms + data structures = evolution programs. 1994;Springer, New York.
48. Moriarty, D. E. (1997). Symbiotic evolution of neural networks in sequential decision tasks. Unpublished doctoral dissertation, University of Texas at Austin (Report AI97-257).
49. Moriarty D.E., Miikkulainen R. Efficient reinforcement learning through symbiotic evolution. Machine Learning. 22:1996;11-32.
50. Moriarty D.E., Miikkulainen R. Forming neural networks through efficient and adaptive coevolution. Evolutionary Computation. 4:(5):1998.
51. NeuralWare (1993). Neural computing: A technology handbook for professional II/Plus. Pittsburgh, PA.
52. Paredis J. Coevolutionary computation. Artificial Life. 2:1995;355-375.
53. Potter M.A., Jong K.A. Cooperative coevolution: an architecture for evolving coadapted subcomponents. Evolutionary Computation. 8:(1):2000;1-29.
54. Prechelt, L. (1994). Proben1 - A set of neural network benchmark problems and benchmarking rules (Technical Report No. 21/94). Karlsruhe, Germany: Fakultät für Informatik, Universität Karlsruhe.
55. Prechelt L. A quantitative study of experimental evaluation of neural network learning algorithms. Neural Networks. 9:1996;457-462.
56. Puppala, N., Sen, S., & Gordin, M. (1998). Shared memory based cooperative coevolution. Proceedings of the Fifth IEEE International Conference on Evolutionary Computation (pp. 570-574). IEEE Publishing.
57. Quinlan J.R. C4.5: Programs for machine learning. 1993;Morgan Kaufmann, San Mateo.
58. Rosenberg, R. S. (1967). Simulation of genetic populations with biochemical properties. Unpublished doctoral dissertation, University of Michigan.
59. Rosin C.D., Belew R.K. New methods for competitive coevolution. Evolutionary Computation. 5:(1):1997;1-29.
60. Schaffer, J. D. (1984). Some experiments in machine learning using vector evaluated genetic algorithm. Unpublished doctoral dissertation, Nashville, TN: Vanderbilt University.
61. Schaffer J.D., Whitley L.D., Eshelman L.J. Combinations of genetic algorithms and neural networks: A survey of the state of the art. Whitley L.D., Schaffer J.D., Proceedings of Cogann-92 International Workshop on Combinations of Genetic Algorithms and Neural Networks. 1992;1-37 IEEE Computer Society Press, Los Alamitos, CA.
62. Sejnowski T.J., Hinton G.E., Touretzky D.S., Faster-learning variations on back-propagation: An empirical study. 1988;Morgan Kaufmann, San Mateo, CA.
63. Shang Y., Wah B.W. Global optimization for neural networks training. IEEE Computer. 29:(3):1996;45-54.
64. Sima J. Loading deep networks is hard. Neural Computation. 6:(5):1994;842-850.
65. Smalz R., Conrad M. Combining evolution with credit apportionment: A new learning algorithm for neural nets. Neural Networks. 7:(2):1994;341-351.
66. Smith J.M. Evolutionary genetics. 1989;Oxford University Press, New York, NY.
67. Smith, J. W., Everhart, J. E., Dickson, W. C., Knowler, W. C., & Johannes, R. S. (1988). Using the ADAP learning algorithm to forecast the onset of diabetes mellitus. Proceedings of the Symposium on Computer Applications and Medical Care (pp. 261-265). IEEE Computer Society Press.
68. SPSS, I. (1999). SPSS ©9.0 advanced models. Chicago, IL: SPSS Inc.
69. Srinivas N., Deb K. Multi-objective function optimization using non-dominated sorting genetic algorithms. Evolutionary Computation. 2:(3):1994;221-248.
70. Steuer R.E. Multiple criteria optimization: Theory, computation, and application. 1986;Wiley, New York, NY.
71. Syswerda G. A study of reproduction in generational and steady-state genetic algorithms. Rawlins G., Foundations of genetic algorithms. 1991;94-101 Morgan Kaufmann, Los Altos, CA.
72. Whitehead B.A., Choate T.D. Cooperative-competitive genetic evolution of radial basis function centers and widths for time series prediction. IEEE Transactions on Neural Networks. 7:(4):1996.
73. Whitley, D. (1989). The GENITOR algorithm and selective pressure. Proceedings of the Third International Conference on Genetic Algorithms (pp. 116-121). Los Altos, CA: Morgan Kaufmann.
74. Whitley, D., & Kauth, J. (1988). GENITOR: a different genetic algorithm. Proceedings of the Rocky Mountain Conference on Artificial Intelligence (pp. 118-130). Denver, CO.
75. Whitley D., Starkweather T. GENITOR II: a distributed genetic algorithm. Journal of the Experimental Theoretical Artificial Intelligence. 1990;189-214.
76. Yao X. Evolving artificial neural networks. Proceedings of the IEEE. 9:(87):1999;1423-1447.
77. Yao X., Liu Y. A new evolutionary system for evolving artificial neural networks. IEEE Transactions on Neural Networks. 8:(3):1997;694-713.
78. Yao, X., & Shi, Y. (1995). A preliminary study on designing artificial neural networks using co-evolution. Proceedings of the IEEE Singapore International Conference on Intelligent Control and Instrumentation (pp. 149-154). Singapore.
79. Zitzler, E. (1999). Evolutionary algorithms for multiobjective optimization: methods and applications. Unpublished doctoral dissertation, Eidgenössische Technische Hochschule Zürich.
80. Zitzler, E., & Thiele, L. (1998). Multi-objective optimization using evolutionary algorithms - a comparative case study. Problem solving from nature - PPSN V (pp. 292-301). Amsterdam, The Netherlands: Springer.