Multi-network evolutionary systems and automatic decomposition of complex problems

International Journal of General Systems

Volumn 35, Issue 3, 2006, Pages 259-274

  Khare, V R ^a,c,d   Yao, X ^a,e,f,g   Sendhoff, B ^b,f,h  

^a UNIVERSITY OF BIRMINGHAM   (United Kingdom)

^b HONDA RESEARCH INSTITUTE EUROPE GMBH   (Germany)

^c Natural Computation Research Group

^d School of Computer Science   (United Kingdom)

^e UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA   (China)

^f IEEE   (United Kingdom)

^g Centre of Excellence for Research in Computational Intelligence and Applications (CERCIA)   (United Kingdom)

^h Centre of Excellence for Research in Computational Intelligence and Applications ^*

Author keywords

Co evolution; Ensembles; Modular neural networks; Multi network systems

Indexed keywords

CO-EVOLUTION; ENSEMBLES; MODULAR NEURAL NETWORKS; MULTI-NETWORK SYSTEMS;

EVOLUTIONARY ALGORITHMS; NEURAL NETWORKS; PROBLEM SOLVING; STATISTICAL METHODS;

LARGE SCALE SYSTEMS;

EID: 33745827410     PISSN: 03081079     EISSN: 15635104     Source Type: Journal    
DOI: 10.1080/14689360600659772     Document Type: Article

References (41)

1. Y.S. Abu-Mostafa, "A method for learning from hints", in Advances in Neural Information Processing Systems, S.J. Hanson, J. Cowan and C.L. Giles, Eds., San Mateo, CA: Morgan Kaufmann, 1993, Vol. 5, pp. 73-80.
2. C.M. Bishop, Neural Networks for Pattern Recognition, Oxford, UK: Oxford University Press, 1996.
3. G. Brown, "Diversity in neural network ensembles", PhD thesis, University of Birmingham, 2004.
4. J.A. Bullinaria, 'To modularize or not to modularize?", in Proceedings of the 2002 U.K. Workshop on Computational Intelligence (UKCI-02), J. Bullinaria, Ed., Birmingham, 2002, pp. 3-10.
5. R. Calabretta, S. Nolfi, D. Parisi and G.P. Wagner, "A case study of the evolution of modularity: towards a bridge between evolutionary biology, artificial life, neuro- and cognitive science", in Proceedings of the Sixth International Conference on Artificial Life (ALIFE VI), C. Adami, R.K. Belew, H. Kitano and C.E. Taylor, Eds., Cambridge, MA: MIT Press, 1998, pp. 275-284.
6. A.D. Ferdinando, R. Calabretta and D. Parisi, "Evolving modular architectures for neural networks", in Proceedings of the Sixth Neural Computation and Psychology Workshop, R.M. French and J.P. Sougné, Eds., New York: Springer Verlag, 2001, pp. 253-264.
7. N. García-Pedrajas, C. Hervás-Martínez and J. Muñoz-Pérez, "Multi-objective cooperative coevolution of artificial neural networks", Neural Networks, 15, pp. 1259-1278, 2002.
8. N. García-Pedrajas, C. Hervás-Martínez and D. Ortiz-Boyer, "Cooperative coevolution of artificial neural network ensembles for pattern classification", IEEE Trans. Evolution. Comput., 9, pp. 271-302, 2005.
9. F. Gruau, D. Whitley and L. Pyeatt, "A comparison between cellular encoding and direct encoding for genetic neural networks", in Genetic Programming 1996: Proceedings of the First Annual Conference, J.R. Koza, D.E. Goldberg, D.B. Fogel and R.L. Riolo, Eds., Cambridge, MA: MIT Press, 1996, pp. 81-89.
10. J.V. Hansen, "Combining predictors: Meta machine learning methods and bias/variance and ambiguity decompositions". PhD thesis, Aarhus Universitet, Datalogisk Institut, 2000.
11. T. Hrycej, Modular Learning in Neural Networks. A Modularized Appproach to Neural Network Classification, New York: Wiley, 1992.
12. M. Hüsken, I.E. Gayko and B. Sendhoff, "Optimization for problem classes - neural networks that learn to learn", in 2000 IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks (ECNN 2000), X. Yao and D.F. Fogel, Eds., New York: IEEE Press, 2000, pp. 98-109.
13. M. Hüsken, C. Igel and M. Toussaint, "Task-dependent evolution of modularity in neural networks", Connect. Sci., 14, pp. 219-229, 2002.
14. C. Igel and M. Hüsken, "Empirical evaluation of the improved Rprop learning algorithm", Neurocomput., 50, pp. 105-123, 2003.
15. R.A. Jacobs, M.I. Jordan and A.G. Barto, "Task decomposition through competition in a modular connectionist architecture: the what and where vision tasks", Cognitive Sci., 15, pp. 219-250, 1991a.
16. R.A. Jacobs, M.I. Jordan, S.J. Nowlan and G.E. Hinton, "Adaptive mixtures of local experts", Neural Comput., 3, pp. 79-87, 1991b.
17. R.E. Jenkins and B.P. Yuhas, "A simplified neural-network solution through problem decomposition: the case of the truck backer-upper", Neural Comput., 4, pp. 647-649, 1992.
18. M.I. Jordan and R.A. Jacobs, "Modular and hierarchical learning systems", in The Handbook of Brain Theory and Neural Networks, M. Arbib, Ed., Cambridge, MA: MIT Press, 1995, pp. 579-582.
19. V.R. Khare, X. Yao, B. Sendhoff, et al. "Credit assignment among neurons in co-evolving populations", in 8th International Conference on Parallel Problem Solving from Nature, PPSN VIII, Lecture Notes in Computer Science, X.Y., Ed., New York: Springer Verlag, 2004, Vol. 3242, pp. 882-891.
20. V.R. Khare, X. Yao, B. Sendhoff, Y. Jin and H. Wersing, "Co-evolutionary modular neural networks for automatic problem decomposition", 2005 IEEE Congress on Evolutionary Computation, CEC 2005, Edinburgh, UK, IEEE Press, 2005, pp. 2691-2698.
21. R. Kohavi and G.H. John, "Wrappers for feature subset selection", Artif. Intell, 97, pp. 273-324, 1997.
22. G.G. Lendaris and K. Mathia, "On prestructuring anns using a priori knowledge", Proceedings of World Conference on Neural Networks (WCNN'94), San Diego, California, Earlbaum/INNS, 1994, pp. 319-324.
23. Y. Liao and J. Moody, "Constructing heterogeneous committees using input feature grouping: application to economic forecasting", Adv. Neural Inf. Process. Syst., 12, pp. 921 -927, 1999.
24. Y. Liu, "Negative correlation learning and evolutionary neural network ensembles", PhD thesis, Canberra, Australia: University College, The University of New South Wales, Australian Defence Force Academy, 1998.
25. E.N. Lorenz, "Deterministic nonperiodic flow", J. Atmosph, Sci., 20, pp. 130-141, 1963.
26. B.-L. Lu and M. Ito, "Task decomposition and module combination based on class relations: a modular neural network for pattern classification", IEEE Trans. Neural Networks, 10, pp. 1244-1256, 1999.
27. M.C. Mackey and L. Glass, "Oscillation and chaos in physiological control systems", Science, 197, pp. 287-289, 1977.
28. O. Maimon and L. Rokach, "Improving supervised learning by feature decomposition", Proceedings of the Second International Symposium on Foundations of Information and Knowledge Systems, Lecture Notes in Computer Science, New York: Springer Verlag, pp. 178-196, 2002.
29. D.E. Moriarty and R. Miikkulainen, "Forming neural networks through efficient and adaptive coevolution", Evolutionary Comput., 5, pp. 373-399, 1997.
30. D. Opitz, "Feature selection for ensembles", Proceedings of the Sixteenth National Conference on Artificial Intelligence, California, AAAI, 1999, pp. 379-384.
31. M.A. Potter and K.A. DeJong, "Cooperative coevolution: an architecture for evolving coadapted subcomponents", Evolution. Comput., 8, pp. 1-29, 2000.
32. E. Ronco, H. Gollee and P. Gawthrop, "Modular neural networks and selfdecomposition", Technical Report CSC-96012, Glasgow, UK: Centre for System and Control, University of Glasgow, 1997.
33. J.G. Rueckl, K.R. Cave and S. Kosslyn, "Why are "what" and "where" processed by separate cortical visual systems? A computational investigation", J. Cognitive Neurosci., 1, pp. 171-186, 1989.
34. G. Schlosser and G.P. Wagner (Eds.). Modularity in Development and Evolution, Chicago: The University of Chicago Press, 2005.
35. B. Sendhoff and M. Kreutz, "Variable encoding of modular neural networks for time series prediction", in Congress on Evolutionary Computation CEC, V. Porto, Ed., New York: IEEE Press, 1999, pp. 259-266.
36. A.J.C. Sharkey (Ed.). Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems, New York: Springer-Verlag, 1999.
37. N.E. Sharkey, "Neural networks for coordination and control: the portability of experiential representations", Robot. Autonom. Syst., 22, pp. 345-359, 1997.
38. V. Sindhwani, S. Rakshit, D. Deodhare, D. Erdogmus, J. Principe and P. Niyogi, "Feature selection in MLPs and SVMs based on maximum output information", IEEE Trans. Neural Networks, 15, pp. 937-948, 2004.
39. H. Yang and J. Moody, "Feature selection based on joint mutual information", Advances in Intelligent Data Analysis (AIDA), Computational Intelligence Methods and Applications (CIMA), International Computer Science Conventions, New York: Rochester, pp. 22-25, 1999.
40. C.H. Yong and R. Miikkulainen, "Cooperative coevolution of multi-agent systems", Technical Report AI01-287, Austin, TX, USA: Department of computer Sciences, The University of Texas at Austin, 2001.
41. M. Zaffalon and M. Hutter, "Robust feature selection by mutual information distributions", Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence, San Francisco: Morgan Kaufmann, 2002, pp. 577-584.