Task-dependent evolution of modularity in neural networks

Connection Science

Volumn 14, Issue 3, 2002, Pages 219-229

  Hüsken, Michael ^a   Igel, Christian ^a   Toussaint, Marc ^a  

^a RUHR UNIVERSITY BOCHUM   (Germany)

Author keywords

Learning; Measures for modularity; Structure evolution; Task decomposition

Indexed keywords

INFORMATION THEORY; LARGE SCALE SYSTEMS; LEARNING SYSTEMS; PROBLEM SOLVING;

MODULAR PROBLEMS;

NEURAL NETWORKS;

EID: 0036768801     PISSN: 09540091     EISSN: None     Source Type: Journal    
DOI: 10.1080/09540090208559328     Document Type: Article

References (25)

1. Bullinaria, J. A., 2001, Simulating the evolution of modular neural systems. In J. D. Moore and K. Stenning (eds) Proceeding of the Twenty-third Annual Conference of the Cognitive Science Society, Edinburgh, Scotland (Mahwah, NJ: USA (Lawrence Erlbaum Associates), pp. 146-151.
2. Elman, J. L., Bates, E. A., Johnson, M. H., Karmiloff-Smith, A., Parisi, D., and Plunkett, K., 1996, Rethinking Innateness-A Connectionist Perspective on Development (Cambridge MA: MIT Press).
3. Ferdinando, A. D., Calabretta, R., and Parisi, D., 2001, Evolving modular architectures for neural networks. In R. French and J. Sougne (eds) Proceedings of the Sixth Neural Computation and Psychology Workshop: Evolution, Learning and Development (Berlin: Springer), pp. 253-262.
4. Fodor, J. A., 1983, The Modularity of Mind: An Essay on Faculty Psychology (Cambridge MA: MIT Press).
5. Friedrich, C. M., and Moraga, C., 1996, An evolutionary method to find good building-blocks for architectures of artificial neural networks. In Sixth International Conference on Information Processing and Management of Uncertainty in Knowledge Based Systems (IPMU '96), Granada, Spain, pp. 951-956.
6. Grau, F., 1995, Automatic definition of modular neural networks. Adaptive Behavior, 3 (2): 151-183.
7. Hüsken, M., Gayko, J. E. and Sendhoff, B. A., 2000, Optimization for problem classes-neural networks that learn to learn. In X. Yao and D. B. Fogel (eds) IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks (ECNN 2000) San Antonio, TX, USA (IEEE Press), pp. 98-109.
8. Hüsken. M., Igel, C. and Toussaint, M., 2001, Task-dependent evolution of modularity in neural networks-a quantitative case study. In E. D. Goodman (ed.) Late-breaking Papers at the Genetic and Evolutionary Computation Conference (GECCO-2001), San Francisco, CA, USA, pp. 187-193.
9. Igel, C., and Hüsken, M., 2003, Empirical evaluation of the improved Rprop learning algorithms. Neurocomputing, January 2003, 50:105-123.
10. Jordan, M. I., and Jacobs, A., 1995, Modular and hierarchical learning systems. In M. A. Arbib (ed.) The Handbook of Brain Theory and Neural Networks (Cambridge MA: MIT Press), pp. 579-582.
11. Kitano, H., 1990, Designing neural networks using genetic algorithms with graph generation system. Complex Systems, 4: 461-476.
12. Lam, C.-H., and Shin, F. G., 1998, Formation and dynamics of modules in a dual-tasking multilayer feed-forward neural network. Physical Review E, 58 (3): 3673-3677.
13. Nolfi, S., and Parisi, D., 1999, Learning and evolution. Autonomous Robots, 7 (1): 89-113.
14. Riedmiller, M., and Braun, H., 1993, A direct adaptive method for faster backpropagation learning: the RPROP algorithm. In Proceedings of the IEEE International Conference on Neural Networks, San Francisco, CA, USA (IEEE Press), pp. 586-591.
15. Rueckl, J. G., Cave, K. R., and Kosslyn, S. M., 1989, Why are 'what' and 'where' processed by separate cortical visual systems? A computational investigation. Journal of Cognitive Neuroscience, 1: 171-186.
16. Schwefel, H.-P., 1995, Evolution and Optimum Seeking. Sixth-Generation Computer Technology Series (New York: John Wiley & Sons).
17. Sendhoff, B. A., 1998. Evolution of Structures-Optimization of Artificial Neural Structures for Information Processing (Aachen: Shaker).
18. Sendhoff, B. A., and Kreutz, M., 1999, Variable encoding of modular neural networks for time series prediction. In Congress on Evolutionary Computation (CEC '99) Washington DC, USA, Vol. 1 (IEEE Press), pp. 259-266.
19. Sharkey, A., 1996. On combining artificial neural networks. Connection Science, 8 (3): 299-314
20. Sharkey, A. 1997, Modularity, combining and artificial neural nets. Connection Science, 9 (1): 3-10.
21. Simon, H. A., 1962, The architecture of complexity. Proceedings of the American Philosophical Society, 106: 467-482.
22. Snoad, N., and Bossomaier. T., 1995, MONSTER-the ghost in the connection machine: modularity of neural systems in theoretical evolutionary research. In Proceedings of the 1995 ACM/IEEE Supercomputing Conference, San Diego, CA, USA, (ACM Press and IEEE Press).
23. Toussaint, M., 2002a, A neural model for multi-expert architectures. In Proceedings of the International Joint Conference on Neural Networks (IJCNN 2002). Honolulu, HI, USA, (IEEE Press), pp. 2755-2760.
24. Toussaint. M., 2002b, On model selection and the disability of neural networks to decompose tasks. In Proceedings of the International Joint Conference on Neural Networks (IJCNN 2002), Honolulu, HI, USA, (IEEE Press), pp. 245-250.
25. Yao, X., 1999, Evolving artificial neural networks. Proceedings of the IEEE, 87 (9): 1423-1447.