Pruning artificial neural networks using neural complexity measures

International Journal of Neural Systems

Volumn 18, Issue 5, 2008, Pages 389-403

  Jorgensen, Thomas D ^a   Haynes, Barry P ^a   Norlund, Charlotte C F ^b  

^a UNIVERSITY OF PORTSMOUTH   (United Kingdom)

^b UNIVERSITY OF SOUTHAMPTON   (United Kingdom)

Author keywords

Neural complexity; Neural networks; Neuroevolution; Pruning; Robot controllers

Indexed keywords

BACKPROPAGATION; ELECTRIC NETWORK TOPOLOGY; IMAGE CLASSIFICATION; INFORMATION THEORY; ROBOTICS; VEGETATION;

ARTIFICIAL NEURAL NETWORKS; BENCHMARK NETWORKS; COMPLEX NETWORKS; LEARNING TIMES; NETWORK TOPOLOGIES; NEURAL COMPLEXITIES; NEURAL COMPLEXITY; NEUROEVOLUTION; PRUNING; PRUNING METHODS; PRUNING TECHNIQUES; REDUCTION OF DIMENSIONALITIES; ROBOT CONTROL DOMAINS; ROBOT CONTROLLERS;

NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; COMPUTER PROGRAM; METHODOLOGY; ROBOTICS; TIME;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; NEURAL NETWORKS (COMPUTER); ROBOTICS; SOFTWARE; TIME FACTORS;

EID: 55849084307     PISSN: 01290657     EISSN: None     Source Type: Journal    
DOI: 10.1142/S012906570800166X     Document Type: Article

References (51)

1. J. Denker, D. Schwartz, B. Wittner, S. Solla R. Howard, L. Jackel and J. Hopfield, Large automatic learning, rule extraction and generalization, Complex. Systems 1 (5) (1987).
2. A. S. Weigend, D. E. Rumelhart and B. A. Huberman, Back-propagation, weight-elimination and time series prediction, in Proc. of the 1990 Summer School on Connectionist Models.
3. S. Nolfi and D. Floreano, Evolutionary Robotics; The Biology, Intelligence, and Technology of Self-Organizing Machines (MIT Press 2000).
4. G. Tononi, O. Sporns, and G. M. Edelman, A measure for brain complexity: Relating functional segregation and integration in the nervous system, in Proc. of the National Academy of Science of USA (May 1994).
5. Y. L. Cun, J. S. Denker, S. A. Solla, Optimal brain damage, Adv. Neural Inform. Process. Syst. 2 (1990).
6. B. Hassibi and D. G. Stork, Second order derivatives for network pruning: Optimal Brain Surgeon, Adv. Neural Inform. Process. Syst. 4 (1993).
7. X. Yao and Y. Liu, A new evolutionary system for Evolving artificial neural networks, IEEE Trans. Neural Networks 8 (3) (May 1997).
8. X. Yao, Evolving artificial neural networks, in Proc. of the IEEE 87 (9) (September 1999).
9. T. Schreiner, Ausdünnungsverfahren für Neuronale Netze, (Pruning methods for neural networks) Diploma Thesis 1140, University of Stuttgart, (1994).
10. R. Jacobs and M. Jordan, Adaptive mixtures of local experts, Neural Comput. 3 (1991).
11. S. E. Fahlman and C. Lebiere, The cascadecorrelation learning architecture, Adv. Neural Inform. Process. Syst. 2 (Los Altos CA, US, 1990).
12. P. Angeline and J. Pollack, Evolutionary module acquisition, in Proc. 2nd. Annu. Conf. on Evol. Prog. (1993).
13. P. Angeline, G. M. Saunders and J. B. Pollack, An evolutionary algorithm that constructs recurrent neural network, IEEE Trans. on Neural Networks (1994).
14. M. Lungarella, and O. Sporns, Information selfstructuring: Key principle for learning and development, in Proc. 4th IEEE Int. Conf. on Devel. and Learn. (2005).
15. O. Sporns, G. Tononi, and G. M. Edelman, Connectivity and complexity: The relationship between neuroanatomy and brain dynamics, in Neural Network 13 (2000).
16. O. Sporns, Small-world connectivity, motif composition, and complexity of fractal neuronal connections, in Biosystems 85 (2006).
17. O. Sporns, and M. Lungarella, Evolving coordinated behaviours by maximizing informational structure, in Proc. 10th Int. Conf. on Artif. Life, (2006).
18. L. S. Yaeger, and O. Sporns, Evolution of neural structure and complexity in a computational ecology, in Proc. 10th Int. Conf. on Artif. Life, (2006).
19. T. Jorgensen and B. Haynes, Evolution of artificial neural networks for robot control using speciation and complexity measures, in Proc. of IADIS - Intelligent Syst. and Agents (Amsterdam 2008).
20. T. Jorgensen and B. Haynes, Complexifying artificial neural networks through topological reorganization, in Proc. of WCE - Int. Conf. Comp. Intel. And Intel. Syst. (London 2008).
21. T. Jorgensen, B. Haynes, Evolving co-operative Behaviour, in Proc. of SAB06 Workshop on Bioinspired Cooperation and Adaptive Behaviours in Robots (Rome 2006).
22. K. O. Stanley, R. Miikkulainen, Continual coevolution through complexification, in Proc. of the Genet. and Evol. Conf. (2002).
23. M. C. Mozer and P. Smolensky, Skeletonization: A technique for trimming the fat from a network via relevance assessment, Adv. Neural Inform. Process. Syst. 1 (1989).
24. E.D. Karnin, A simple procedure for pruning backpropagation trained neural networks, IEEE Trans. Neural Networks 1 (2) (June 1990).
25. R. Reed, Pruning algorithms - A survey, IEEE Trans. Neural Networks 4 (5) (September 1993).
26. B. Hassibi, D. G. Stork and G. Wolff, Optimal brain surgeon: Extensions and performance comparisons, Proc. IEEE Int. Conf. Neural Network (San Francisco, 1993).
27. J. Xu, A node pruning algorithm based on optimal brain surgeon for feedforward neural networks, in Proc. 3rd Int. Symposium on Neural Network (China, 2006).
28. M. W. Pedersen, L. K. Hansen and J. Larsen, Pruning with generalization based weight saliencies: γOBD, γOBS, Adv. Neural Inform. Process. Syst. 9 (1996).
29. M. W. Pedersen and L. K. Hansen, Recurrent networks: Second order properties and pruning, Adv. Neural Inform. Process. Syst. 8 (1995).
30. A. Stahlberger and M. Riedmiller, Fast network pruning and feature extraction using the unit-OBS algorithm, Adv. Neural Inform. Process. Syst. 10 (1997).
31. S. G. Romaniuk, Pruning divide & conquer network, Network: Comput. Neural Syst. 4 (1993).
32. D. James and P. Tucker, A comparative analysis of simplification and complexification in the evolution of neural network topologies, in Proc. of the Genet. and Evol. Conf. (2004).
33. R. Setiono, A penalty function approach for pruning feedforward neural networks, Neural Comp 9 (1997).
34. T. Q. Huynh and R. Setiono, Effective neural network pruning using cross-validation, in Proc. Int. Joint Conf. Neural Networks (Montreal, Canada, 2005).
35. M. Marchesi, G. Orlandi, F. Piazza, G. Pignotti and A. Uncini, Dynamic topology neural network, in Proc. 3rd Italian Workshop on Parallel Arch. and Neural Networks, 1990.
36. Y. Chauvin, A back-propagation algorithm with optimal use of hidden units, Adv. Neural Inform. Process. Syst. 1 (1989).
37. S. J. Nowlan and G. E. Hinton, Simplifying neural networks by soft weights-Sharing, Neural Comput. 4 (1992).
38. E. Cantú-Paz, Pruning neural networks with distribution estimation algorithms, in Proc. of the Genet. and Evol. Conf. (2003).
39. D. Whitley and G. Bogart, The evolution of connectivity: Pruning neural networks using genetic algorithms, in Proc. Int. Joint Conf. Neural Networks 1 (Washington, 1990).
40. P. Hancock, Pruning neural nets by genetic algorithm, in Proc. Int. Conf. Neural Networks (1992).
41. M. Schmidt and T. Stidsen, Using GA to train NN using weight sharing, weight pruning and unit pruning, Technical report, Aarhus University, Computer Science Department (Aarhus, Denmark, 1995).
42. J. K. Kruschke, Creating local and distributed bottlenecks in hidden layers of back-propagation networks, in Proc. of the 1988 Connectionist Models Summer School.
43. J. K. Kruschke and J. Rodriguez-Movellan, Benefits of gain: Speeded learning and minimal hidden layers in back-propagation networks, IEEE Trans. Syst., Man and Cyhern. 21 (1991).
44. J. K. Kruschke, improving generalization in backpropagation networks with distributed bottlenecks, in Proc. Int. Joint Conf. Neural Networks 1 (Washington, 1989).
45. S. Fiori, F. Grimani, and P. Burrascano, Novel neural network feature selection procedure by generalization maximization with application to automatic robot guidance, Int. J. of Smart Eng. System Design, 4 (2) (2002) 91-106.
46. S. Fiori and P. Burrascano, Neural network feature selection applied to robot motion control, in Smart Engineering System Design: Neural Networks, Fuzzy logic, Evolutionary Programming, Complex Systems and Data Mining, (2000).
47. T. Jorgensen and B. Haynes, Simplification of artificial neural networks using complexity measures, in Proc. of Towards Autonomous Robotic Systems (2008).
48. G. N. Martin, Human Neuropsychology (Prentice Hall 1998, Reprinted 1999).
49. G. Edelman, Neural Darwinism - The Theory of Neuronal Group Selection (Basic Books New York 1989, Print by Oxford Press 1990).
50. S. M. Stahl, Essential Psychopharmacology: Neuro-Scientific Basis and Practical Applications (Cambridge University Press 2000).
51. C. E. Shannon, A Mathematical Theory of Communica-tion, The Bell Syst. Tech. J. 27 (1948).