Bio-inspired and gradient-based algorithms to train MLPs: The influence of diversity

Information Sciences

Volumn 179, Issue 10, 2009, Pages 1441-1453

  Pasti, Rodrigo ^a   de Castro, Leandro Nunes ^b  

^a UNIVERSITY OF CAMPINAS   (Brazil)

^b MACKENZIE PRESBYTERIAN UNIVERSITY   (Brazil)

Author keywords

Artificial immune systems; Backpropagation; Diversity; Ensembles; Evolutionary algorithm; Gradient based algorithms; Multi layer perceptrons; Particle swarm optimization

Indexed keywords

ALGORITHMS; BACKPROPAGATION; CLASSIFIERS; CONJUGATE GRADIENT METHOD; CYBERNETICS; IMMUNOLOGY; LEARNING SYSTEMS; LOCOMOTIVES; NEURAL NETWORKS; NEWTON-RAPHSON METHOD; PARTICLE SWARM OPTIMIZATION (PSO); PATTERN RECOGNITION SYSTEMS; RAILROAD CARS;

ARTIFICIAL IMMUNE SYSTEMS; DIVERSITY; ENSEMBLES; GRADIENT-BASED ALGORITHMS; MULTI-LAYER PERCEPTRONS; PARTICLE SWARM OPTIMIZATION;

BACKPROPAGATION ALGORITHMS;

EID: 61449099344     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2008.11.034     Document Type: Article

References (31)

1. Bäck T., Fogel D.B., and Michalewicz Z. Evolutionary Computation 1 Basic Algorithms and Operators (2000), Institute of Physics Publishing (IOP), Bristol and Philadelphia
2. Bäck T., Fogel D.B., and Michalewicz Z. Evolutionary Computation 2 Advanced Algorithms and Operators (2000), Institute of Physics Publishing (IOP), Bristol and Philadelphia
3. Breiman L. Bagging predictors. Machine Learning 24 2 (1996) 123-140
4. de Castro L.N., and Timmis J. Artificial Immune Systems: A New Computational Intelligence Approach (2002), Springer-Verlag
5. L.N. de Castro, J. Timmis, An artificial immune network for multimodal function optimization, in: Proceedings of the IEEE Congress on Evolutionary Computation, vol. 1, 2002, pp. 699-674.
6. de Castro L.N. Fundamentals of Natural Computing: Basic Concepts, Algorithms, and Applications (2006), CRC Press
7. de Castro L.N. Fundamentals of natural computing: an overview. Physics of Life Reviews 4 (2007) 1-36
8. Eiben A.E., and Smith J.E. Introduction to Evolutionary Computing (2003), Springer-Verlag
9. Y. Freund, R. Shapire, Experiments with new boosting algorithm, in: Proceedings of the 13th International Conference on Machine Learning, 1996, pp. 149-156.
10. Glover F.W., and Kochenberger G.A. Handbook of Metaheuristics (2003), Springer
11. Hansen L.K., and Salamon P. Neural network ensemble. IEEE Transactions On Pattern Analysis and Machine Intelligence 12 10 (1996)
12. Hashem S., and Schmeiser B. Improving model accuracy using optimal linear combinations of trained neural networks. IEEE Transactions on Neural Networks 6 3 (1995)
13. Hashem S. Optimal linear combinations of neural networks. Neural Networks 10 4 (1997) 599-614
14. Haykin S. Neural Networks: A Comprehensive Foundation. second ed. (1999), Prentice Hall
15. Jerne N.K. Towards a network theory of the immune system. Annales d'immunologie (Paris) 125C (1974) 373-389
16. J. Kennedy, R. Eberhart, Particle swarm optimization, in: Proceedings of the IEEE International Conference on Neural Networks, Perth, Australia, vol. 4, 1995, pp. 1942-1948.
17. Kennedy J. Particle swarm: optimization bases on sociocognition, Capi{dotless}́tulo do livro recent developments. In: de Castro L.N., and Von Zuben F.J. (Eds). Biologically Inspired Computing (2004), Idea Group Publishing, Capi{dotless}́tulo X 235-268
18. Kuncheva L.I., and Whitaker C.J. Measure of diversity in ensembles an their relationship with ensemble accuracy. Machine Learning 51 (2003) 181-207
19. Michalewicz Z., and Fogel D.B. How to Solve it: Modern Heuristics (2000), Springer-Verlag
20. Moller M.F. A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks 6 (1993) 525-533
21. D.J. Newman, S. Hettich, C.L. Blake, C.J. Merz, UCI Repository of machine learning databases, Department of Information and Computer Science, University of California, Irvine, CA, 1998, .
22. R. Pasti, L.N. de Castro, An immune and a gradient-based method to train multi-layer perceptron neural networks, in: Proceedings of the International Joint Conference on Neural Networks (World Congress of Computational Intelligence), 2006, pp. 4077-4084.
23. Pearlmutter B.A. Fast exact calculation by the hessian. Neural Computation 6 (1994) 147-160
24. Prechelt L. Automatic early stopping using cross validation: quantifying the criteria. Neural Networks 11 4 (1998) 761-767
25. L. Prechelt, Early stopping - but when?, in: G.B. Orr, K.-R. Müller, (Eds.), Neural Networks: Tricks of the Trade, Lecture Notes in Computer Science, vol. 1524, 1998, pp. 55-69.
26. D.E. Rumelhart, J.L. McClelland, The PDP Research Group, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1 Foundations, The MIT Press, 1986.
27. Shepherd A.J. Second-Order Methods for Neural Networks: Fast and reliable Methods for Multi-Layer Perceptrons (1997), Springer
28. Tumer K., and Ghosh J. Error correlation and error reduction in ensemble classifiers, connection Science. Special Issue on Combining Artificial Neural: Ensemble Approaches 8 34 (1996) 385-404
29. Van Der Smagt P.P. Minimization methods for training feedfoward neural networks. Neural Networks 1 7 (1994)
30. Witten I.H., and Frank E. Data Mining: Practical Learning Tool and Techniques. second ed. (2005), Morgan Kauffman Publishers
31. Zhou Z., Wu J., and Tang W. Ensembling neural networks: many could be better than all. Artificial Intelligence 137 1-2 (2002) 239-263