A multi-objective memetic and hybrid methodology for optimizing the parameters and performance of artificial neural networks

Neurocomputing

Volumn 73, Issue 7-9, 2010, Pages 1438-1450

  Almeida, Leandro M ^a   Ludermir, Teresa B ^a  

^a FEDERAL UNIVERSITY OF PERNAMBUCO   (Brazil)

Author keywords

Artificial neural networks; Evolutionary algorithms; Memetic algorithms and hybrid intelligent systems; Particle swarm optimization

Indexed keywords

ARTIFICIAL NEURAL NETWORK; ARTIFICIAL NEURAL NETWORKS; EVOLUTION STRATEGIES; GLOBAL SEARCH; HIDDEN NODES; HYBRID INTELLIGENT SYSTEM; HYBRID METHODOLOGIES; INITIAL WEIGHTS; LEVENBERG-MARQUARDT TRAINING ALGORITHM; LOCAL SEARCH; MANUAL PROCESS; MEMETIC; MEMETIC ALGORITHMS; MULTI OBJECTIVE; MULTI-LAYER PERCEPTRONS; NEURAL NETWORK PARAMETERS; SPECIFIC PROBLEMS; TIME SPENT; TRAINING ALGORITHMS;

ALGORITHMS; INTELLIGENT SYSTEMS; PARAMETER ESTIMATION; PARTICLE SWARM OPTIMIZATION (PSO); PATTERN RECOGNITION SYSTEMS;

NEURAL NETWORKS;

ALGORITHM; ANALYTIC METHOD; ARTICLE; ARTIFICIAL NEURAL NETWORK; BACK PROPAGATION; EVOLUTIONARY ALGORITHM; GENETIC ALGORITHM; INTERMETHOD COMPARISON; LEVENBERG MARQUARDT ALGORITHM; MACHINE LEARNING; MATHEMATICAL ANALYSIS; PARTICLE SWARM OPTIMIZATION; PRIORITY JOURNAL;

EID: 77949266538     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2009.11.007     Document Type: Article

References (51)

1. Abraham A. Meta learning evolutionary artificial neural networks. Neurocomputing 56 (2004) 1-38
2. L.M. Almeida, T.B. Ludermir, Automatically searching near-optimal artificial neural networks, in: Proceedings of the European Symposium on Artificial Neural Networks (ESANN'07), 2007, pp. 549-554.
3. L.M. Almeida, T.B. Ludermir, An evolutionary approach for tuning artificial neural network parameters, in: Proceedings of the Third International Workshop on Hybrid Artificial Intelligence Systems (HAIS'08), 2008, pp. 156-163.
4. L.M. Almeida, T.B. Ludermir, An improved method for automatically searching near-optimal artificial neural networks, in: IEEE International Joint Conference on Neural Networks (IJCNN'08) (IEEE World Congress on Computational Intelligence), 2008, pp. 2235-2242.
5. A. Asuncion, D. Newman, UCI machine learning repository (2007). URL: 〈http://www.ics.uci.edu/∼mlearn/MLRepository.html〉.
6. T. Back, H. Hoffmeister, H. Schwefel, A survey of evolution strategies, in: Proceedings of the Fourth International Conference on Genetic Algorithms, 1991, pp. 2-9.
7. Bishop C.M. Neural Networks for Pattern Recognition (1995), Oxford University Press, Oxford
8. Bourlard H.A., and Morgan N. Connectionist Speech Recognition: A Hybrid Approach (1993), Kluwer Academic Publishers, Dordrecht
9. Cai X., Zhang N., Venayagamoorthy G.k., and WunschII D.C. Time series prediction with recurrent neural networks trained by a hybrid pso-ea algorithm. Neurocomputing 70 13-15 (2007) 2342-2353
10. Cantú-Paz E., and Kamath C. An empirical comparison of combinations of evolutionary algorithms and neural networks for classification problems. IEEE Transactions on Systems, Man, and Cybernetics, Part B 35 5 (2005) 915-927
11. Chen Y., Abraham A., and Yang B. Hybrid flexible neural tree based intrusion detection systems. International Journal of Intelligent Systems 22 (2007) 1-16
12. Chen Y., Abraham A., and Yang B. Feature selection and classification using flexible neural tree. Neurocomputing 70 1-3 (2006) 305-313
13. Chen Y., Yang B., and Abraham A. Flexible neural trees ensemble for stock index modeling. Neurocomputing 70 4-6 (2007) 697-703
14. Chen Y., Yang B., Dong J., and Abraham A. Time series forecasting using flexible neural tree model. Information Sciences 174 3-4 (2005) 219-235
15. Coelho C.A.C., Lamont G.B.L., and Veldhuizen D.A.V. Evolutionary Algorithms for Solving Multi-objective Problems (Genetic and Evolutionary Computation) (2006), Springer-Verlag New York, Inc., Secaucus, NJ, USA
16. Cortez P., Rocha M., and Neves J. Evolving time series forecasting arma models. Journal of Heuristics 10 4 (2004) 415-429
17. Dirst M., and Weigend A.S. Time Series Prediction: Forecasting the Future and Understanding the Past (1994), Addison-Wesley, Reading, MA
18. Dorigo M., and Caro G.D. Ant colony optimization: a new meta-heuristic. Proceedings of the Congress on Evolutionary Computation (1999) 1470-1477
19. Eiben A.E., and Smith J.E. Introduction to Evolutionary Computing (2003), Springer, Berlin
20. Emmert-Streib F. Influence of the neural network topology on the learning dynamics. Neurocomputing 69 10-12 (2006) 1179-1182
21. Ferentinos K.P. Biological engineering applications of feedforward neural networks designed and parameterized by genetic algorithms. Neural Networks 18 7 (2005) 934-950
22. Fletcher R. Practical Methods of Optimization. second ed. (1987), Wiley-Interscience, New York, NY, USA
23. García-Pedrajas N., and Fyfe C. Construction of classifier ensembles by means of artificial immune systems. Journal of Heuristics 14 3 (2008) 285-310
24. García-Pedrajas N., Hervás-Martínez C., and Ortiz-Boyer D. Cooperative coevolution of artificial neural network ensembles for pattern classification. IEEE Transactions on Evolutionary Computation 9 3 (2005) 271-302
25. García-Pedrajas N., Ortiz-Boyer D., and Hervás-Martínez C. Cooperative coevolution of generalized multi-layer perceptrons. Neurocomputing 56 (2004) 257-283
26. Goldberg D.E. Genetic Algorithms in Search, Optimization and Machine Learning (1989), Addison-Wesley, Longman Publishing Co., Reading, MA, New York
27. S. Goonatilake, S. Khebbal (Eds.), Intelligent Hybrid Systems, Wiley, New York, NY, USA, 1994.
28. Hanne T. Global multiobjective optimization using evolutionary algorithms. Journal of Heuristics 6 3 (2000) 347-360
29. Haykin S. Neural Networks: A Comprehensive Foundation. second ed. (1999), Prentice-Hall, Englewood Cliffs, NJ
30. Hwarng H.B., and Ang H.T. A simple neural network for ARMA(p,q) time series. Omega 29 4 (2001) 319-333
31. Islam M.M., and Murase K. A new algorithm to design compact two-hidden-layer artificial neural networks. Neural Networks 14 9 (2001) 1265-1278
32. Kasabov N.K. Foundations of Neural Networks, Fuzzy Systems, and Knowledge Engineering (1996), MIT Press, Cambridge, MA, USA
33. J. Kennedy, R. Eberhart, Particle swarm optimization, in: Proceedings of the International Joint Conference on Neural Networks, vol. 4, 1995, pp. 1942-1948.
34. Kleijnen J.P.C., Sanchez S.M., Lucas T.W., and Cioppa T.M. State-of-the-art review: a user's guide to the brave new world of designing simulation experiments. INFORMS Journal on Computing 17 3 (2005) 263-289
35. Kordos M., and Duch W. A survey of factors influencing mlp error surface. Control and Cybernetics 33 (2004) 611-631
36. Levenberg K. A method for the solution of certain non-linear problems in least squares. Quarterly Applied Mathematics 2 (1944) 164-168
37. L.B. Liu, Y.J. Wang, D. Huang, Designing neural networks using pso-based memetic algorithm, in: Proceedings of the Fourth International Symposium on Neural Networks (ISNN'07), 2007, pp. 219-224.
38. Ma L., and Khorasani K. New training strategies for constructive neural networks with application to regression problems. Neural Networks 17 4 (2004) 589-609
39. Mandischer M. A comparison of evolution strategies and back-propagation for neural network training. Neurocomputing 42 1 (2002) 87-117
40. Marquardt D.W. An algorithm for least-squares estimation of nonlinear parameters. SIAM Journal on Applied Mathematics 11 2 (1963) 431-441
41. Masters T. Signal and Image Processing with Neural Networks: a C ++ Sourcebook (1994), Wiley, New York
42. Medsker L.R. Hybrid Intelligent Systems (1995), Kluwer Academic Publishers, Norwell, MA, USA
43. Miller B.L., and Goldberg D.E. Genetic algorithms, tournament selection, and the effects of noise. Complex Systems 9 (1995) 193-212
44. Møller M.F. A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks 6 4 (1993) 525-533
45. P. Moscato, On evolution search, optimization, genetic algorithms and martial arts: towards memetic algorithms, Technical Report C3P 826, California Institute of Technology, Pasadena, CA, 1989.
46. Rumelhart D.E., and McClelland J.L. Explorations in the microstructure of cognition. Parallel Distributed Processing: Foundations, vol. 1 (1986), MIT Press, Cambridge, MA, USA
47. Rumelhart D.E., Smolensky P., McClelland J.L., and Hinton G.E. Schemata and sequential thought processes in pdp models. Parallel Distributed Processing: Psychological and Biological Models, vol. 2 (1986), MIT Press, Cambridge, MA 7-57
48. Torres J.J., Mu noz M.A., Marro J., and Garrido P.L. Influence of topology on the performance of a neural network. Neurocomputing 58-60 (2004) 229-234
49. Yao X. Evolving artificial neural networks. Proceedings of the IEEE 87 9 (1999) 1423-1447
50. Yu J., Wang S., and Xi L. Letters: evolving artificial neural networks using an improved pso and dpso. Neurocomputing 71 4-6 (2008) 1054-1060
51. J.P.T. Yusiong, P.C. Naval Jr., Training neural networks using multiobjective particle swarm optimization, Lecture Notes in Computer Science-Advances in Natural Computation vol. 4221 (2006) 879-888.