Let a biogeography-based optimizer train your Multi-Layer Perceptron

Information Sciences

Volumn 269, Issue , 2014, Pages 188-209

  Mirjalili, Seyedali ^a   Mirjalili, Seyed Mohammad ^b   Lewis, Andrew ^a  

^a GRIFFITH UNIVERSITY   (Australia)

^b Zharfa Pajohesh System (ZPS) Co   (Iran)

Author keywords

BBO; Biogeography Based Optimization; Evolutionary algorithm; FNN; Learning neural network; Neural network

Indexed keywords

BACKPROPAGATION; CLASSIFICATION (OF INFORMATION); ECOLOGY; EVOLUTIONARY ALGORITHMS; HEURISTIC ALGORITHMS; LEARNING SYSTEMS; NEURAL NETWORKS; OPTIMIZATION;

BIOGEOGRAPHY-BASED OPTIMIZATIONS; BIOGEOGRAPHYBASED OPTIMIZATIONS (BBO); CLASSIFICATION DATASETS; EXTREME LEARNING MACHINE; FUNCTION APPROXIMATION; LEARNING NEURAL NETWORKS; MULTI LAYER PERCEPTRON; NEURAL NETWORKS (NNS);

MULTILAYER NEURAL NETWORKS;

EID: 84897064334     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2014.01.038     Document Type: Article

References (81)

1. V. Abedifar, M. Eshghi, S. Mirjalili, S.M. Mirjalili, An optimized virtual network mapping using PSO in cloud computing, in: Electrical Engineering (ICEE), 2013 21st Iranian Conference on, 14-16 May, 2013, pp.1-6, http://dx.doi.org/10.1109/IranianCEE.2013.6599723.
2. H. Adeli, and S. Hung An adaptive conjugate gradient learning algorithm for efficient training of neural networks Appl. Math. Comput. 62 1994 81 102
3. P. Auer, H. Burgsteiner, and W. Maass A learning rule for very simple universal approximators consisting of a single layer of perceptrons Neural Networks 21 2008 786 795
4. M. Barakat, D. Lefebvre, M. Khalil, F. Druaux, and O. Mustapha Parameter selection algorithm with self adaptive growing neural network classifier for diagnosis issues Int. J. Mach. Learn. Cybern. 4 3 2013 217 233
5. C. Blake, C.J. Merz, 1998. UCI Repository of Machine Learning Databases. < http://www.ics.uci.edu/~mlearn/MLRepository.html > (accessed).
6. C. Blum, K. Socha, Training feed-forward neural networks with ant colony optimization: an application to pattern classification, in: IEEE, 2005, 6 pp.
7. I. Boussaid, J. Lepagnot, and P. Siarry A survey on optimization metaheuristics Inform. Sci. 237 2013 82 117 ISSN 0020-0255, http://dx.doi.org/ 10.1016/j.ins.2013.02.041 (10.07.13)
8. J. Branke, Evolutionary Algorithms for Neural Network Design and Training, Citeseer, 1995.
9. C. Charalambous, Conjugate gradient algorithm for efficient training of artificial neural networks, in: IET, 1992, pp. 301-310.
10. B.C. Csáji Approximation with Artificial Neural Networks 2001 Faculty of Sciences, Etvs Lornd University Hungary
11. D. Dasgupta, and Z. Michalewicz Evolutionary Algorithms in Engineering Applications 2001 Springer
12. J. Derrac, S. García, D. Molina, and F. Herrera A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms Swarm Evol. Comput. 2011
13. S.E. Fahlman, An Empirical Study of Learning Speed in Back-propagation Networks, Technical report, 1988, < http://repository.cmu.edu/cgi/ viewcontent.cgi?article=2799&context=compsci >.
14. T.L. Fine Feedforward Neural Network Methodology 1999 Springer Verlag
15. S. García, D. Molina, M. Lozano, and F. Herrera A study on the use of non-parametric tests for analyzing the evolutionary algorithms' behaviour: a case study on the CEC'2005 special session on real parameter optimization J. Heuristics 15 2009 617 644
16. M.W. Gardner, and S.R. Dorling Artificial neural networks (the multilayer perceptron) - a review of applications in the atmospheric sciences Atmos. Environ. 32 1998 2627 2636
17. M. Gori, and A. Tesi On the problem of local minima in backpropagation IEEE Trans. Pattern Anal. Mach. Intell. 14 1992 76 86
18. V.G. Gudise, G.K. Venayagamoorthy, Comparison of particle swarm optimization and backpropagation as training algorithms for neural networks, in: IEEE, 2003, pp. 110-117.
19. Z.X. Guo, W.K. Wong, and M. Li Sparsely connected neural network-based time series forecasting Inform. Sci. 193 2012 54 71 ISSN 0020-0255, http://dx.doi.org/10.1016/j.ins.2012.01.011 (15.06.12)
20. W. Guo, L. Wang, and Q. Wu An analysis of the migration rates for biogeography-based optimization Inform. Sci. 254 2014 111 140 ISSN 0020-0255, http://dx.doi.org/10.1016/j.ins.2013.07.018 (01.01.14)
21. M.T. Hagan, and M.B. Menhaj Training feedforward networks with the Marquardt algorithm IEEE Trans. Neural Networks 5 1994 989 993
22. Y.C. Ho, and D.L. Pepyne Simple explanation of the no-free-lunch theorem and its implications J. Optim. Theory Appl. 115 2002 549 570
23. K. Hornik, M. Stinchcombe, and H. White Multilayer feedforward networks are universal approximators Neural Networks 2 1989 359 366
24. G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew Extreme learning machine: theory and applications Neurocomputing 70 2006 489 501
25. G.-B. Huang, D.H. Wang, and Y. Lan Extreme learning machines: a survey Int. J. Mach. Learn. Cybern. 2 2 2011 107 122
26. D.R. Hush, and B.G. Horne Progress in supervised neural networks IEEE Signal Process. Mag. 10 1993 8 39
27. A.S. Ismail Wdaa, Differential Evolution for Neural Network Learning Enhancement, Master, Universiti Teknologi Malaysia (UTM), 2005.
28. R.A. Jacobs Increased rates of convergence through learning rate adaptation Neural Networks 1 1988 295 307
29. Y. Lee, S.H. Oh, and M.W. Kim An analysis of premature saturation in back propagation learning Neural Networks 6 1993 719 728
30. F.H.F. Leung, H. Lam, S. Ling, and P.K.S. Tam Tuning of the structure and parameters of a neural network using an improved genetic algorithm IEEE Trans. Neural Networks 14 2003 79 88
31. H. Ma, D. Simon, M. Fei, and Z. Xie Variations of biogeography-based optimization and Markov analysis Inform. Sci. 220 2013 492 506 ISSN 0020-0255, http://dx.doi.org/10.1016/j.ins.2012.07.007 (20.01.13)
32. G. Magoulas, M. Vrahatis, and G. Androulakis On the alleviation of the problem of local minima in back-propagation Nonlinear Anal. 30 1997 4545 4550
33. O.L. Mangasarian, and W.H. Wolberg Cancer Diagnosis via Linear Programming 1990 University of Wisconsin-Madison, Computer Sciences Department
34. W.S. McCulloch, and W. Pitts A logical calculus of the ideas immanent in nervous activity Bull. Math. Biol. 5 1943 115 133
35. P. Patricia Melin, D. Sanchez, and O. Castillo Genetic optimization of modular neural networks with fuzzy response integration for human recognition Inform. Sci. 197 2012 1 19 ISSN 0020-0255, http://dx.doi.org/10.1016/j.ins.2012. 02.027 (15.08.12)
36. R. Mendes, P. Cortez, M. Rocha, J. Neves, Particle swarms for feedforward neural network training, in: IJCNN'02. Proceedings of the 2002 International Joint Conference on Neural Networks, 2002, IEEE, pp. 1895-1899.
37. S. Mirjalili, Hybrid Particle Swarm Optimization and Gravitational Search Algorithm for Multilayer Perceptron Learning. Master, Universiti Teknologi Malaysia (UTM), 2011.
38. S. Mirjalili, and A. Lewis S-shaped versus V-shaped transfer functions for binary particle swarm optimization Swarm Evol. Comput. 9 2013 1 14 (ISSN 2210-6502, http://dx.doi.org/10.1016/j.swevo.2012.09.002 (April 2013))
39. S. Mirjalili, S.Z.M. Hashim, A new hybrid PSOGSA algorithm for function optimization, in: 2010 International Conference on Computer and Information Application (ICCIA), 3-5 December 2010, pp. 374-377.
40. S. Mirjalili, S.M. Mirjalili, and A. Lewis Grey wolf optimizer Adv. Eng. Softw. 69 2014 46 61 10.1016/j.advengsoft.2013.12.007
41. S. Mirjalili, S.M. Mirjalili, X.-S. Yang, Binary bat algorithm, Neural Comput. Appl. (2013), in press, http://dx.doi.org/10.1007/s00521-013-1525-5.
42. S. Mirjalili, S.Z. Mohd Hashim, and H. Moradian Sardroudi Training feedforward neural networks using hybrid particle swarm optimization and gravitational search algorithm Appl. Math. Comput. 218 22 2012 11125 11137
43. S.M. Mirjalili, K. Abedi, S. Mirjalili, Light property and optical buffer performance enhancement using particle swarm optimization in oblique Ring-Shape-Hole photonic crystal waveguide, in: Photonics Global Conference (PGC), 2012, pp. 1-4.
44. S.M. Mirjalili, K. Abedi, and S. Mirjalili Optical buffer performance enhancement using particle swarm optimization in ring-shape-hole photonic crystal waveguide Optik - Int. J. Light Electron Opt. 124 23 2013 5989 5993 ISSN 0030-4026, http://dx.doi.org/10.1016/j.ijleo.2013.04.114
45. S.M. Mirjalili, S. Mirjalili, Oval-Shaped-Hole photonic crystal waveguide design by MoMIR framework, Photonics Technol. Lett. IEEE (2014), in press, http://dx.doi.org/10.1109/LPT.2014.2302478.
46. S.M. Mirjalili, S. Mirjalili, and A. Lewis A novel multi-objective optimization framework for designing photonic crystal waveguides Photonics Technol. Lett. IEEE 26 2014 146 149 10.1109/LPT.2013.2290318
47. S.M. Mirjalili, S. Mirjalili, A. Lewis, K. Abedi, A tri-objective particle swarm optimizer for designing line defect photonic crystal waveguides, Photon Nanostruct. Fundam. Appl. (2013), http://dx.doi.org/10.1016/j.photonics. 2013.11.001.
48. S. Mizuta, T. Sato, D. Lao, M. Ikeda, and T. Shimizu Structure design of neural networks using genetic algorithms Complex Syst. 13 2001 161 176
49. S. Ng, C. Cheung, S. Leung, A. Luk, Fast convergence for backpropagation network with magnified gradient function, in: IEEE, vol. 3, 2003, pp. 1903-1908.
50. E. Oja Unsupervised learning in neural computation Theoret. Comput. Sci. 287 2002 187 207
51. M. Ovreiu, D. Simon, Biogeography-based optimization of neuro-fuzzy system parameters for diagnosis of cardiac disease, in: Genetic and Evolutionary Computation Conference, 2010, pp. 1235-1242.
52. I.C. Parmee Evolutionary and Adaptive Computing in Engineering Design 2001 Springer Verlag
53. R.D. Reed, and R.J. Marks Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks 1998 MIT Press
54. F. Rosenblatt The Perceptron, A Perceiving and Recognizing Automaton Project Para 1957 Cornell Aeronautical Laboratory
55. S. Saremi, S.M. Mirjalili, and S. Mirjalili Unit cell topology optimization of line defect photonic crystal waveguide Procedia Technol. 12 2014 174 179 ISSN 2212-0173, http://dx.doi.org/10.1016/j.protcy.2013.12.472
56. S. Saremi, S.M. Mirjalili, and S. Mirjalili Chaotic krill herd optimization algorithm Procedia Technol. 12 2014 180 185 ISSN 2212-0173, http://dx.doi.org/10.1016/j.protcy.2013.12.473
57. U. Seiffert, Multiple Layer Perceptron Training using Genetic Algorithms, Citeseer, 2001, pp. 159-164.
58. T.J. Sejnowski Unsupervised Learning: Foundations of Neural Computation 1999 The MIT Press
59. D. Silva, L. Pacifico, T. Ludermir, An evolutionary extreme learning machine based on group search optimization, in: 2011 IEEE Congress on Evolutionary Computation (CEC), 2011, pp. 574-580.
60. D. Simon Biogeography-based optimization IEEE Trans. Evol. Comput. 12 2008 702 713
61. D. Simon, R. Rarick, M. Ergezer, and D. Du Analytical and numerical comparisons of biogeography-based optimization and genetic algorithms Inform. Sci. 181 7 2011 1224 1248 ISSN 0020-0255, http://dx.doi.org/10.1016/j.ins.2010. 12.006 (01.04.11)
62. UCI Machine Learning Repository. < http://archive.ics.uci.edu/ml/ datasets.html > (accessed 2012).
63. A. van Ooyen, and B. Nienhuis Improving the convergence of the back-propagation algorithm Neural Networks 5 1992 465 471
64. T.P. Vogl, J. Mangis, A. Rigler, W. Zink, and D. Alkon Accelerating the convergence of the back-propagation method Biol. Cybern. 59 1988 257 263
65. G. Wang, L. Guo, H. Wang, H. Duan, L. Liu, J. Li, Incorporating mutation scheme into krill herd algorithm for global numerical optimization, Neural Comput. Appl. (2012), http://dx.doi.org/10.1007/s00521-012-1304-8.
66. G.-G. Wang, A.H. Gandomi, and A.H. Alavi A chaotic particle-swarm krill herd algorithm for global numerical optimization Kybernetes 42 6 2013 962 978
67. G.-G. Wang, A.H. Gandomi, A.H. Alavi, An effective krill herd algorithm with migration operator in biogeography-based optimization, Appl. Math. Model. (2013), in press, http://dx.doi.org/10.1016/j.apm.2013.10.052.
68. G.-G. Wang, A.H. Gandomi, and A.H. Alavi Stud krill herd algorithm Neurocomputing 128 2014 363 370 10.1016/j.neucom.2013.08.031
69. G.-G. Wang, A.H. Gandomi, A.H. Alavi, G.-S. Hao, Hybrid krill herd algorithm with differential evolution for global numerical optimization, Neural Comput. Appl. (2013), http://dx.doi.org/10.1007/s00521-013-1485-9.
70. X.Z. Wang, A. Chen, and H. Feng Upper integral network with extreme learning mechanism Neurocomputing 74 16 2011 2520 2525
71. X.Z. Wang, Q.Y. Shao, M. Qing, and J.H. Zhai Architecture selection for networks trained with extreme learning machine using localized generalization error model Neurocomputing 102 2013 3 9
72. M.K. Weir A method for self-determination of adaptive learning rates in back propagation Neural Networks 4 1991 371 379
73. P. Werbos, Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences, Ph.D. thesis, Harvard University, 1974.
74. P.J. Werbos Neurocontrol and supervised learning: An overview and evaluation Handbook Intell. Control 65 1992 89
75. W. Wienholt Minimizing the system error in feedforward neural networks with evolution strategy S. Gielen, B. Kappen, ICANN '93 1993 Springer London 490 493
76. F. Wilcoxon Individual comparisons by ranking methods Biometrics Bull. 1 1945 80 83
77. D.H. Wolpert, and W.G. Acready No free lunch theorems for optimization IEEE Trans. Evol. Comput. 1 1997 67 82
78. X. Yao Evolutionary artificial neural networks Int. J. Neural Syst. 4 1993 203 222
79. J. Yu, S. Wang, and L. Xi Evolving artificial neural networks using an improved PSO and DPSO Neurocomputing 71 2008 1054 1060
80. J.R. Zhang, J. Zhang, T.M. Lok, and M.R. Lyu A hybrid particle swarm optimization - back-propagation algorithm for feedforward neural network training Appl. Math. Comput. 185 2007 1026 1037
81. N. Zhang An online gradient method with momentum for two-layer feedforward neural networks Appl. Math. Comput. 212 2009 488 498