Evolutionary multi-objective generation of recurrent neural network ensembles for time series prediction

Neurocomputing

Volumn 143, Issue , 2014, Pages 302-311

  Smith, Christopher ^a   Jin, Yaochu ^a  

^a UNIVERSITY OF SURREY   (United Kingdom)

Author keywords

Ensembles; Hybrid multi objective evolutionary algorithms; Recurrent neural networks; Selection; Time series prediction

Indexed keywords

EVOLUTIONARY ALGORITHMS; FORECASTING; RECURRENT NEURAL NETWORKS;

ENSEMBLES; GENERALIZATION PERFORMANCE; INDIVIDUAL MODELING; INDIVIDUAL PREDICTION; MULTI OBJECTIVE EVOLUTIONARY ALGORITHMS; SELECTION; TIME SERIES PREDICTION; TRAINING ALGORITHMS;

TIME SERIES;

ARTICLE; ARTIFICIAL NEURAL NETWORK; COMPUTATIONAL FLUID DYNAMICS; COMPUTER ANALYSIS; COMPUTER PREDICTION; CONTROLLED STUDY; EVOLUTIONARY ALGORITHM; GENETIC ALGORITHM; INTERMETHOD COMPARISON; MATHEMATICAL ANALYSIS; MATHEMATICAL MODEL; PRIORITY JOURNAL; PROCESS DEVELOPMENT; PROCESS OPTIMIZATION; TIME SERIES ANALYSIS; VALIDATION PROCESS;

EID: 84904797921     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2014.05.062     Document Type: Article

References (55)

1. Gheyas I.A., Smith L.S. A novel neural network ensemble architecture for time series forecasting. Neurocomputing 2011, 74(18):3855-3864.
2. Hippert H.S., Pedreira C.E., Souza R.C. Neural networks for short-term load forecasting. a review and evaluation. IEEE Trans. Power Syst. 2001, 16(1):44-55.
3. SIDC-team, The International Sunspot Number, Monthly Report on the International Sunspot Number, online catalogue, 〈〉. http://www.sidc.be/sunspot-data/.
4. Kim D., Kim C. Forecasting time series with genetic fuzzy predictor ensemble. IEEE Trans. Fuzzy Syst. 1997, 5(4):523-535.
5. Hassan M.R., Nath B., Kirley M., Kamruzzaman J. A hybrid of multiobjective evolutionary algorithm and HMM-fuzzy model for time series prediction. Neurocomputing 2012, 81:1-11.
6. G. Huang, L. Wang, Hybrid neural network models for hydrologic time series forecasting based on genetic algorithm, in: 2011 Fourth International Joint Conference on Computational Sciences and Optimization (CSO), 2011, pp. 1347-1350.
7. Yao X., Islam M.M. Evolving artificial neural network ensembles. IEEE Comput. Intell. Mag. 2008, 3(1):31-42.
8. Hansen L.K., Salamon P. Neural network ensembles. IEEE Trans. Pattern Anal. Mach. Intell. 1990, 12(10):993-1001.
9. Yao X., Liu Y. Making use of population information in evolutionary artificial neural networks. IEEE Trans. Syst. Man Cybern. Part B: Cybern. 1998, 28(3):417-425.
10. Chandra A., Yao X. DIVACE: Diverse and accurate ensemble learning algorithm. Intelligent Data Engineering and Automated Learning, IDEAL 2004, Lecture Notes in Computer Science 2004, vol. 3177:619-625. Springer, Berlin, Heidelberg.
11. Y. Jin, T. Okabe, B. Sendhoff, Neural network regularization and ensembling using multi-objective evolutionary algorithms, in: IEEE Congress on Evolutionary Computation, vol. 1, 2004, pp. 1-8.
12. D.W. Opitz, J.W. Shavlik, Generating accurate and diverse members of a neural-network ensemble, in: Advances in Neural Information Processing Systems, MIT Press, Massachusetts, 1996, pp. 535-541.
13. H. Abbass, Pareto neuro-evolution: constructing ensemble of neural networks using multi-objective optimization, in: IEEE Congress on Evolutionary Computation, vol. 3, 2003, pp. 2074-2080.
14. Jin Y., Sendhoff B. Pareto-based multiobjective machine learning. an overview and case studies. IEEE Trans. Syst. Man Cybern. Part C: Appl. Rev. 2008, 38:397-445.
15. A. Krogh, J. Vedelsby, Neural network ensembles, cross validation, and active learning, in: Advances in Neural Information Processing Systems, MIT Press, Massachusetts, 1995, pp. 231-238.
16. Haykin S. Neural Networks and Learning Machines 2008, Prentice Hall, New Jersey. 3rd Edition.
17. Chandra R., Zhang M. Cooperative coevolution of Elman recurrent neural networks for chaotic time series prediction. Neurocomputing 2012, 86:116-123.
18. Wang X., Ma L., Wang B., Wang T. A hybrid optimization-based recurrent neural network for real-time data prediction. Neurocomputing 2013, 120:547-559.
19. Ardalani-Farsa M., Zolfaghari S. Chaotic time series prediction with residual analysis method using hybrid elman-narx neural networks. Neurocomputing 2010, 73(13-15):2540-2553.
20. Q. Ma, Q. Zheng, H. Peng, T. Zhong, L. Xu, Chaotic time series prediction based on evolving recurrent neural networks, in: International Conference on Machine Learning and Cybernetics, vol. 6, 2007, pp. 3496-3500.
21. Y.-C. Lin, Y.-C. Lin, K.-L. Su, Evolutionary neural networks for time series prediction, in: International Conference on Genetic and Evolutionary Computing (ICGEC), 2010, pp. 219-223.
22. Zemouri R., Racoceanu D., Zerhouni N. Recurrent radial basis function network for time-series prediction. Eng. Appl. Artif. Intell. 2003, 16:453-463.
23. Lukoseviciute K., Ragulskis M. Evolutionary algorithms for the selection of time lags for time series forecasting by fuzzy inference systems. Neurocomputing 2010, 73(10-12):2077-2088.
24. D. Brezak, T. Bacek, D. Majetic, J. Kasac, B. Novakovic, A comparison of feed-forward and recurrent neural networks in time series forecasting, in: 2012 IEEE Conference on Computational Intelligence for Financial Engineering Economics (CIFEr), 2012, pp. 1-6.
25. A.A. Ferreira, T.B. Ludermir, R.R. de Aquino, Comparing recurrent networks for time-series forecasting, in: The 2012 International Joint Conference on Neural Networks (IJCNN), 2012, pp. 1-8.
26. Pearlmutter B.A. Gradient calculations for dynamic recurrent neural networks. a survey. IEEE Trans. Neural Netw. 1995, 6(5):1212-1228.
27. Han M., Xi J., Xu S., Yin F.-L. Prediction of chaotic time series based on the recurrent predictor neural network. IEEE Trans. Signal Process. 2004, 52(12):3409-3416.
28. Blanco A., Delgado M.A., Pegalajar M.C. A real-coded genetic algorithm for training recurrent neural networks. Neural Netw. 2001, 14(1):93-105.
29. M.P. Cuellar, M.A. Delgado, M.C. Pegalajar, Topology optimization and training of recurrent neural networks with Pareto-based multi-objective algorithms: a experimental study, in: Computational and Ambient Intelligence, Lecture Notes in Computer Science, vol. 4507, Springer, 2007, pp. 359-366.
30. Chihi H., Arous N. Recurrent neural networks design by means of multi-objective genetic algorithm. Int. J. Comput. Sci. 2011, 8(1):296-302.
31. Delgado M.A., Cuellar M.P., Pegalajar M.C. Multiobjective hybrid optimization and training of recurrent neural networks. IEEE Trans. Syst. Man. Cybern. Part B: Cybern. 2008, 38(2):381-403.
32. J.H. Ang, C.K. Goh, E.J. Teoh, A.A. Mamun, Multi-objective evolutionary recurrent neural networks for system identification, in: IEEE Congress on Evolutionary Computation, 2007, pp. 1586-1592.
33. Subrahmanya N., Shin Y.C. Constructive training of recurrent neural networks using hybrid optimization. Neurocomputing 2010, 73(13-15):2624-2631.
34. Ku K.W., Mak M.-W., Siu W.-C. A study of the lamarckian evolution of recurrent neural networks. IEEE Trans. Evolut. 2000, 4(1):31-42.
35. Yao X. Evolving artificial neural networks. Proc. IEEE 1999, 87(9):1423-1447.
36. Zhou Z.-H. Ensemble Methods: Foundations and Algorithms. Chapman & Hall/CRC Data Mining and Knowledge Discovery Series 2012, Taylor & Francis, Boca Raton, FL.
37. Brown G., Wyatt J., Harris R., Yao X. Diversity creation methods. a survey and categorisation. J. Inf. Fusion 2005, 6(1):1-28.
38. Coello C.A.C., Lamont G.B., van Veldhuizen D.A. Evolutionary Algorithms for Solving Multi-Objective Problems 2007, Springer, New York. 2nd Edition.
39. Zhou Z.-H., Wu J., Tang W. Ensembling neural networks. many could be better than all. Artif. Intell. 2002, 137:239-263.
40. D. Ruta, B. Gabrys, Neural network ensembles for time series prediction, in: International Joint Conference on Neural Networks, 2007, pp. 1204-1209.
41. D.K. Barrow, S.F. Crone, N. Kourentzes, An evaluation of neural network ensembles and model selection for time series prediction, in: International Joint Conference on Neural Networks, 2010, pp. 1-8.
42. H. Yang, J. Shi, A three-stage SVM ensemble algorithm for chaotic time series prediction, in: 2010 Second International Workshop on Education Technology and Computer Science (ETCS), vol. 3, 2010, pp. 345-347.
43. Salomon R. Evolutionary algorithms and gradient search. similarities and differences. IEEE Trans. Evolut. Comput. 1998, 2(2):45-55.
44. Deb K. Multi-Objective Optimization using Evolutionary Algorithms 2001, John Wiley and Sons, Ltd, Chichester.
45. Husken M., Stagge P. Recurrent neural networks for time series classification. Neurocomputing 2003, 50:223-235.
46. Igel C., Heidrich-Meisner V., Glasmachers T. Shark. J. Mach. Learn. Res. 2008, 9:993-996.
47. Das I. On characterizing the "knee" of the Pareto curve based on normal-boundary intersection. Struct. Optim. 1999, 18(2-3):107-115.
48. L. Rachmawati, D. Srinivasan, A multi-objective evolutionary algorithm with weighted-sum niching for convergence on knee regions, in: Genetic and Evolutionary Computation Conference, ACM, New York, NY, USA, 2006, pp. 749-750.
49. Jin Y., Gruna R., Sendhoff B. Pareto analysis of evolutionary and learning systems. Front. Comput. Sci. China 2009, 3(1):4-17.
50. E. Sanchez, S. Pintos, N.V. Queipo, Toward an optimal ensemble of kernel-based approximations with engineering applications, in: International Joint Conference on Neural Networks, 2006, pp. 2152-2158.
51. Y. Jin, B. Sendhoff, Reducing fitness evaluations using clustering techniques and neural networks ensembles, in: Proceedings of the Genetic and Evolutionary Computation Conference, Springer, 2004, pp. 688-699.
52. Mackey M.C., Glass L. Oscillation and chaos in physiological control systems. Science 1977, 1977(4300):287-289.
53. Deb K., Pratap A., Agarwal S., Meyarivan T. A fast and elitist multiobjective genetic algorithm. NSGA-II. IEEE Trans. Evolut. Comput. 2002, 6(April (2)):182-197.
54. Igel C., Husken M. Empirical evaluation of the improved Rprop learning algorithms. Neurocomputing 2003, 50:105-123.
55. Jin Y. Surrogate-assisted evolutionary computation. recent advances and future challenges. Swarm Evolut. Comput. 2011, 1(2):61-70.