Extended Kalman filter-based Elman networks for industrial time series prediction with GPU acceleration

Neurocomputing

Volumn 118, Issue , 2013, Pages 215-224

  Zhao, Jun ^a   Zhu, Xiaoliang ^a   Wang, Wei ^a   Liu, Ying ^a  

^a DALIAN UNIVERSITY OF TECHNOLOGY   (China)

Author keywords

EKF; Elman network; GPU; Industrial data; Time series prediction

Indexed keywords

EKF; ELMAN NETWORK; GPU; INDUSTRIAL DATUM; TIME SERIES PREDICTION;

COMPUTER GRAPHICS; EXTENDED KALMAN FILTERS; FORECASTING; INDUSTRIAL APPLICATIONS; INDUSTRY; JACOBIAN MATRICES; PROGRAM PROCESSORS; RECURRENT NEURAL NETWORKS; STEELMAKING; TIME SERIES;

COMPUTER GRAPHICS EQUIPMENT;

ACCURACY; ARTICLE; BACK PROPAGATION; ELMAN NETWORK; EXTENDED KALMAN FILTER; GAS FLOW; INTERMETHOD COMPARISON; IRON AND STEEL INDUSTRY; KERNEL METHOD; MACHINE LEARNING; MATHEMATICAL MODEL; NONLINEAR SYSTEM; PREDICTION; PRIORITY JOURNAL; PROCESS OPTIMIZATION; PROCESS TECHNOLOGY; RELIABILITY; SYSTEMATIC ERROR; VALIDATION PROCESS; WHITE NOISE;

EID: 84881246633     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2013.02.031     Document Type: Article

References (37)

1. Shi Z.W., Han M. Ridge regression leaning in ESN for chaotic time series prediction. Control Decision 2007, 22:258-267.
2. Kwon Y.K., Moon B.R. A hybrid neurogenetic approach for stock forecasting. IEEE Trans. Neural Networks 2007, 18:851-864.
3. Bashir Z.A. Applying wavelets to short-term load forecasting using pso-based neural networks. IEEE Trans. Power Syst. 2009, 24:20-27.
4. Yu L., Lai K.K., Wang S.Y. Multistage RBF neural network ensemble learning for exchange rates forecasting. Neurocomputing 2008, 71:3295-3302.
5. Sheng C.Y., Zhao J., Liu Y., Wang W. Prediction for noisy nonlinear time series by echo state network based on dual estimation. Neurocomputing 2012, 82:186-195.
6. Zhao J., Liu Q.L., Wang W., Pedrycz W., Cong L.Q. Hybrid neural prediction and optimized adjustment for coke oven gas system in steel industry. IEEE Trans. Neural Networks Learn. Syst. 2012, 23:439-450.
7. Zhang C., Wu W., Chen X.H., Xiong Y. Convergence of BP algorithm for product unit neural networks with exponential weights. Neurocomputing 2008, 72:513-520.
8. H. Jaeger, Tutorial on training recurrent neural networks, covering BPTT, RTRL, EKF and "Echo State Network" approach, Technical Report GMD Report 159, German National Research Center for Information Technology (2002).
9. Sedki A., Ouazar D., Mazoudi E.E. Evolving neural network using real coded genetic algorithm for daily rainfall-runoff forecasting. Expert Syst. Appl. 2009, 36:4523-4527.
10. Wang Y.N., Han M. Multivariate time series online predictor with Kalman filter trained reservoir. Acta Autom. Sin. 2010, 36:169-173.
11. Lukoševičius M., Jaeger H. Reservoir computing approaches to recurrent neural network training. Comput. Sci. Rev. 2009, 3:127-149.
12. Skowronski M.D., Harris J.G. Automatic speech recognition using a predictive echo state network classifier. Neural Networks 2007, 20:414-423.
13. Li D., Han M., Wang J. Chaotic time series prediction based on a novel robust echo state network. IEEE Trans. Neural Networks Learn. Syst. 2012, 23:787-799.
14. Yang H., Ni J. Dynamic neural network modeling for nonlinear, nonstationary machine tool thermally induced error,. Int. J. Mach. Tools Manuf. 2005, 45:455-465.
15. Mirikitani D.T., Nikolaev N. Recursive Bayesian recurrent neural networks for time-series modeling. IEEE Trans. Neural Networks 2010, 21:262-274.
16. Leclercq E., Drucux F., Lefebvre D., Zerkaoui S. Autonomous leaning algorithm for fully connected recurrent networks. Neurocomputing 2005, 63:25-44.
17. Choi J., Lima A.C., Haykin S. Kalman filter-trained recurrent neural equalizers for time-varying channels. IEEE Trans. Commun. 2005, 53:472-480.
18. Welch G., Bishop G. An introduction to the Kalman filter, Technical Report TR 95-041 1995, University of North Carolina, Department of Computer Science.
19. Haykin S., Puskorius G.V., Feldkamp L.A., Patel G.S., Becker S., Racine R., Wan E.A., Nelson A.T., Rowels S.T., Ghahramani Z. Kalman Filtering and Neural Networks 2002, John Wiley and Sons, NY.
20. S. Singhal, L. Wu, Training feed-forward networks with the extended Kalman filter algorithm, in: Proceedings of the 1989 International Conference on Acoustics, Speech, and Signal Processing (1989) 2, 1187-1190.
21. Wang X.Y., Huang Y. Convergence study in extended Kalman filter-based training of recurrent neural networks. IEEE Trans. Neural Networks 2011, 22:588-600.
22. Sum J., Chan L., Leung C., Young G. Extended Kalman filter-based pruning method for recurrent neural networks,. Neural Comput. 1998, 10:1481-1506.
23. J. Ma, J.F. Ten, Predict chaotic time-series using unscented Kalman filter, in: Proceedings of the International Conference on Machine Learning and Cybernetics (2004) 687-690.
24. Heeswijk M.V., Miche Y., Oja E., Lendasse A. GPU-accelerated and parallelized ELM ensembles for large-scale regression. Neurocomputing 2011, 74:2430-2437.
25. Scanzio S., Cumani S., Gemello R., Mana F., Laface P. Parallel implementation of artificial neural network training for speech recognition. Pattern Recognition Lett. 2010, 31:1302-1309.
26. NVIDIA, NVIDIA CUDA compute unified device architecture programming guide, V. 1.0 (2008).
27. Lowe E.W., Butkiewicz M., Woetzel N., Meiler J. GPU-accelerated machine learning techniques enable QSAR modeling of large HTS data. IEEE Symp. Comput. Intell. Bioinf. Comput. Biol. 2012, 314-320.
28. Lotrič U., Dobnikar A. Parallel implementations of recurrent neural network learning. Adaptive Nat. Comput. Algorithms Lect. Comput. Sci. 2009, 5495:99-108.
29. Hong W.G., Feng Q., Yan C.L., Wen L.D., Lu W. Identification and control nonlinear systems by a dissimilation particle swarm optimization-based Elman neural network. Nonlinear Anal. Real World Appl. 2008, 9:1345-1360.
30. Li X., Chen Z.Q., Yuan Z.Z. Nonlinear stable adaptive control based upon Elman networks. Appl. Math. J. Chin. Univ. Ser. B 2000, 15:332-340.
31. Liou C.Y., Huang J.C., Yang W.C. Modeling word perception using the Elman network. Neurocomputing 2008, 71:3150-3157.
32. Köker R. Reliability-based approach to the inverse kinematics solution of robots using Elman's networks. Eng. Appl. Artif. Intell. 2005, 18:685-693.
33. Lindholm E., Nickolls J., Oberman S., Montrym J. NVIDIA Tesla: a unified graphics and computing architecture. IEEE Micro 2008, 28:39-55.
34. S. Lahabar, P.J. Narayanan, Singular value decomposition on GPU using CUDA, in: Proceedings of the IEEE International Symposium on Parallel and Distributed Processing, 2009.
35. Guo Z.W., Bai G.C. Application of least squares support vector machine for regression to reliability analysis. Chin. J. Aeronaut. 2009, 22:160-166.
36. Liang J.L., Wang Z.D., Liu X.H. Distributed state estimation for discrete-time sensor networks with randomly varying nonlinearities and missing measurements. IEEE Trans. Neural Networks 2011, 22:486-496.
37. Wang Z.D., Wang Y., Liu Y.R. Global synchronization for discrete-time stochastic complex networks with randomly occurred nonlinearities and mixed time delays. IEEE Trans. Neural Networks 2010, 21:11-25.