A linear recurrent kernel online learning algorithm with sparse updates

Neural Networks

Volumn 50, Issue , 2014, Pages 142-153

  Fan, Haijin ^a   Song, Qing ^a  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

Hybrid recurrent training; Kernel methods; Linear recurrent; Weight convergence

Indexed keywords

LEARNING ALGORITHMS;

ADAPTIVE LEARNING RATES; CONVERGENCE CONDITIONS; KERNEL ALGORITHMS; KERNEL METHODS; LINEAR RECURRENT; ONLINE LEARNING ALGORITHMS; RECURRENT TRAINING; WEIGHT CONVERGENCE;

ITERATIVE METHODS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; KERNEL METHOD; LEARNING ALGORITHM; LINEAR SYSTEM; MATHEMATICAL ANALYSIS; ONLINE MONITORING; PRIORITY JOURNAL; SIGNAL PROCESSING; SUPPORT VECTOR MACHINE; VELOCITY;

HYBRID RECURRENT TRAINING; KERNEL METHODS; LINEAR RECURRENT; WEIGHT CONVERGENCE;

ALGORITHMS; COMPUTER SIMULATION; HUMANS; LEARNING; LINEAR MODELS; NEURAL NETWORKS (COMPUTER); ONLINE SYSTEMS;

EID: 84888802656     PISSN: 08936080     EISSN: 18792782     Source Type: Journal    
DOI: 10.1016/j.neunet.2013.11.011     Document Type: Article

References (39)

1. Aronszajn N. Theory of reproducing kernels 1951, Harvard University, Cambridge, Mass.
2. Baltersee J., Chambers J. Nonlinear adaptive prediction of speech with a pipelined recurrent neural network. IEEE Transactions on Signal Processing 1998, 46(8):2207-2216.
3. Bhotto M., Antoniou A. Robust set-membership affine-projection adaptive-filtering algorithm. IEEE Transactions on Signal Processing 2012, 60(1):73-81.
4. Billings S.A., Wei H.-L., Balikhin M.A. Generalized multiscale radial basis function networks. Neural Networks 2007, 20(10):1081-1094.
5. Burges C. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 1998, 2(2):121-167.
6. Campbell C. Kernel methods: a survey of current techniques. Neurocomputing 2002, 48(1-4):63-84.
7. Chen B., Zhao S., Zhu P., Príncipe J. Quantized kernel least mean square algorithm. IEEE Transactions on Neural Networks and Learning Systems 2012, 23(1):22-32.
8. Cluett W., Shan S., Fisher D. Robustness analysis of discrete-time adaptive control systems using input-output stability theory: a tutorial. IEE Proceedings D: Control Theory and Applications 1988, 135(2):133-141.
9. Diniz P., Werner S. Set-membership binormalized data-reusing LMS algorithms. IEEE Transactions on Signal Processing 2003, 51(1):124-134.
10. Engel Y., Mannor S., Meir R. The kernel recursive least-squares algorithm. IEEE Transactions on Signal Processing 2004, 52(8):2275-2285.
11. Guo Y., Gao J., Kwan P. Twin kernel embedding. IEEE Transactions on Pattern Analysis and Machine Intelligence 2008, 30(8):1490-1495.
12. Kivinen J., Smola A., Williamson R. Online learning with kernels. IEEE Transactions on Signal Processing 2004, 52(8):2165-2176.
13. Li D., Han M., Wang J. Chaotic time series prediction based on a novel robust echo state network. IEEE Transactions on Neural Networks and Learning Systems 2012, 23(5):787-799.
14. Liu W., Park I., Príncipe J. An information theoretic approach of designing sparse kernel adaptive filters. IEEE Transactions on Neural Networks 2009, 20(12):1950-1961.
15. Liu W., Pokharel P., Príncipe J. The kernel least-mean-square algorithm. IEEE Transactions on Signal Processing 2008, 56(2):543-554.
16. Liu W., Príncipe J. Kernel affine projection algorithms. EURASIP Journal on Advances in Signal Processing 2008.
17. Memisevic R., Sigal L., Fleet D. Shared kernel information embedding for discriminative inference. IEEE Transactions on Pattern Analysis and Machine Intelligence 2012, 34(4):778-790.
18. Mimura, M., Hamada, N., & Furukawa, T. (2002). A recurrent RBF network for non-linear channel with time-varying characteristic. In 2002 IEEE international conf. on acoustics, speech, and signal process, ICASSP, Vol. 1 (pp. I-1081-I-1084).
19. Nagaraj S., Gollamudi S., Kapoor S., Huang Y.-F. Beacon: an adaptive set-membership filtering technique with sparse updates. IEEE Transactions on Signal Processing 1999, 47(11):2928-2941.
20. Ozay N., Sznaier M., Lagoa C., Camps O. A sparsification approach to set membership identification of switched affine systems. IEEE Transactions on Automatic Control 2012, 57(3):634-648.
21. Pearlmutter B.A. Gradient calculations for dynamic recurrent neural networks: a survey. IEEE Transactions on Neural Networks 1995, 6(5):1212-1228.
22. Platt J. A resource-allocating network for function interpolation. Neural Computation 1991, 3(2):213-225.
23. Qian N. On the momentum term in gradient descent learning algorithms. Neural Networks 1999, 12(1):145-151.
24. Qiu W., Chang C., Liu W., Poon P., Hu Y., Lam F., et al. Real-time data-reusing adaptive learning of a radial basis function network for tracking evoked potentials. IEEE Transactions on Biomedical Engineering 2006, 53(2):226-237.
25. Qu, H.-N., Oussar, Y., Dreyfus, G., & Xu, W. (2009). Regularized recurrent least squares support vector machines (pp. 508-511).
26. Rasmussen C. Gaussian processes in machine learning. Lecture notes in computer science 2004, Vol. 3176:63-71. Springer, Berlin, Heidelberg. O. Bousquet, U. von Luxburg, G. Rätsch (Eds.).
27. Richard C., Bermudez J., Honeine P. Online prediction of time series data with kernels. IEEE Transactions on Signal Processing 2009, 57(3):1058-1067.
28. Roy S., Shynk J.J. Analysis of the momentum LMS algorithm. IEEE Transactions on Acoustics, Speech and Signal Processing 1990, 38(12):2088-2098.
29. Schōlkopf B., Herbrich R., Smola A. A generalized representer theorem. Lecture notes in computer science 2001, Vol. 2111:416-426. Springer, Berlin, Heidelberg.
30. Sebald D., Bucklew J. Support vector machine techniques for nonlinear equalization. IEEE Transactions on Signal Processing 2000, 48(11):3217-3226.
31. Sharma R., Sethares W., Bucklew J. Analysis of momentum adaptive filtering algorithms. IEEE Transactions on Signal Processing 1998, 46(5):1430-1434.
32. Slavakis K., Theodoridis S. Sliding window generalized kernel affine projection algorithm using projection mappings. EURASIP Journal on Advances in Signal Processing 2008.
33. Song Q. Robust initialization of a Jordan network with recurrent constrained learning. IEEE Transactions on Neural Networks 2011, 22(12):2460-2473.
34. Song Q., Wu Y., Soh Y. Robust adaptive gradient-descent training algorithm for recurrent neural networks in discrete time domain. IEEE Transactions on Neural Networks 2008, 19(11):1841-1853.
35. Soni R., Gallivan K., Jenkins W. Low-complexity data reusing methods in adaptive filtering. IEEE Transactions on Signal Processing 2004, 52(2):394-405.
36. Sperduti A. On the computational power of recurrent neural networks for structures. Neural Networks 1997, 10(3):395-400.
37. Š Raudys How good are support vector machines?. Neural Networks 2000, 13(1):17-19.
38. Suykens J., Vandewalle J. Recurrent least squares support vector machines. IEEE Transactions on Circuits and Systems. I 2000, 47(7):1109-1114.
39. Xie, J. (2009). Time series prediction based on recurrent LS-SVM with mixed kernel. In Asia-Pacific conf. on information process., APCIP 2009, Vol. 1 (pp. 113-116).