Model-based online learning with kernels

IEEE Transactions on Neural Networks and Learning Systems

Volumn 24, Issue 3, 2013, Pages 356-369

  Li, Guoqi ^a   Wen, Changyun ^b   Li, Zheng Guo ^c   Zhang, Aimin ^d   Yang, Feng ^e   Mao, Kezhi ^b  

^a DATA STORAGE INSTITUTE   (Singapore)

^b NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

^c INSTITUTE FOR INFOCOMM RESEARCH   (Singapore)

^d XI'AN JIAOTONG UNIVERSITY   (China)

^e INSTITUTE OF HIGH PERFORMANCE COMPUTING   (Singapore)

Author keywords

Classification; Kernels; Novelty detection; Online learning; Regression; Reproducing kernel hilbert space

Indexed keywords

KERNELS; NOVELTY DETECTION; ONLINE LEARNING; REGRESSION; REPRODUCING KERNEL HILBERT SPACES;

ALGORITHMS; CLASSIFICATION (OF INFORMATION); CONSTRAINED OPTIMIZATION; E-LEARNING; LEAST SQUARES APPROXIMATIONS; REGRESSION ANALYSIS; SUPPORT VECTOR MACHINES;

ITERATIVE METHODS;

ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; COMPUTER SIMULATION; HUMAN; THEORETICAL MODEL; TRENDS;

ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; HUMANS; MODELS, THEORETICAL; NEURAL NETWORKS (COMPUTER);

EID: 84879125836     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2012.2229293     Document Type: Article

References (34)

1. B. Schölkopf and A. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond. Cambridge, MA: MIT Press, 2002.
2. G. Li, C. Wen, W. Zheng, and Y. Chen, "Identification of a class of nonlinear autoregressive with exogenous inputs models based on Kernel machines," IEEE Trans. Signal Process., vol. 59, no. 5, pp. 2146-2158, May 2011.
3. M. Ramona, G. Richard, and B. David, "Multiclass feature selection with Kernel gram-matrix-based criteria," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 10, pp. 1611-1623, Oct. 2012.
4. J. Kivinen, A. Smola, and R. Williamson, "Online learning with Kernels," IEEE Trans. Signal Process., vol. 100, no. 10, pp. 2165-2176, Oct. 2004.
5. D. J. Sebald and J. A. Bucklew, "Support vector machine techniques for nonlinear equalization," IEEE Trans. Signal Process., vol. 48, no. 11, pp. 3217-3226, Nov. 2000.
6. E. Hu, S. Chen, D. Zhang, and X. Yin, "Semisupervised Kernel matrix learning by Kernel propagation," IEEE Trans. Neural Netw., vol. 21, no. 11, pp. 1831-1841, Nov. 2010.
7. H. Ning, X. Jing, and L. Cheng, "Online identification of nonlinear spatiotemporal systems using Kernel learning approach," IEEE Trans. Neural Netw., vol. 22, no. 9, pp. 1381-1394, Sep. 2011.
8. G. Li and C. Wen, "Identification of Wiener systems with clipped observations," IEEE Trans. Signal Process., vol. 60, no. 7, pp. 3845-3852, Jul. 2012.
9. S. Liwicki, S. Zafeiriou, G. Tzimiropoulos, and M. Pantic, "Efficient online subspace learning with an indefinite Kernel for visual tracking and recognition," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 10, pp. 1624-1636, Oct. 2012.
10. S. I. Gallant, "Perceptron-based learning algorithms," IEEE Trans. Neural Netw., vol. 1, no. 2, pp. 179-191, Feb. 1990.
11. M. Fernandez-Delgado, J. Ribeiro, E. Cernadas, and S. B. Ameneiro, "Direct parallel perceptrons (DPPs): Fast analytical calculation of the parallel perceptrons weights with margin control for classification tasks," IEEE Trans. Neural Netw., vol. 22, no. 11, pp. 1837-1848, Nov. 2011.
12. Y. Freund and R. E. Schapire, "Large margin classification using the perceptron algorithm," Mach. Learn., vol. 37, no. 3, pp. 277-296, 1999. (Pubitemid 32210619)
13. O. Dekel, S. S. Shwartz, and Y. Singer, "The forgetron: A Kernel-based Perceptron on a budget," SIAM J. Comput., vol. 37, no. 5, pp. 1342-1372, 2008.
14. J. Duchi, E. Hazan, and Y. Singer, "Adaptive subgradient methods for online learning and stochastic optimization," J. Mach. Learn. Res., vol. 12, pp. 2121-2159, Jan. 2011.
15. Y. Engel, S. Mannor, and R. Meir, "The Kernel recursive least squares algorithm," IEEE Trans. Sig. Process., vol. 52, no. 8, pp. 2275-2185, Aug. 2004.
16. W. He, "Limited stochastic meta-descent for Kernel-based online learning," Neural Comput., vol. 21, no. 9, pp. 2667-2686, 2009.
17. F. Orabona, J. Keshet, and B. Caputo, "Bounded Kernel-based online learning," J. Mach. Learn. Res., vol. 10, pp. 2643-2666, Nov. 2009.
18. P. Bouboulis, K. Slavakis, and S. Theodoridis, "Adaptive learning in complex reproducing Kernel Hilbert spaces employing Wirtinger's subgradients," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 3, pp. 425-438, Mar. 2012.
19. M. Davy, F. Desobry, A. Gretton, and C. Doncarli, "An online support vector machine for abnormal events detection," Sig. Process., vol. 86, pp. 2009-2025, Sep. 2006. (Pubitemid 43795456)
20. J. Kivinen, M. K. Warmuth, and B. Hassibi, "The p-norm generalization of the LMS algorithm for adaptive filtering," IEEE Trans. Sig. Process., vol. 54, no. 5, pp. 1782-1793, May 2006.
21. L. Cheng, M. Gong, D. Schuurmans, and T. Caelli, "Real-time discriminative background subtraction," IEEE Trans. Image Process., vol. 20, no. 10, pp. 1401-1414, Oct. 2011.
22. V. Vapnik, Statistical Learning Theroy. New York: Wiley, 1998.
23. J. C. Platt, "Fast training of support vector machines using sequential minimal optimization, in Advances in Kernel Methods: Support Vector Machines. Cambridge MA: MIT Press, 1998.
24. E. A. D. Oliveira and N. Caticha, "Inference from aging information," IEEE Trans. Neural Netw., vol. 21, no. 6, pp. 1015-1020, Jun. 2010.
25. Y. Dong and G. N. DeSouza, "Adaptive learning of multi-subspace for foreground detection under illuminiation changes," Comput. Vis. Image Understan., vol. 115, no. 1, pp. 31-39, 2011.
26. H. Sun, "Mercer theorem for RKHS on noncompact sets," J. Complex., vol. 21, no. 3, pp. 337-349, 2005. (Pubitemid 40608774)
27. S. Vijayakumar and H. Ogawa, "RKHS-based functional analysis for exact incremental learning," Neurocompting, vol. 29, pp. 85-113, Oct. 1999. (Pubitemid 29440947)
28. S. Boyd and L. Vandenberghe, Convex Optimization. (2004) [Online]. Available: http://www.stanford.edu/∼boyd/cvxbook/
29. S. G. Nash and A. Sofer, Linear and Nonlinear Programming. New York: McGraw-Hill, 1996.
30. B. Scholkopf, R. Herbric, and A. J. Smola, "A generalized representer theory," in Proc. Annu. Conf. Comput. Learn. Theory, 2001, pp. 416-426.
31. G. Li, C. Wen, G. B. Huang, and Y. Chen, "Error tolerance based support vector machine for regression," Neurocompting, vol. 74, no. 5, pp. 771-782, 2011.
32. B. Schölkopf, J. Platt, J. Shawe-Taylor, A. J. Smola, and R. C. Williamson, "Estimating the support of a high-dimensional distribution," Neural Comput., vol. 13, no. 7, pp. 1443-1471, 2001.
33. J. A. K. Suykens and J. Vandewalle, "Least squares support vector machine classifiers," Neural Process. Lett., vol. 9, no. 3, pp. 293-300, 1999.
34. J. Zhong and S. Sclaroff, "Segmenting foreground objects from a dynamic textured background via a robust Kalman filter," in Proc. IEEE Int. Conf. Comput. Vis., Oct. 2003, pp. 44-50.