Online learning with kernels

IEEE Transactions on Signal Processing

Volumn 52, Issue 8, 2004, Pages 2165-2176

  Kivinen, Jyrki ^a   Smola, Alexander J ^a   Williamson, Robert C ^a  

^a AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL METHODS; CONVERGENCE OF NUMERICAL METHODS; FUNCTIONS; LEARNING SYSTEMS; NEURAL NETWORKS; OPTIMIZATION; RANDOM PROCESSES; REGRESSION ANALYSIS; THEOREM PROVING;

KERNEL BASED ALGORITHMS; LARGE MARGIN CLASSIFIERS; NOVELTY DETECTION; ONLINE LEARNING; REPRODUCING KERNEL HILBERT SPACES; STOCHASTIC GRADIENT DESCENT; VECTOR MACHINES;

LEARNING ALGORITHMS;

EID: 3543110224     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2004.830991     Document Type: Article

References (35)

1. B. Schölkopf and A. J. Smola, Learning With Kernels. Cambridge, MA: MIT Press, 2001.
2. D. J. Sebald and J. A. Bucklew, "Support vector machine techniques for nonlinear equalization," IEEE Trans. Signal Processing, vol. 48, pp. 3217-3226, Nov. 2000.
3. G. S. Kimeldorf and G. Wahba, "Some results on Tchebycheffian spline functions," J. Math. Anal. Applic., vol. 33, pp. 82-95, 1971.
4. B. Schölkopf, A. Smola, R. C. Williamson, and P. L. Bartlett, "New support vector algorithms," Neural Comput., vol. 12, pp. 1207-1245, 2000.
5. M. Herbster, "Learning additive models online with fast evaluating kernels," in Proc. Fourteenth Annu. Conf. Comput. Learning Theory, vol. 2111, Springer Lecture Notes in Computer Science, D. P. Helmbold and B. Williamson, Eds., 2001, pp. 444-460.
6. S. V. N. Vishwanathan and A. J. Smola, "Fast kernels for string and tree matching," Adv. Neural Inform. Process. Syst., vol. 15, pp. 569-576, 2003.
7. G. Cauwenberghs and T. Poggio, "Incremental and decremental support vector machine learning," in Advances in Neural Information Processing Systems 13, T. K. Leen, T. G. Dietterich, and V. Tresp, Eds. Cambridge, MA: MIT Press, 2001, pp. 409-415.
8. L. Csató and M. Opper, "Sparse representation for Gaussian process models," in Advances in Neural Information Processing Systems 13, T. K. Leen, T. G. Dietterich, and V. Tresp, Eds. Cambridge, MA: MIT Press, 2001, pp. 444-450.
9. C. Gentile, "A new approximate maximal margin classification algorithm," J. Machine Learning Res., vol. 2, pp. 213-242, Dec. 2001.
10. T. Graepel, R. Herbrich, and R. C. Williamson, "From margin to sparsity," in Advances in Neural Information Processing Systems 13, T. K. Leen, T. G. Dietterich, and V. Tresp, Eds. Cambridge, MA: MIT Press, 2001, pp. 210-216.
11. Y. Li and P. M. Long, "The relaxed online maximum margin algorithm," Machine Learning, vol. 46, no. 1, pp. 361-387, Jan. 2002.
12. M. Herbster and M. Warmuth, "Tracking the best linear predictor," J. Machine Learning Res., vol. 1, pp. 281-309, 2001.
13. P. Auer and M. Warmuth, "Tracking the best disjunction," Machine Learning J., vol. 32, no. 2, pp. 127-150, 1998.
14. J. Kivinen, A. J. Smola, and R. C. Williamson, "Large margin classification for moving targets," in Proc. 13th Int. Conf. Algorithmic Learning Theory, N. Cesa-Bianchi, M. Numao, and R. Reischuk, Eds., Berlin, Germany, Nov. 2002, pp. 113-127.
15. C. Mesterharm, "Tracking linear-threshold concepts with Winnow," in Proc. 15th Annu. Conf. Comput. Learning Theory, J. Kivinen and B. Sloan, Eds., Berlin, Germany, July 2002, pp. 138-152.
16. N. Littlestone, "Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm," Machine Learning, vol. 2, pp. 285-318, 1988.
17. O. Bousquet and M. K. Warmuth, "Tracking a small set of experts by mixing past posteriors," J. Machine Learning Res., vol. 3, pp. 363-396, Nov. 2002.
18. J. Platt, "Fast training of support vector machines using sequential minimal optimization," in Advances in Kernel Methods - Support Vector Learning, B. Schölkopf, C. J. C. Burges, and A. J. Smola, Eds. Cambridge, MA: MIT Press, 1999, pp. 185-208.
19. M. Vogt, "SMO algorithms for support vector machines without bias term," Technische Univ. Darmstadt, Inst. Automat. Contr., Lab. Contr. Syst. Process Automat., Darmstadt, Germany, 2002.
20. K. P. Bennett and O. L. Mangasarian, "Robust linear programming discrimination of two linearly inseparable sets," Optimization Methods Software, vol. 1, pp. 23-34, 1992.
21. B. Schölkopf, J. Platt, J. Shawe-Taylor, A. J. Smola, and R. C. Williamson, "Estimating the support of a high-dimensional distribution," Neural Comput., vol. 13, no. 7, 2001.
22. P. J. Huber, "Robust statistics: A review," Ann. Statist., vol. 43, pp. 1041-1067, 1972.
23. V. Vapnik, S. Golowich, and A. Smola, "Support vector method for function approximation, regression estimation, and signal processing," in Advances in Neural Information Processing Systems 9, M. C. Mozer, M. I. Jordan, and T. Petsche, Eds. Cambridge, MA: MIT Press, 1997, pp. 281-287.
24. S. Haykin, Adaptive Filter Theory, Second ed. Englewood Cliffs, NJ: Prentice-Hall, 1991.
25. B. Schölkopf, R. Herbrich, and A. J. Smola, "A generalized representer theorem," in Proc. Annu. Conf. Comput. Learning Theory, 2001, pp. 416-426.
26. J. Kivinen, A. J. Smola, and R. C. Williamson, "Online learning with kernels," in Advances in Neural Information Processing Systems 14, T. G. Dietterich, S. Becker, and Z. Ghahramani, Eds. Cambridge, MA: MIT Press, 2002, pp. 785-792.
27. R. Herbrich, Learning Kernel Classifiers: Theory and Algorithms. Cambridge, MA: MIT Press, 2002.
28. J. Friedman, T. Hastie, and R. Tibshirani, "Additive logistic regression: A statistical view of boosting," Dept. Statist., Stanford Univ., Stanford, CA, 1998.
29. A. B. J. Novikoff, "On convergence proofs on perceptrons," in Proc. Symp. Math. Theory Automata, vol. 12, 1962, pp. 615-622.
30. C. Gentile and N. Littlestone, "The robustness of the p-norm algorithms," in Proc. 12th Annu. Conf. Comput. Learning Theory, New York, NY, 1999, pp. 1-11.
31. Y. Freund and R. E. Schapire, "Large margin classification using the perceptron algorithm," Machine Learning, vol. 37, no. 3, pp. 277-296, 1999.
32. P. Auer, N. Cesa-Bianchi, and C. Gentile, "Adaptive and self-confident on-line learning algorithms," J. Comput. Syst. Sci., vol. 64, no. 1, pp. 48-75, Feb. 2002.
33. N. Cesa-Bianchi, P. Long, and M. Warmuth, "Worst-case quadratic loss bounds for on-line prediction of linear functions by gradient descent," IEEE Trans. Neural Networks, vol. 7, pp. 604-619, May 1996.
34. M.K. Warmuth and A. Jagota. Continuous and discrete time non-linear gradient descent: Relative loss bounds and convergence. presented at Fifth Int. Symp. Artif. Intell. Math.. [Online], Available: http://rutcor.rutgers.edu/~amai
35. N. Cesa-Bianchi, A. Conconi, and C. Gentile, "On the generalization ability of on-line learning algorithms," in Advances in Neural Information Processing Systems 14, T.G. Dietterich, S. Becker, and Z. Ghahramani, Eds. Cambridge, MA: MIT Press, 2002 pp. 359-366.