Incremental support vector learning: Analysis, implementation and applications

Journal of Machine Learning Research

Volumn 7, Issue , 2006, Pages 1909-1936

  Laskov, Pavel ^a   Gehl, Christian ^a   Krüger, Stefan ^a   Müller, Klaus Robert ^a,b  

^a FRAUNHOFER FIRST   (Germany)

^b UNIVERSITY OF POTSDAM   (Germany)

Author keywords

Drug discovery; Incremental SVM; Intrusion detection; Online learning

Indexed keywords

ALGORITHMS; COMPUTATIONAL COMPLEXITY; CONDITION MONITORING; CONVERGENCE OF NUMERICAL METHODS; ONLINE SYSTEMS;

ALGORITHMIC COMPLEXITY; DRUG DISCOVERY; INCREMENTAL SVM; INTRUSION DETECTION; ONLINE LEARNING; ONLINE MONITORING; SUPPORT VECTOR MACHINES (SVM);

LEARNING SYSTEMS;

EID: 33745777639     PISSN: 15337928     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (38)

1. D. Angluin. Queries and concept learning. Machine Learning, 2:319-342, 1988.
2. C. M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, 1995.
3. B.. Blankertz, G. Dornhege, C. Schäfer, R. Krepki, J. Kohlmorgen, K.-R. Müller, V. Kunzmann, F. Losch, and G. Curio. BCI bit rates and error detection for fast-pace motor commands based on single-trial EEG analysis. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 11:127-131, 2003.
4. A. Bordes, S. Ertekin, J. Wesdon, and L. Bottou. Fast kernel classifiers for online and active learning. Journal of Machine Learning Research, 6:1579-1619, 2005.
5. G. Cauwenberghs and T. Poggio. Incremental and decremental support vector machine learning. In T. K. Leen, T. G. Dietterich, and V. Tresp, editors, Advances in Neural Information Processing Systems, volume 13, pages 409-415. MIT Press, 2001.
6. S. Chakrabarti. Mining the Web: Discovering Knowledge from Hypertext Data. Morgan-Kaufmann, 2002. ISBN 1-55860-754-4.
7. C.-C. Chang and C.-J. Lin. Libsvm: Introduction and benchmarks. Technical report, Department of Computer Science and Information Engineering, National Taiwan University, Taipei, 2000.
8. R. Collobert and S. Bengio. SVM Torch: Support vector machines for large-scale regression problems. Journal of Machine Learning Research, 1:143-160, 2001.
9. C. Cortes and M. Mohri. AUC optimization vs. error rate minimization. In Proc. NIPS'2003, 2004.
10. E. Eskin, A. Arnold, M. Prerau, L. Portnoy, and S. Stolfo. Applications of Data Mining in Computer Security, chapter A geometric framework for unsupervised anomaly detection: detecting intrusions in unlabeled data. Kluwer, 2002.
11. G. H. Golub and C. F. van Loan. Matrix Computations. John Hopkins University Press, Baltimore, London, 3rd edition, 1996.
12. T. Joachims. Making large-scale SVM learning practical. In B. Schölkopf, C. J. C. Burges, and A. J. Smola, editors, Advances in Kernel Methods -Support Vector Learning, pages 169-184, Cambridge, MA, 1999. MIT Press.
13. J. Kivinen, A. J. Smola, and R. C. Williamson. Online learning with kernels. In T. G. Diettrich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Inf. Proc. Systems (NIPS 01), pages 785-792, 2001.
14. F. Klee and G. J. Minty. How good is the simplex algorithm? In O. Sisha, editor, Inequalities III, pages 159-175. Academic Press, 1972.
15. P. Laskov. Feasible direction decomposition algorithms for training support vector machines. Machine Learning, 46:315-349, 2002.
16. P. Laskov, C. Schäfer, and I. Kotenko. Intrusion detection in unlabeled data with quarter-sphere support vector machines. In Proc. DIMVA, pages 71-82, 2004.
17. Y. LeCun, L. Bottou, G. B. Orr, and K.-R. Müller. Efficient backprop. In G. Orr and K.-R. Müller, editors, Neural Networks: Tricks of the Trade, volume 1524, pages 9-53, Heidelberg, New York, 1998. Springer LNCS.
18. N. Littlestone, P. M. Long, and M. K. Warmuth. On-line learning of linear functions. Technical Report CRL-91-29, University of California at Santa Cruz, October 1991.
19. J. Ma and S. Perkins. Time-series novelty detection using one-class Support Vector Machines. In IJCNN, 2003. to appear.
20. J. Ma, J. Theiler, and S. Perkins. Accurate online support vector regression. http://nis-www.lanl.gov/~jt/Papers/aosvr.pdf, 2003.
21. M. Martin. On-line Support Vector Machines for function approximation. Technical report, Universitat Politècnica de Catalunya, Departement de Llengatges i Sistemes Informàtics, 2002.
22. N. Murata. A statistical study on the asymptotic theory of learning. PhD thesis, University of Tokyo (In Japanese), 1992.
23. N. Murata, M. Kawanabe, A. Ziehe, K.-R. Müller, and S.-I. Amari. On-line learning in changing environments with applications in supervised and unsupervised learning. Neural Networks, 15 (4-6):743-760, 2002.
24. N. Murata, K.-R. Müller, A. Ziehe, and S. i. Amari. Adaptive on-line learning in changing environments. In M. C. Mozer, M. I. Jordan, and T. Petsche, editors, Advances in Neural Information Processing Systems, volume 9, page 599. The MIT Press, 1997.
25. G. Orr and K.-R. Müller, editors. Neural Networks: Tricks of the Trade, volume 1524. Springer LNCS, 1998.
26. J. Platt. Fast training of support vector machines using sequential minimal optimization. In B. Schölkopf, C. J. C. Burges, and A. J. Smola, editors, Advances in Kernel Methods - Support Vector Learning, pages 185-208, Cambridge, MA, 1999. MIT Press.
27. L. Ralaivola and F. d'Alché Buc. Incremental support vector machine learning: A local approach. Lecture Notes in Computer Science, 2130:322-329, 2001.
28. H. Robbins and S. Munro. A stochastic approximation method. Ann. Math. Stat., 22:400-407, 1951.
29. S. Rüping. Incremental learning with support vector machines. Technical Report TR-18, Universität Dortmund, SFB475, 2002.
30. D. Saad, editor. On-line learning in neural networks. Cambridge University Press, 1998.
31. B. Schölkopf, S. Mika, C. J. C. Burges, P. Knirsch, K.-R. Müller, G. Ratsch, and A. J. Smola. Input space vs. feature space in kernel-based methods. IEEE Transactions on Neural Networks, 10(5): 1000-1017, September 1999.
32. B. Schölkopf, J. Platt, J. Shawe-Taylor, A. J. Smola, and R. C. Williamson. Estimating the support of a high-dimensional distribution. Neural Computation, 13(7):1443-1471, 2001.
33. N. A. Syed, H. Liu, and K. K. Sung. Incremental learning with support vector machines. In SVM workshop, IJCAI, 1999.
34. D. Tax and R. Duin. Data domain description by support vectors. In M. Verleysen, editor, Proc. ESANN, pages 251-256, Brussels, 1999. D. Facto Press.
35. D. M. J. Tax and P. Laskov. Online SVM learning: from classification to data description and back. In C. et al. Molina, editor, Proc. NNSP, pages 499-508, 2003.
36. I. Tsang, J. Kwok, and P.-M. Cheung. Core Vector Machines: fast SVM training on very large data sets. Journal of Machine Learning Research, 6:363-392, 2005.
37. V. N. Vapnik. Statistical Learning Theory. Wiley, New York, 1998.
38. M. K. Warmuth, J. Liao, G. Rätsch, M. Mathieson, S. Putta, and C. Lemmem. Support Vector Machines for active learning in the drug discovery process. Journal of Chemical Information Sciences, 43(2):667-673, 2003.