Online training on a budget of support vector machines using twin prototypes

Statistical Analysis and Data Mining

Volumn 3, Issue 3, 2010, Pages 149-169

  Wang, Zhuang ^a   Vucetic, Slobodan ^a  

^a Temple University   (United States)

Author keywords

Budgeted learning; CCCP; Online learning; Ramp loss; Support vector machines

Indexed keywords

BUDGET CONTROL; E-LEARNING; VECTOR SPACES; VECTORS;

BUDGETED LEARNING; CCCP; DECISION BOUNDARY; ONLINE LEARNING; ONLINE TRAINING; RAMP LOSS; SPACETIME; SUBLINEAR TIME; SUPPORT VECTOR MACHINES ALGORITHMS; SUPPORT VECTORS MACHINE;

SUPPORT VECTOR MACHINES;

EID: 77954636582     PISSN: 19321872     EISSN: 19321864     Source Type: Journal    
DOI: 10.1002/sam.10075     Document Type: Article

References (47)

1. J. S. Viter, Random sampling with a reservoir, ACM Trans Math Software 11 (1985), 37-57.
2. F. Rosenblatt, The perceptron: a probabilistic model for information storage and organization in the brain, Psychol Rev 65 (1958), 386-408.
3. S. Haykin, Adaptive Filter Theory, New Jersey, Prentice Hall, 2002.
4. N. Cesa-Bianchi and C. Gentile, Tracking the best hyperplane with a simple budget perceptron, Mach Learn 69 (2007), 143-167.
5. K. Crammer, J. Kandola, and Y. Singer, Online classification on a budget, In Advances in Neural Information Processing Systems, Vol. 16, Cambridge, MA, MIT Press, 2004, 225-232.
6. O. Dekel, S. S. Shwartz, and Y. Singer, The forgetron: a kernel-based perceptron on a budget, SIAM J Comput 37 (2008), 1342-1372.
7. S. Vucetic, V. Coric, and Z. Wang, Compressed kernel perceptrons, In Proceedings of Data Compression Conference, 2009, 153-162.
8. Z. Wang and S. Vucetic, Tighter perceptron with improved dual use of cached data for model representation and validation, In Proceedings of International Joint Conference on Neural Networks, 2009, 2766-2771.
9. V. N. Vapnik, Statistical Learning Theory, New York, John Wiley and Sons, Inc., 1998.
10. J. L. Balcázar, Y. Dai, J. Tanaka, and O. Watanabe, Provably fast training algorithms for support vector machines, Theor Comput Syst 42 (2008), 568-595.
11. R. Collobert, F. Sinz, J. Weston, and L. Bottou, Trading convexity for scalability, In Proceedings of International Conference on Machine Learning, 2006, 201-208.
12. Y.-J. Lee and O. L. Mangasarian, RSVM: reduced support vector machines, In Proceedings of SIAM Conference on Data Mining, 2001.
13. D. Pavlov, D. Chudova, and P. Smyth, Towards scalable support vector machines using squashing, In Proceedings of ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2000, 295-299.
14. I. W. Tsang, J. T. Kwok, and P.-M. Cheung, Core vector machines: fast SVM training on very large data sets, J Mach Learn Res 8 (2005), 291-301.
15. S. V. N. Vishwanathan, A. J. Smola, and M. N. Murty. SimpleSVM, In Proceedings of International Conference on Machine Learning, 2003, 760-767.
16. H. Yu, J. Yang, J. Han, and X.-L. Li, Making SVMs scalable to large data sets using hierarchical cluster indexing, Data Mining Knowl Discov 11 (2005), 295-321.
17. I. Steinwart, Sparseness of support vector machines, J Mach Learn Res 4 (2003), 1071-1105.
18. A. L. Yuille and A. Rangarajan, The concave convex procedure (CCCP), In Advances in Neural Information Processing Systems, Vol. 14, Cambridge, MA, MIT Press, 2002, 409-415.
19. G. Cauwenberghs and T. Poggio, Incremental and decremental support vector machine learning, In Advances in Neural Information Processing Systems, Vol. 13, Cambridge, MA, MIT Press, 2001, 409-415.
20. Z. Wang and S. Vucetic, Fast online training of ramp loss support vector machines, In Proceedings of International Conference on Data Mining, 2009, 569-577.
21. D. Nguyen and T. Ho, An efficient method for simplifying support vector machines, In Proceedings of International Conference on Machine Learning, 2005, 617-624.
22. T. Hastie, S. Rosset, R. Tibshirani, and J. Zhu, The entire regularization path for the support vector machine, J Mach Learn Res 5 (2004), 1391-1415.
23. C. P. Diehl and G. Cauwenberghs, SVM incremental learning, adaptation and optimization, In Proceedings of the International Joint Conference on Neural Networks, 2003, 2685-2690.
24. C.-C. Chang and C.-J. Lin, LIBSVM: a library for support vector machines, http://www.csie.ntu.edu.tw/~cjlin/libsvm [Last accessed 2001].
25. S. Tong and D. Koller, Support vector machine active learning with applications to text classification, J Mach Learn Res 2 (2001), 45-66.
26. P. Laskov, H. Gehl, S. Krüger, and K.-R. Müller, Incremental support vector learning: analysis, implementation and applications, J Mach Learn Res 7 (2006), 1909-1936.
27. C. Domeniconi and D. Gunopulos, Incremental support vector machine construction, In Proceedings of International Conference on Data Mining, 2001, 589-592.
28. A. Bordes, S. Ertekin, J. Wesdon, and L. Bottou, Fast kernel classifiers for online and active learning, J Mach Learn Res 6 (2005), 1579-1619.
29. G. Loosli, S. Canu, and L. Bottou, Training invariant support vector machines using selective sampling, In Large Scale Kernel Machines, Cambridge, MA, MIT Press, 2007, 301-320.
30. C. Gentile, A new approximate maximal margin classification algorithm, J Mach Learn Res 2 (2001), 213-242.
31. Y. Li and P. Long, The relaxed online maximum margin algorithm, Mach Learn 46 (2002), 361-387.
32. J. Kivinen, A. J. Smola, and R. C. Williamson, Online learning with kernels, IEEE Trans Signal Process 52 (2002), 2165-2176.
33. K. Crammer, O. Dekel, J. Keshet, S. Shalev-Shwartz, and Y. Singer, Online passive-agressive algorithms, J Mach Learn Res 7 (2006), 551-585.
34. S. Shalev-Shwartz, Y. Singer, and N. Srebro, Pegasos: primal estimated sub-gradient solver for svm, Proceedings of International Conference on Machine Learning, 2007, 807-814.
35. S. Shalev-Shwartz and Y. Singer, Online learning meets optimization in the dual, In Proceedings of Annual Conference on Learning Theory, 2006, 423-437.
36. C. J. C. Burges, Simplified support vector decision rules, In Proceedings of International Conference on Machine Learning, 1996, 71-77.
37. B. Schökopf, S. Mika, C. J. C. Burges, P. Knirsch, K. Müler, G. Räsch, and A. J. Smola, Input space versus feature space in kernel-based methods, IEEE Trans Neural Networks 10 (1999), 1000-1017.
38. O. Dekel and Y. Singer, Support vector machines on a budget, In Advances in Neural Information Processing Systems, Vol. 19, Cambridge, MA, MIT Press, 2006, 345-352.
39. M.-R. Wu, B. Schökopf, and G. Barik, Building sparse large margin classifiers, In Proceedings of International Conference on Machine Learning, 2005, 996-1003.
40. S. S. Keerthi, O. Chapelle, and D. DeCoste, Building support vector machines with reduced classifier complexity, J Mach Learn Res 7 (2006), 1493-1515.
41. B. Li, M. Chi, J. Fan, and X. Xue, Support cluster machine, In Proceedings of International Conference on Machine Learning, 2007, 505-512.
42. H. Shin and S. Cho, Neighborhood property based pattern selection for support vector machines, Neural Comput 19 (2007), 816-855.
43. J. Weston, A. Bordes, and L. Bottou, Online (and offline) on an even tighter budget, In Proceedings of International Workshop on Artificial Intelligence and Statistics, 2005, 413-420.
44. Y. Engel, S. Mannor, and R. Meir, Sparse online greedy support vector regression, In Proceedings of European Conference on Machine Learning, 2002, 84-96.
45. F. Orabona, J. Keshet, and B. Caputo, The projectron: a bounded kernel-based perceptron, In Proceedings of International Conference on Machine Learning, 2008, 720-727.
46. H.-Q. Wang, P. Li, F.-R. Gao, Z.-H. Song, and S. X. Ding, Kernel classifier with adaptive structure and fixed memory for process diagnosis, AIChE Journal 52 (2006), 3515-3531.
47. S. Agarwal, V. V. Saradhi, and H. Karnick, Kernel-based online machine learning and support vector reduction, Neurocomputing 71 (2008), 1230-1237.