Which is better? Regularization in RKHS vs ℝm on reduced SVMs

Statistics, Optimization and Information Computing

Volumn 1, Issue 1, 2013, Pages 82-106

  Zhou, Shuisheng ^a  

^a XIDIAN UNIVERSITY   (China)

Author keywords

Newton algorithms; Regularizer; Representer theorem; RSVM; SVMs

Indexed keywords

EID: 84978884081     PISSN: 2311004X     EISSN: 23105070     Source Type: Journal    
DOI: 10.19139/soic.v1i1.27     Document Type: Article

References (52)

1. V. N. Vapnik. An overview of statistical learning theory. IEEE trans on Neural Network, 10(5):988-999, 1999.
2. V. N. Vapnik. The Nature of Statistical Learning Theory. Springer-Verlag, New York, 2000.
3. I. Steinwart. Sparseness of Support Vector Machines. Journal of Machine Learning Research, 4:1071-1105, 2003.
4. I. Steinwart and A. Christmann. Support Vector Machines. Springer, 2008.
5. S. S. Keerthi, O. Chapelle, and D. Decoste. Building Support Vector Machines with reduced classifier complexity. J. of Machine Learning Research, 7:1493-1515, 2006.
6. I. Steinwart, D. Hush, and C. Scovel. Training SVMs without offset. Journal of Machine Learning Research, 12:141-202, 2011.
7. B. Schölkopf, A. Smola, and K.-R. Müller. Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput., 10(5):1299-1319, 1998.
8. S. Mika, G. Rätsch, J. Weston, B. Schölkopf, and K. R. Müller. Fisher discriminant analysis with kernels. In E. Wilson Y. H. Hu, J. Larsen and S. Douglas, editors, Proc. NNSP'99, pages 41-48. IEEE, 1999.
9. G. S. Kimeldorf and G. Wahba. A correspondence between bayesian estimation on stochastic processes and smoothing by splines. Annals of Mathematical Statistics, 41:495-502, 1970.
10. G. S. Kimeldorf and G. Wahba. Some results on Tchebycheffian spline functions. Journal of Mathematical Analysis Application, 33:82-95, 1971.
11. Bernhard Schölkopf, Ralf Herbrich, and Alex J. Smola. A generalized representer theorem. In Proceedings of the Annual Conference on Computational Learning Theory, pages 416-426. Springer, 2001.
12. E. De Vito, L. Rosasco, A. Caponnetto, M. Piana, and A. Verri. Some properties of regularized kernel methods. Journal of Machine Learning Research, 5:1363-1390, 2004.
13. J. Abernethy, F. Bach, T. Evgeniou, and J.-P. Vert. A new approach to collaborative filtering: Operator estimation with spectral regularization. J. of Machine Learning Research, 10:803-826, 2009.
14. A. Argyriou, C. A. Micchelli, and M. Pontil. When is there a representer theorem? Vector versus matrix regularizers. J. of Machine Learning Research, 10:2507-2529, 2009.
15. Manfred K. Warmuth, Wojciech Kotlowski, and Shuisheng Zhou. Kernelization of matrix updates, when and how? In Proceedings of the 23rd international conference on Algorithmic Learning Theory, ALT'12, pages 350-364, Berlin, Heidelberg, 2012. Springer-Verlag.
16. G. H. Golub and C. F. V. Loan. Matrix Computations. The John Hopkins University Press, Baltimore, Maryland, 1996.
17. O. L. Mangasarian. Generalized Support Vector Machine. In A. J. Smola, P. Bartlett, B. Schökopf, and D. Schuurmans, editors, Advances in Large Margin Classifiers, pages 135-146. MIT Press, 2000.
18. Y.-J. Lee and O. L. Mangasarian. SSVM: A smooth Support Vector Machine for classification. Computational Optimization and Applications, 20(1):5-22, 2001.
19. Y.-J. Lee and Olvi L. Mangasarian. RSVM: Reduced Support Vector Machines. In CD Proceedings of the SIAM International Conference on Data Mining, pages 1-17, Chicago, April 2001. SIAM.
20. O. L. Mangasarian. A finite Newton method for classification. Optimization Methods and Software, 17(5):1155-1178, 2002.
21. G. Fung and O. L. Mangasarian. Proximal Support Vector Machine classifiers. In F. Provost and R. Srikant, editors, Proceedings KDD-2001: Knowledge Discovery and Data Mining, August 26-29, San Francisco, CA, pages 77-86, New York, 2001. ACM.
22. O. Chapelle. Training a Support Vector Machine in the primal. Neural Computation, 19(5):1155-1178, 2007.
23. light, Support Vector Machine, 1998.
24. J. C. Platt. Fast training of Support Vector Machines using Sequential Minimal Optimization. In B. Schölkopf, C. J. Burges, and A. J. Smola, editors, Advances in Kernel Method-Support Vector Learning, pages 185-208. MIT Press, 1999.
25. J. Suykens and J. Vandewalle. Least square Support Vector Machine classifiers. Neural Processing Letters, 9(3):293-300, 1999.
26. N. Cristianini and J. Shawe-Taylor. An Introduction to Support Vector Machines. Cambridge University Press, Cambridge, 2000.
27. S. Fine and K. Scheinberg. Efficient SVM training using low-rank kernel representations. Journal of Machine Learning Research, 2:243-264, 2001.
28. M. C. Ferris and T. S. Munson. Semismooth Support Vector Machines. Mathematical Programming, Ser. B-101:185-204, 2004.
29. S. P. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, Cambridge, 7th edition, 2009.
30. J. A. K. Suykens, T. V. Gestel, J. De Brabanter, B. De Moor, and J. Vandewalle. Least squares Support Vector Machines. World Scientific, River Edge, NJ, 2002.
31. K.-M. Lin and C.-J. Lin. A study on reduced Support Vector Machines. IEEE Trans. on Neural Networks, 14(6):1449-1459, 2003.
32. Glenn M. Fung and O. L. Mangasarian. Multicategory proximal Support Vector Machine classifiers. Mach. Learn., 59(1-2):77-97, 2005.
33. O. L. Mangasarian and E. W. Wild. Multisurface proximal Support Vector Machine classification via generalized eigenvalues. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(1):69-74, 2006.
34. S. Ghorai, S. J. Hossain, A. Mukherjee, and P. K. Dutta. Newton's method for nonparallel plane proximal classifier with unity norm hyperplanes. Signal Processing, 90(1):93-104, 2010.
35. Chih-Chung Chang and Chih-Jen Lin. LIBSVM: A library for support vector machines. ACM Trans. on Intelligent Systems and Technology, 2:1-27, 2011.
36. S. S. Keerthi, S. K. Shevade, C. Bhattacharyya, and K. R. K. Murthy. Improvements to Platt's SMO algorithm for SVM classifier design. Neural Computation, 13:637-649, 2001.
37. O. L Mangasarian and D. R. Musicant. Successive overrelaxation for Support Vector Machines. IEEE Transactions on Neural Networks, 10(5):1032-1037, 1999.
38. O. L. Mangasarian and D. R. Musicant. Lagrangian Support Vector Machines. Journal of Machine Learning Research, 1:161-177, 2001.
39. Shuisheng Zhou, Hongwei Liu, Feng Ye, and Lihua Zhou. A new iterative algorithm training SVM. Optimization Methods and Software, 24(6):913-932, 2009.
40. Jon Dattorro. Convex Optimization & Euclidean Distance Geometry. Meboo Publishing USA, 2012.
41. T. Joachims. Making large-scale SVM learning practical. In B. Schölkopf, C. Burges, and A. Smola, editors, Advances in Kernel Methods-Support Vector Learning, pages 169-184. MIT Press, 1999.
42. Shuisheng Zhou, Jiangtao Cui, Feng Ye, Hongwei Liu, and Qiang Zhu. New smoothing SVM algorithm with tight error bound and efficient reduced techniques. Computational Optimization and Applications, 56(3):599-618, 2013. DOI 10.1007/s10589-013-9571-6.
43. O. Chapelle. Support vector machines in the primal, 2006.
44. C.-J. Lin. On the convergence of the decomposition method for Support Vector Machines. IEEE Trans. on Neural Networks, 12:1288-1298, 2001.
45. C.-J. Lin. Asymptotic convergence of an SMO algorithm without any assumptions. IEEE Trans. on Neural Networks, 13:248-250, 2002.
46. Shuisheng Zhou, Hongwei Liu, Lihua Zhou, and Feng Ye. Semismooth Newton Support Vector Machine. Pattern Recognition Letters, 28:2054-2062, 2007.
47. Feng Ye, Hongwei Liu, Shuisheng Zhou, and Sanyang Liu. A smoothing trust-region Newton-CG method for minimax problem. Applied Mathematics and Computation, 199(2):581-589, 2008.
48. L. Qi and D. Sun. A survey of some nonsmooth equations and smoothing Newton methods. In A. Eberhard, B. Glover, R. Hill, and D. Ralph, editors, Progress in Optimization, volume 30 of Applied Optimization, pages 121-146, Dordrecht, 1999. Kluwer Academic Publishers.
49. J. Sun. On piecewise quadratic Newton and trust region problems. Mathematical programming, 76:451-467, 1997.
50. P. J. Huber. Robust Statistics. John Wiley, New York, 1981.
51. T. K. Ho and E. M. Kleinberg. Building projectable classifiers of arbitrary complexity. In Proceedings of the 13th International Conference on Pattern Recognition, pages 880-885, Vienna, Austria, 1996.
52. Rong-En Fan and Chih-Jen Lin. LIBSVM Data, 2010.