An algebraic characterization of the optimum of regularized kernel methods

Machine Learning

Volumn 74, Issue 3, 2009, Pages 315-345

  Dinuzzo, Francesco ^a   De Nicolao, Giuseppe ^a  

^a UNIVERSITY OF PAVIA   (Italy)

Author keywords

Degrees of freedom; Kernel methods; Regularization theory; Representer theorem; Support vector machines

Indexed keywords

CONSTRAINT THEORY; FUNCTIONS; IMAGE RETRIEVAL; MECHANICS; PROBABILITY DENSITY FUNCTION; PROGRAMMING THEORY; RISK PERCEPTION; SUPPORT VECTOR MACHINES;

ALGEBRAIC EQUATIONS; CHANGE OF VARIABLES; CLASSIFICATION METHODS; CLOSED-FORM SOLUTIONS; DEGREES OF FREEDOM; FINITE NUMBERS; FIXED-POINT EQUATIONS; KERNEL FUNCTIONS; KERNEL METHODS; KERNEL REGRESSION METHODS; LINEAR COMBINATIONS; LOSS FUNCTIONS; NON SMOOTHES; OPTIMIZATION REFORMULATIONS; REGULARIZATION THEORY; REPRESENTER THEOREM; VARIATIONAL PROBLEMS;

CONSTRAINED OPTIMIZATION;

EID: 60349100006     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-008-5095-1     Document Type: Article

References (32)

1. Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In B. N. Petrov & F. Csáki (Eds.), Second international symposium on information theory. Budapest: Académiai Kiadó.
2. O. Chapelle 2007 Training a support vector machine in the primal Neural Computation 19 5 1155 1178
3. Chu, W., Keerthi, S. S., & Ong, C. J. (2001). A unified loss function in Bayesian framework for support vector regression. In Proceedings of the 18th international conference on machine learning (pp. 51-58). San Francisco: Morgan Kaufmann.
4. D. Cox F. O'Sullivan 1990 Asymptotic analysis of penalized likelihood and related estimators The Annals of Statistics 18 1676 1695
5. P. Craven G. Wahba 1979 Smoothing noisy data with spline functions Numerische Mathematik 31 377 403
6. E. De Vito L. Rosasco A. Caponnetto M. Piana A. Verri 2004 Some properties of regularized kernel methods Journal of Machine Learning Research 5 1363 1390
7. O. Dekel S. Shalev-Shwartz Y. Singer 2005 Smooth ε-insensitive regression by loss symmetrization Journal of Machine Learning Research 6 711 741
8. F. Dinuzzo M. Neve G. De Nicolao U. P. Gianazza 2007 On the representer theorem and equivalent degrees of freedom of SVR Journal of Machine Learning Research 8 2467 2495
9. B. Efron 2004 The estimation of prediction error: covariance penalties and cross-validation Journal of the American Statistical Association 99 14 619 632
10. Fukushima, M., & Qi, L. (1999). Reformulation: nonsmooth, piecewise smooth, semismooth and smoothing methods. Dordrecht: Kluwer Academic.
11. L. Gunter J. Zhu 2007 Efficient computation and model selection for the support vector regression Neural Computation 19 1633 1655
12. Hastie, T. J., Tibshirani, R. J., & Friedman, J. (2001). The elements of statistical learning. Data mining, inference and prediction. Canada: Springer.
13. G. Kimeldorf G. Wahba 1971 Some results on Tchebycheffian spline functions Journal of Mathematical Analysis and Applications 33 1 82 95
14. Y. J. Lee O. L. Mangasarian 2001 SSVM: A smooth support vector machine for classification Computational Optimization and Applications 20 5 22
15. p Technometrics 15 661 675
16. Y. Nesterov 2005 Smooth minimization of non-smooth functions Mathematical Programming 103 127 152
17. Osborne, M. R. (2001). Simplicial algorithms for minimizing polyhedral functions. Cambridge: Cambridge University Press.
18. F. Perez-Cruz C. Bousono-Calzon A. Artes-Rodriguez 2005 Convergence of the IRWLS procedure to the support vector machine solution Neural Computation 17 1 7 18
19. Platt, J. (1998). Fast training of support vector machines using sequential minimal optimization. In B. Schölkopf, C. Burges, & A. Smola (Eds.), Advances in kernel methods-support vector learning. Cambridge: MIT Press.
20. Poggio, T., & Girosi, F. (1992). A theory of networks for approximation and learning. In Foundation of neural networks (pp. 91-106).
21. M. Pontil A. Verri 1998 Properties of support vector machines Neural Computation 10 955 974
22. B. Schölkopf R. Herbrich A. J. Smola 2001 A generalized representer theorem Neural Networks and Computational Learning Theory 81 416 426
23. Schölkopf, B., & Smola, A. J. (2001). Learning with kernels: support vector machines, regularization, optimization, and beyond. Adaptive computation and machine learning. Cambridge: MIT Press.
24. G. Schwarz 1978 Estimating the dimension of a model The Annals of Statistics 6 461 464
25. Shalev-Shwartz, S., Singer, Y., & Srebro, N. (2007). PEGASOS: primal estimated sub-gradient solver for svm. In ICML '07: proceedings of the 24th international conference on machine learning (pp. 807-814). New York: ACM.
26. C. Stein 1981 Estimation of the mean of a multivariate normal distribution The Annals of Statistics 9 1135 1151
27. I. Steinwart 2003 Sparseness of support vector machines Journal of Machine Learning Research 4 1071 1105
28. Vapnik, V. (1995). The nature of statistical learning theory. New York: Springer.
29. Wahba, G. (1990). Spline models for observational data. Philadelphia: SIAM.
30. Wahba, G. (1998). Support vector machines, reproducing kernel Hilbert spaces and randomized GACV (Tech. Rep. 984). Department of Statistics, University of Wisconsin.
31. J. Ye 1998 On measuring and correcting the effects of data mining and model selection Journal of the American Statistical Association 93 120 131
32. H. Zou T. Hastie R. Tibshirani 2007 On the degrees of freedom of the lasso The Annals of Statistics 35 5 2173 2192