The connection between regularization operators and support vector kernels

Neural Networks

Volumn 11, Issue 4, 1998, Pages 637-649

  Smola, Alex J ^a   Schölkopf, Bernhard ^a   Müller, Klaus Robert ^a  

^a FRAUNHOFER FIRST   (Germany)

Author keywords

Conditionally positive definite functions; Green's functions; Mercer kernel; Polynomial kernels; Radial basis functions; Regularization networks; Ridge regression; Support vector machines

Indexed keywords

ALGORITHMS; GREEN'S FUNCTION; PATTERN RECOGNITION; POLYNOMIALS; REGRESSION ANALYSIS;

LINEAR ALGORITHMS; RADIAL BASIS FUNCTIONS; REGULARIZATION NETWORKS; RIDGE REGRESSION; SUPPORT VECTOR MACHINES;

NEURAL NETWORKS;

ALGORITHM; ARTICLE; COMPUTER ANALYSIS; LEARNING; MACHINE; MATHEMATICAL ANALYSIS; NERVE CELL NETWORK; PRIORITY JOURNAL; THEORY;

EID: 0032098361     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(98)00032-X     Document Type: Article

References (38)

1. Aizerman M.A., Braverman E.M., Rozonoér L.I. Theoretical foundations of the potential function method in pattern recognition learning. Automation and Remote Control. 25:1964;821-837.
2. Amari S., Murata N., Müller K.-R., Finke M., Yang H. Asymptotic statistical theory of overtraining and cross-validation. IEEE Trans. Neural Networks. 8(5):1997;985-996.
3. Bishop, C.M. (1995). Neural Networks for Pattern Recognition. Oxford: Clarendon Press.
4. Bochner, S. (1959). Lectures on Fourier integral. Princeton, NJ: Princeton University Press.
5. Boser, B.E., Guyon, I.M., & Vapnik, V.N. (1992). A training algorithm for optimal margin classifiers. In: D. Haussler (ed.), Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, Pittsburgh, PA. ACM Press, pp. 144-152.
6. Cortes C., Vapnik V. Support vector networks. In Mach. Learning. 20:1995;273-297.
7. Dyn, N. (1991). Interpolation and approximation by radial and related functions. In: C.K. Chui, L.L. Schumaker, D.J. Ward, (Eds.), Approximation Theory, vol. VI. New York: Academic Press, pp. 211-232.
8. Ferraro M., Caelli T.M. Lie transformation groups, integral transforms, and invariant pattern recognition. Spatial Vision. 8:1994;33-44.
9. Girosi, F. (1997). An equivalence between sparse approximation and support vector machines. A.I. Memo No. 1606, Artificial Intelligence Laboratory, Massachusetts Institute of Technology (to appear in Neural Computation).
10. Girosi, F., Jones, M., & Poggio, T. (1993). Priors, stabilizers and basis functions, From regularization to radial, tensor and additive splines. A.I. Memo No. 1430, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 1993.
11. Kearns M. A bound on the error of cross validation using the approximation and estimation rates, with consequences for the training-test split. Neural Comput. 9(5):1997;1143-1161.
12. Kimeldorf G.S., Wahba G. A correspondence between Bayesan estimation on stochastic processes and smoothing by splines. Ann. Math. Statist. 2:1971;495-502.
13. MacKay, D.J.C. (1991). Bayesian modelling and neural networks. Ph.D. thesis, Computation and Neural Systems, California Institute of Technology, Pasadena, CA.
14. Madych W.R., Nelson S.A. Multivariate interpolation and conditionally positive definite functions. II. Math. Comput. 54(189):1990;211-230.
15. Micchelli C.A. Interpolation of scattered data, distance matrices and conditionally positive definite functions. Constr. Approx. 2:1986;11-22.
16. More J.J., Toraldo G. On the solution of large quadratic programming problems with bound counstraints. SIAM J. Optimization. 1(1):1991;93-113.
17. Poggio T. On optimal nonlinear associative recall. Biolog. Cybernetics. 19:1975;201-209.
18. Riesz, F., & Nagy, B.Sz. (1955). Functional Analysis. Frederick Ungar, New York, 1955.
19. Rissanen J. Minimum-description-length principle. Ann. Statist. 6:1985;461-464.
20. Schoenberg I.J. (1938a). Metric spaces and completely monotone functions. Ann. Math., 39, 811-841.
21. Schoenberg I.J. (1938b). Metric spaces and positive definite functions. Trans. Amer. Math. Soc., 44, 522-536.
22. Schölkopf, B. (1997). Support vector learning, Ph.D. thesis, Technische Universität Berlin. Also, GMD-Bericht Nr. 287, Oldenbourg Verlag, Munich.
23. Schölkopf, B., Burges, C., & Vapnik, V. (1995). Extracting support data for a given task. In: U.M. Fayyad, R. Uthurusamy (Eds.), Proceedings, First International Conference on Knowledge Discovery and Data Mining, Menlo Park. AAAI Press.
24. Schölkopf, B., Simard, P.Y., Smola, A.J., & Vapnik, V.N. (1998). Prior knowledge in support vector kernels. In: M.I. Jordan, M.J. Kearns, & S.A. Solla (Eds.), Advances in Neural information Processings Systems, vol. 10. Cambridge, MA: MIT Press (in press).
25. Schölkopf, B., Smola, A.J. and Müller, K.R. (1996). Nonlinear component analysis as a kernel eigenvalue problem. Technical Report 44, MPI für biologische Kybernetik (to appear in Neural Computation).
26. Schölkopf B., Sung K., Burges C., Girosi F., Niyogi P., Poggio T., Vapnik V. Comparing support vector machines with gaussian kernels to radial basis function classifiers. IEEE Trans. Sign. Process. 45:1997;2758-2765.
27. Segman J., Rubinstein J., Zeevi Y.Y. The canonical coordinates method for pattern deformation, Theoretical and computational considerations. IEEE Trans. Pattern Anal. Mach. Intell. 14:1992;1171-1183.
28. Smola, A.J., & Schölkopf, B. (1997). On a kernel-based method for pattern recognition, regression, approximation and operator inversion. Technical Report 1064, GMD FIRST. Algorithmica, 1998, in press.
29. Smola, A.J., & Schölkopf, B. (1998). From regularization operators to support vector kernels. In: M.I. Jordan, M.J. Kearns, S.A. Solla (Eds.), Advances in Neural Information Processings Systems, vol. 10. Cambridge, MA: MIT Press (in press).
30. Smola, A.J., Schölkopf, B., & Müller, K.-R. (1998). General cost functions for support vector regression. In: Proceedings of the ACNN'98, Australian Congress on Neural Networks (in press).
31. Tikhonov, A.N., & Arsenin, V.Y. (1977). Solution of Ill-Posed Problems. Washington, DC: Winston.
32. Unser M., Aldroubi A., Eden M. Fast B-spline transforms for continuous image representation and interpolation. IEEE Trans. Pattern Anal. Mach. Intell. 13(3):1991;277-285.
33. Vanderbei, R.J. (1994). LOQO, An interior point code for quadratic programming. Technical Report SOR 94-15, Princeton University.
34. Vapnik, V.N. (1982). Estimation of Dependences Based on Empirical Data. Berlin: Springer.
35. Vapnik, V.N. (1995). The Nature of Statistical Learning Theory. New York: Springer.
36. Vapnik, V.N., Golowich, S., & Smola, A.J. (1997) Support vector method for function approximation, regression estimation, and signal processing, in: NIPS 9, San Mateo, CA.
37. Wahba, G. (1990). Splines Models for Observational Data, Series in Applied Mathematics, vol. 59. Philadelphia: SIAM.
38. Yuille, A., & Grzywacz, N. (1998). The motion coherence theory, in: Proceedings of the International Conference on Computer Vision, pp. 344-354. Washington, D.C., December 1988. IEEE Computer Society Press.