Input space versus feature space in kernel-based methods

IEEE Transactions on Neural Networks

Volumn 10, Issue 5, 1999, Pages 1000-1017

  Schölkopf, Bernhard ^a   Mika, Sebastian ^b   Burges, Chris J C ^c   Knirsch, Philipp ^d   Müller, Klaus Robert ^b   Rätsch, Gunnar ^b   Smola, Alexander J ^b  

^a MICROSOFT RESEARCH   (United Kingdom)

^b FRAUNHOFER FIRST   (Germany)

^c LUCENT TECHNOLOGIES   (United States)

^d MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; COMPUTATIONAL COMPLEXITY; DECISION THEORY; FUNCTIONS; PATTERN RECOGNITION; POLYNOMIALS; SET THEORY; STATISTICAL METHODS; VECTORS;

PRINCIPAL COMPONENT ANALYSIS (PCA); SUPPORT VECTOR MACHINES (SVM);

NEURAL NETWORKS;

EID: 0032594954     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.788641     Document Type: Article

References (38)

1. M. Aizerman, E. Braverman, and L. Rozonoer, "Theoretical foundations of the potential function method in pattern recognition learning," Automat. Remote Contr., vol. 25, pp. 821-837, 1964.
2. N. Aronszajn, "Theory of reproducing kernels," Trans. Amer. Math. Soc., vol. 68, pp. 337-404, 1950.
3. K. L. Blackmore, R. C. Williamson, and I. M. Y. Mareels, "Decision region approximation by polynomials or neural networks," IEEE Trans. Inform. Theory, vol. 43, pp. 903-907, 1997.
4. B. E. Boser, I. M. Guyon, and V. N. Vapnik, "A training algorithm for optimal margin classifiers," in Proc. 5th Annu. ACM Wkshp. Comput. Learning Theory, D. Haussler, Ed. Pittsburgh, PA: ACM, July 1992, pp. 144-152.
5. J. M. Buhmann, "Data clustering and learning," in The Handbook of Brain Theory and Neural Networks, M. A. Arbib, Ed. Cambridge, MA: MIT Press, 1995, pp. 278-281.
6. C. J. C. Burges, "Simplified support vector decision rules," in Proc. 13th Int. Conf Machine Learning, L. Saitta, Ed. San Mateo, CA: Morgan Kaufmann, 1996, pp. 71-77.
7. _, "A tutorial on support vector machines for pattern recognition," Data Mining and Knowledge Discovery, vol. 2, no. 2, pp. 1-47, 1998.
8. _, "Geometry and invariance in kernel based methods," in Advances in Kernel Methods - Support Vector Learning, B. Schölkopf, C. Burges, and A. Smola, Eds. Cambridge, MA: MIT Press, 1999, pp. 89-116.
9. C. J. C. Burges and B. Schölkopf, "Improving the accuracy and speed of support vector learning machines," in Advances in Neural Information Processing Systems 9, M. Mozer, M. Jordan, and T. Petsche, Eds. Cambridge, MA: MIT Press, 1997, pp. 375-381.
10. B. Carl and I. Stephani, Entropy, Compactness, and the Approximation of Operators. Cambridge, U.K.: Cambridge Univ. Press, 1990.
11. K. I. Diamantaras and S. Y. Kung, Principal Component Neural Networks. New York: Wiley, 1996.
12. T. Frieß and R. F. Harrison, "Linear programming support vector machines for pattern classification and regression estimation; and the SR algorithm: Improving speed and tightness of VC bounds in SV algorithms," University of Sheffield, U.K., Department Automatic Control and Systems Engineering, Res. Rep. 706, 1998.
13. F. Girosi, "An equivalence between sparse approximation and support vector machines," Neural Comput., vol. 10, no. 6, pp. 1455-1480, 1998.
14. T. Hastie and W. Stuetzle, "Principal curves," JASA, vol. 84, pp. 502-516, 1989.
15. T. J. Hastie and R. J. Tibshirani, Generalized Additive Models, vol. 43 of Monographs on Statistics and Applied Probability. London: Chapman & Hall, 1990.
16. Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. J. Jackel, "Backpropagation applied to handwritten zip code recognition," Neural Comput., vol. 1, pp. 541-551, 1989.
17. S. Mallat and Z. Zhang, "Matching Pursuit in a time-frequency dictionary," IEEE Trans. Signal Processing, vol. 41, pp. 3397-3415, 1993.
18. D. Mattera, personal communication, 1998.
19. C. A. Micchelli, "Interpolation of scattered data: Distance matrices and conditionally positive definite functions," Constructive Approximation, vol. 2, pp. 11-22, 1986.
20. S. Mika, "Nichtlineare Signalverarbeitung in Feature-Räumen," Diplom degree thesis, Technische Universität Berlin, Germany, 1998 (in German).
21. S. Mika, B. Schölkopf, A. Smola, K.-R. Müller, M. Scholz, and G. Rätsch, "Kernel PCA and denoising in feature spaces," in Advances Neural Inform. Processing Syst. 11, 1999.
22. E. Osuna and F. Girosi, "Reducing run-time complexity in support vector machines," in Advances in Kernel Methods - Support Vector Learning, B. Schölkopf, C. Burges, and A. Smola, Eds. Cambridge, MA: MIT Press, 1999, pp. 271-283.
23. S. Saitoh, Theory of Reproducing Kernels and its Applications. Harlow, U.K.: Longman, 1988.
24. B. Schölkopf, Support Vector Learning, R. Oldenbourg Verlag, München, Doktor degree thesis, TU Berlin, 1997 (in German).
25. B. Schölkopf, P. Knirsch, A. Smola, and C. Burges, "Fast approximation of support vector kernel expansions, and an interpretation of clustering as approximation in feature spaces," in Mustererkennung 1998-20. DAGM-Symp., P. Levi, M. Schanz, R.-J. Ahlers, and F. May, Eds., Informatik aktuell Berlin, Germany: Springer-Verlag, 1998, pp. 124-132.
26. B. Schölkopf, S. Mika, A. Smola, G. Rätsch, and K.-R. Müller, "Kernel PCA pattern reconstruction via approximate preimages," in Proc. 8th Int. Conf Artificial Neural Networks, Perspectives in Neural Computing, L. Niklasson, M. Bodén, and T. Ziemke, Eds. Berlin, Germany: Springer-Verlag, 1998, pp. 147-152.
27. B. Schölkopf, J. Shawe-Taylor, A. J. Smola, and R. C. Williamson, "Kernel-dependent support vector error bounds," in Proc. ICANN, 1999, to be published.
28. B. Schölkopf, P. Simard, A. Smola, and V. Vapnik, "Prior knowledge in support vector kernels," in Advances in Neural Information Processing Systems 10, M. Jordan, M. Kearns, and S. Solla, Eds. Cambridge, MA: MIT Press, 1998, pp. 640-646.
29. B. Schölkopf, A. Smola, and K.-R. Müller, "Nonlinear component analysis as a kernel eigenvalue problem," Neural Comput., vol. 10, pp. 1299-1319, 1998.
30. B. Schölkopf, K. Sung, C. Burges, F. Girosi, P. Niyogi, T. Poggio, and V. Vapnik, "Comparing support vector machines with Gaussian kernels to radial basis function classifiers," IEEE Trans. Signal Processing, vol. 45, pp. 2758-2765, 1997.
31. A. Smola and B. Schölkopf, "From regularization operators to support vector kernels," in Advances in Neural Information Processing Systems 10, M. Jordan, M. Kearns, and S. Solla, Eds. Cambridge, MA: MIT Press, 1998, pp. 343-349.
32. _, "A tutorial on support vector regression," NeuroCOLT, Rep. 19, 1998. Available http://svm.first.gmd.de
33. K. Tsuda, "Support vector classifier with asymmetric kernel function," in Proc. ESANN, M. Verleysen, Ed. Brussels: D Facto, 1999, pp. 183-188.
34. V. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
35. _, Statistical Learning Theory. New York: Wiley, 1998.
36. G. Wahba, Spline Models for Observational Data, vol. 59 of CBMS-NSF Regional Conference Series in Applied Mathematics. Philadelphia, PA: SIAM, 1990.
37. R. Williamson, A. Smola, and B. Schölkopf, "Entropy numbers, operators, and support vector kernels," in Advances in Kernel Methods - Support Vector Learning, B. Schölkopf, C. Burges, and A. Smola, Eds. Cambridge, MA: MIT Press, 1999, pp. 127-144.
38. _, "Generalization performance of regularization networks and support vector machines via entropy numbers of compact operators," NeuroCOLT, Tech. Rep. 19, 1998. Available http://www.neurocolt.com