Kernel methods in machine learning

Annals of Statistics

Volumn 36, Issue 3, 2008, Pages 1171-1220

  Hofmann, Thomas ^a   Schölkopf, Bernhard ^b   Smola, Alexander J ^c  

^a DARMSTADT UNIVERSITY OF TECHNOLOGY   (Germany)

^b MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

^c CSIRO   (Australia)

Author keywords

Graphical models; Machine learning; Reproducing kernels; Support vector machines

Indexed keywords

EID: 51049096780     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/009053607000000677     Document Type: Review

References (157)

1. AIZERMAN, M. A., BRAVERMAN, É. M. and ROZONOÉR, L. I. (1964). Theoretical foundations of the potential function method in pattern recognition learning. Autom. Remote Control 25 821-837.
2. ALLWEIN, E. L., SCHAPIRE, R. E. and SINGER, Y. (2000). Reducing multiclass to binary: A unifying approach for margin classifiers. In Proc. 17th International Conf. Machine Learning (P. Langley, ed.) 9-16. Morgan Kaufmann, San Francisco, CA. MR1884092
3. ALON, N., BEN-DAVID, S., CESA- BIANCHI, N. and HAUSSLER, D. (1993). Scale-sensitive dimensions, uniform convergence, and learnability. In Proc. of the 34rd Annual Symposium on Foundations of Computer Science 292-301. IEEE Computer Society Press, Los Alamitos, CA. MR1328428
4. ALTUN, Y., HOFMANN, T. and SMOLA, A. J. (2004). Gaussian process classification for segmenting and annotating sequences. In Proc. International Conf. Machine Learning 25-32. ACM Press. New York.
5. ALTUN, Y., SMOLA, A. J. and HOFMANN, T. (2004). Exponential families for conditional random fields. In Uncertainty in Artificial Intelligence (UAI) 2-9. AUAI Press, Arlington, VA.
6. ALTUN, Y., TSOCHANTARIDIS, I. and HOFMANN, T. (2003). Hidden Markov support vector machines. In Proc. Intl. Conf. Machine Learning 3-10. AAAI Press, Menlo Park. CA.
7. ARONSZAJN, N. (1950). Theory of reproducing kernels. Trans. Amer. Math. Soc. 68 337-404, MR0051437
8. BACH, F. R. and JORDAN, M. I. (2002). Kernel independent component analysis. J. Mach. Learn. Res. 31-48, MR 1966051
9. BAKIR, G., HOFMANN, T., SCHÖLKOPF, B., SMOLA, A., TASKAR, B. and VISHWANATHAN, S. V. N. (2007). Predicting Structured Data. MIT Press, Cambridge, MA.
10. BAMBER, D. (1975). The area above the ordinal dominance graph and the area below the receiver operating characteristic graph. J. Math. Psych. 12 387-415. MR0384214
11. BARNDORFF-NIELSEN, O. E. (1978). Information and Exponential Families in Statistical Theory. Wiley, New York. MR0489333
12. BARTLETT, P. L. and MENDELSON, S. (2002). Rademacher and gaussian complexities: Risk bounds and structural results. J. Mach. Learn. Res. 3 463-482. MR 1984026
13. BASILICO, J. and HOFMANN, T. (2004). Unifying collaborative and content-based filtering. In Proc. Intl. Conf. Machine Learning 65-72. ACM Press, New York.
14. BAUM, L. E. (1972). An inequality and associated maximization technique in statistical estimation of probabilistic functions of a Markov process. Inequalities 3 1-8. MR0341782
15. BEN-DAVID, S., EIRON, N. and LONG, P. (2003). On the difficulty of approximately maximizing agreements. J. Comput. System Sci. 66 496-514. MR1981222
16. BENNETT, K. P., DEMIRIZ, A. and SHAWE- TAYLOR, J. (2000). A column generation algorithm for boosting. In Proc. 17th International Conf. Machine Learning (P. Langley, ed.) 65-72. Morgan Kaufmann, San Francisco, CA.
17. BENNETT, K. P. and MANGASARIAN, O. L. (1992). Robust linear programming discrimination of two linearly inseparable sets. Optim. Methods Softw. 1 23-34.
18. BERG, C., CHRISTENSEN, J. P. R. and RESSEL, P. (1984). Harmonic Analysis on Semigroups. Springer, New York. MR0747302
19. BERTSIMAS, D. and TSITSIKLIS, J. (1997). Introduction to Linear Programming. Athena Scientific, Nashua, NH.
20. BLOOMFIELD, P. and STEIGER, W. (1983). Least Absolute Deviations: Theory. Applications and Algorithms. Birkhäuser, Boston. MR0748483
21. BOCHNER, S. (1933). Monotone Funktionen, Stieltjessche Integrale und harmonische Analyse. Math. Ann. 108 378-410. MR1512856
22. BORGWARDT, K. M., GRETTON, A., RASCH, M. J., KRIEGEL, H.-P., SCHÖLKOPF, B. and SMOLA, A. J. (2006). Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics (ISMB) 22 e49-e57.
23. BOSER, B., GUYON, I. and VAPNIK, V. (1992). A training algorithm for optimal margin classifiers. In Proc. Annual Conf. Computational Learning Theory (D. Haussler, ed.) 144-152. ACM Press, Pittsburgh, PA.
24. BOUSQUET, O., BOUCHERON, S. and LUGOSI, G. (2005). Theory of classification: A survey of recent advances. ESAIM Probab. Statist. 9 323-375. MR2182250
25. B URGES, C. J. C. (1998). A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Discov. 2 121-167.
26. CARDOSO, J.-F. (1998). Blind signal separation: Statistical principles. Proceedings of the IEEE 90 2009-2026.
27. CHAPELLE, O. and HARCHAOUI, Z. (2005). A machine learning approach to conjoint analysis. In Advances in Neural Information Processing Systems 17 (L. K. Saul, Y. Weiss and L. Bottou, eds.) 257-264. MIT Press, Cambridge, MA.
28. CHEN, A. and BICKEL, P. (2005). Consistent independent component analysis and prewhitening. IEEE Trans. Signal Process. 53 3625-3632. MR2239886
29. CHEN, S., DONOHO, D. and SAUNDERS, M. (1999). Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20 33-61. MR1639094
30. COLLINS, M. (2000). Discriminative reranking for natural language parsing. In Proc. 17th International Conf. Machine Learning (P. Langley, ed.) 175-182. Morgan Kaufmann, San Francisco, CA.
31. COLLINS, M. and DUFFY, N. (2001). Convolution kernels for natural language. In Advances in Neural Information Processing Systems 14 (T. G. Dietterich, S. Becker and Z. Ghahramani, eds.) 625-632. MIT Press, Cambridge, MA.
32. COOK, D., BUJA, A. and CABRERA, J. (1993). Projection pursuit indices based on orthonormal function expansions. J. Comput. Graph. Statist. 2 225-250. MR1272393
33. CORTES, C., MOHRI, M. and WESTON, J. (2005). A general regression technique for learning transductions. In ICML'05: Proceedings of the 22nd International Conference on Machine Learning 153-160. ACM Press, New York.
34. CORTES, C. and VAPNIK, V. (1995). Support vector networks. Machine Learning 20 273-297.
35. CRAMMER, K. and SINGER, Y. (2001). On the algorithmic implementation of multiclass kernel-based vector machines. J. Mach. Learn. Res. 2 265-292.
36. CRAMMER, K. and SINGER, Y. (2005). Loss bounds for online category ranking. In Proc. Annual Conf. Computational Learning Theory (P. Auer and R. Meir, eds.) 48-62. Springer, Berlin. MR2203253
37. CRISTIANINI, N. and SHAWE-TAYLOR, J. (2000). An Introduction to Support Vector Machines. Cambridge Univ. Press.
38. CRISTIANINI, N., SHAWE-TAYLOR, J., ELISSEEFF, A. and KANDOLA, J. (2002). On kernel-target alignment. In Advances in Neural Information Processing Systems 14 (T. G. Dietterich, S. Becker and Z. Ghahramani, eds.) 367-373. MIT Press, Cambridge, MA.
39. CULOTTA, A., KULP, D. and MCCALLUM, A. (2005). Gene prediction with conditional random fields. Technical Report UM-CS-2005-028, Univ. Massachusetts, Amherst.
40. DARROCH, J. N. and RATCLIFF, D. (1972). Generalized iterative scaling for log-linear models. Ann. Math. Statist. 43 1470-1480. MR0345337
41. DAS, D. and SEN, P. (1994). Restricted canonical correlations. Linear Algebra Appl. 210 29-47. MR1294769
42. DAUXOIS, J. and NKIET, G. M. (1998). Nonlinear canonical analysis and independence tests. Ann. Statist. 26 1254-1278. MR1647653
43. DAWID, A. P. (1992). Applications of a general propagation algorithm for probabilistic expert systems. Stat. Comput. 2 25-36.
44. DECOSTE, D. and SCHÖLKOPF, B. (2002). Training invariant support vector machines. Machine Learning 46 161-190.
45. DEKEL, O., MANNING, C. and SINGER, Y. (2004). Log-linear models for label ranking. In Advances in Neural Information Processing Systems 16 (S. Thrun, L. Saul and B. Schölkopf, eds.) 497-504. MIT Press, Cambridge, MA.
46. DELLA PIETRA, S., DELLA PIETRA, V. and LAFFERTY, J. (1997). Inducing features of random fields. IEEE Trans. Pattern Anal. Machine Intelligence 19 380-393.
47. EINMAL, J. H. J. and MASON, D. M. (1992). Generalized quantile processes. Ann. Statist. 20 1062-1078. MR1165606
48. ELISSEEFF, A. and WESTON, J. (2001). A kernel method for multi-labeled classification. In Advances in Neural Information Processing Systems 14 681-687. MIT Press, Cambridge, MA.
49. FIEDLER, M. (1973). Algebraic connectivity of graphs. Czechoslovak Math. J. 23 298-305. MR0318007
50. FITZGERALD, C. H., MICCHELLI, C. A. and PINKUS, A. (1995). Functions that preserve families of positive semidefinite matrices. Linear Algebra Appl. 221 83-102. MR1331791
51. FLETCHER, R. (1989). Practical Methods of Optimization. Wiley, New York. MR0955799
52. FORTET, R. and MOURIER, E. (1953). Convergence de la réparation empirique vers la réparation théorique. Ann. Scient. École Norm. Sup. 70 266-285. MR0061325
53. FREUND, Y. and SCHAPIRE, R. E. (1996). Experiments with a new boosting algorithm. In Proceedings of the International Conference on Machine Learing 148-146. Morgan Kaufmann, San Francisco, CA.
54. FRIEDMAN, J. H. (1987). Exploratory projection pursuit. J. Amer. Statist. Assoc. 82 249-266. MR0883353
55. FRIEDMAN, J. H. and TUKEY, J. W. (1974). A projection pursuit algorithm for exploratory data analysis. IEEE Trans. Comput. C-23 881-890.
56. GÄRTNER, T. (2003). A survey of kernels for structured data. SIGKDD Explorations 5 49-58.
57. GREEN, P. and YANDELL, B. (1985). Semi-parametric generalized linear models. Proceedings 2nd International GLIM Conference. Lecture Notes in Statist. 32 44-55. Springer, New York.
58. GRETTON, A., BOUSQUET, O., SMOLA, A. and SCHÖLKOPF, B. (2005). Measuring statistical dependence with Hilbert-Schmidt norms. In Proceedings Algorithmic Learning Theory (S. Jain, H. U. Simon and E. Tomita, eds.) 63-77. Springer, Berlin. MR2255909
59. GRETTON, A., SMOLA, A., BOUSQUET, O., HERBRICH, R., BELITSKI, A., AUGATH, M., MURAYAMA, Y., PAULS, J., SCHÖLKOPF, B. and LOGOTHETIS, N. (2005). Kernel constrained covariance for dependence measurement. In Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics (R. G. Cowell and Z. Ghahramani, eds.) 112-119. Society for Artificial Intelligence and Statistics, New Jersey.
60. HAM, J., LEE, D., MIKA, S. and SCHÖLKOPF, B. (2004). A kernel view of the dimensionality reduction of manifolds. In Proceedings of the Twenty-First International Conference on Machine Learning 369-376. ACM Press, New York.
61. HAMMERSLEY, J. M. and CLIFFORD, P. E. (1971). Markov fields on finite graphs and lattices. Unpublished manuscript.
62. HAUSSLER, D. (1999). Convolutional kernels on discrete structures. Technical Report UCSC-CRL-99-10, Computer Science Dept., UC Santa Cruz.
63. HEIN, M., BOUSQUET, O. and SCHÖLKOPF, B. (2005). Maximal margin classification for metric spaces. J. Comput. System Sci. 71 333-359. MR2168357
64. HERBRICH, R. (2002). Learning Kernel Classifiers: Theory and Algorithms. MIT Press, Cambridge, MA.
65. HERBRICH, R., GRAEPEL, T. and OBERMAYER, K. (2000). Large margin rank boundaries for ordinal regression. In Advances in Large Margin Classifiers (A. J. Smola, P. L. Bartlett, B. Schölkopf and D. Schuurmans, eds.) 115-132. MIT Press, Cambridge, MA. MR1820960
66. HETTICH, R. and KORTANEK, K. O. (1993). Semi-infinite programming: Theory, methods, and applications. SIAM Rev. 35 380-429. MR1234637
67. HILBERT, D. (1904). Grundzüge einer allgemeinen Theorie der linearen Integralgleichungen. Nachr. Akad. Wiss. Göttingen Math.-Phys. Kl. II 49-91.
68. HOERL, A. E. and KENNARD, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12 55-67.
69. HOFMANN, T., SCHÖLKOPF, B. and SMOLA, A. J. (2006). A review of kernel methods in machine learning. Technical Report 156, Max-Planck-Institut für biologische Kybernetik.
70. HOTELLING, H. (1936). Relations between two sets of variates. Biometrika 28 321-377.
71. HUBER, P. J. (1981). Robust Statistics. Wiley, New York. MR0606374
72. HUBER, P. J. (1985). Projection pursuit. Ann. Statist. 13 435-475. MR0790553
73. HYVÄRINEN, A., KARHUNEN, J. and OJA, E. (2001). Independent Component Analysis. Wiley, New York.
74. JAAKKOLA, T. S. and HAUSSLER, D. (1999). Probabilistic kernel regression models. In Proceedings of the 7th International Workshop on AI and Statistics. Morgan Kaufmann, San Francisco, CA.
75. JEBARA, T. and KONDOR, I. (2003). Bhattacharyya and expected likelihood kernels. Proceedings of the Sixteenth Annual Conference on Computational Learning Theory(B. Scholkopf and M. Warmuth, eds.) 57-71. Lecture Notes in Comput. Sci.2777. Springer, Heidelberg.
76. JENSEN, F. V., LAURITZEN, S. L. and OLESEN, K. G. (1990). Bayesian updates in causal probabilistic networks by local computation. Comput. Statist. Quaterly 4 269-282. MR1073446
77. JOACHIMS, T. (2002). Learning to Classify Text Using Support Vector Machines: Methods, Theory, and Algorithms. Kluwer Academic, Boston.
78. JOACHIMS, T. (2005). A support vector method for multivariate performance measures. In Proc. Intl. Conf. Machine Learning 377-384. Morgan Kaufmann, San Francisco, CA.
79. JONES, M. C. and SIBSON, R. (1987). What is projection pursuit? J. Roy. Statist. Soc. Ser. A 150 1-36. MR0887823
80. JORDAN, M. I., BARTLETT, P. L. and. MCAULIFFE, J. D. (2003). Convexity, classification, and risk bounds. Technical Report 638, Univ. California, Berkeley.
81. KARUSH, W. (1939). Minima of functions of several variables with inequalities as side constraints. Master's thesis, Dept. Mathematics, Univ. Chicago.
82. KASHIMA, H., TSUDA, K. and INOKUCHI, A. (2003). Marginalized kernels between labeled graphs. In Proc. Intl. Conf. Machine Learning 321-328. Morgan Kaufmann, San Francisco, CA.
83. KETTENRING, J. R. (1971). Canonical analysis of several sets of variables. Biometrika 58 433-451. MR0341750
84. KIM, K., FRANZ, M. O. and SCHÖLKOPF, B. (2005). Iterative kernel principal component analysis for image modeling. IEEE Trans. Pattern Analysis and Machine Intelligence 27 1351-1366.
85. KLMELDORF, G. S. and WAHBA, G. (1971). Some results on Tchebycheffian spline functions. J. Math. Anal. Appl. 33 82-95. MR0290013
86. KOLTCHINSKII, V. (2001). Rademacher penalties and structural risk minimization. IEEE Trans. Inform. Theory 47 1902-1914. MR1842526
87. KONDOR, I. R. and LAFFERTY, J. D. (2002). Diffusion kernels on graphs and other discrete structures. In Proc. International Conf. Machine Learning 315-322. Morgan Kaufmann, San Francisco, CA.
88. KUHN, H. W. and TUCKER, A. W. (1951). Nonlinear programming. Proc. 2nd Berkeley Symposium on Mathematical Statistics and Probabilistics 481-492. Univ. California Press, Berkeley. MR0047303
89. LAFFERTY, J., ZHU, X. and LIU, Y. (2004). Kernel conditional random fields: Representation and clique selection. In Proc. International Conf. Machine Learning 21 64. Morgan Kaufmann, San Francisco, CA.
90. LAFFERTY, J. D., MCCALLUM, A. and PEREIRA, F. (2001). Conditional random fields: Probabilistic modeling for segmenting and labeling sequence data. In Proc. International Conf. Machine Learning 18 282-289. Morgan Kaufmann, San Francisco, CA.
91. LEE, T.-W., GIROLAMI, M., BELL, A. and SEJNOWSKI, T. (2000). A unifying framework for independent component analysis. Comput. Math. Appl. 39 1-21. MR1766376
92. LESLIE, .C, ESKIN, E. and NOBLE, W. S. (2002). The spectrum kernel: A string kernel for SVM protein classification. In Proceedings of the Pacific Symposium on Biocomputing 564-575. World Scientific Publishing, Singapore.
93. LIÈVE, M. (1978). Probability Theory II, 4th ed. Springer, New York. MR0651018
94. MAGERMAN, D. M. (1996). Learning grammatical structure using statistical decision-trees. Proceedings ICGI. Lecture Notes in Artificial Intelligence 1147 1-21. Springer, Berlin.
95. MANGASARIAN, O. L. (1965). Linear and nonlinear separation of patterns by linear programming. Oper. Res. 13 444-452. MR0192918
96. MCCALLUM., A., BELLARE, K. and PEREIRA, F. (2005). A conditional random field for discriminatively- trained finite-state string edit distance. In Conference on Uncertainty in AI (UAI)388. AUAI Press, Arlington, VA.
97. MCCULLAGH, P. and NELD.ER, J. A. (1983). Generalized Linear Models. Chapman and Hall, London. MR0727836
98. MENDELSON, S. (2003). A few notes on statistical learning theory. Advanced Lectures on Machine Learning (S. Mendelson and A. J. Smola, eds.). Lecture Notes in Artificial Intelligence 2600 1-40. Springer, Heidelberg.
99. MERCER, J. (1909). Functions of positive and negative type and their connection with the theory of integral equations. Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. A 209 415-446.
100. MIKA, S., RÄTSCH, G., WESTON, J., SCHÖLKOPF, B., SMOLA, A. J. and MÜLLER, K.-R. (2003). Learning discriminative and invariant nonlinear features. IEEE Trans. Pattern Analysis and Machine Intelligence 25 623-628.
101. MINSKY, M. and PAPERT, S. (1969). Perceptrons: An Introduction to Computational Geometry. MIT Press, Cambridge, MA.
102. MOROZOV, V. A. (1984). Methods for Solving Incorrectly Posed Problems. Springer, New York. MR0766231
103. MURRAY, M. K. and RICE, J. W. (1993). Differential Geometry and Statistics. Chapman and Hall, London. MR1293124
104. OLIVER, N., SCHÖLKOPF, B. and SMOLA, A. J. (2000). Natural regularization in SVMs. In Advances in Large Margin Classifiers (A. J. Smola, P. L. Bartlett, B. Schölkopf and D. Schuurmans, eds.) 51-60. MIT Press, Cambridge, MA. MR1820960
105. O'SULLIVAN, F., YANDELL, B. and RAYNOR, W. (1986). Automatic smoothing of regression functions in generalized linear models. J. Amer. Statist. Assoc. 81 96-103. MR0830570
106. PARZEN, E. (1970). Statistical inference on time series by RKHS methods. In Proceedings 12th Biennial Seminar (R. Pyke, ed.) 1-37. Canadian Mathematical Congress, Montreal. MR0275616
107. PLATT, J. (1999). Fast training of support vector machines using sequential minimal optimization. In Advances in Kernel Methods - Support Vector Learning (B. Scholkopf, C. J. C. Burges and A. J. Smola, eds.) 185-208. MIT Press, Cambridge, MA.
108. POGGIO, T. (1975). On optimal nonlinear associative recall. Biological Cybernetics 19 201-209. MR0503978
109. POGGIO, T. and GIROSI, F. (1990). Networks for approximation and learning. Proceedings of the IEEE 78 1481-1497.
110. PRESS, W. H., TEUKOLSKY, S. A., VETTERLING, W. T. and FLANNERY, B. P. (1994). Numerical Recipes in C. The Art of Scientific Computation. Cambridge Univ. Press. MR1880993
111. RASMUSSEN, C. E. and WILLIAMS, C. K. I. (2006). Gaussian Processes for Machine Learning. MIT Press, Cambridge, MA.
112. RÄTSCH, G., SONNENBURG, S., SRINIVASAN, J., WITTE, H., MÜLLER, K.-R., SOMMER, R. J. and SCHÖLKOPF, B. (2007). Improving the Caenorhabditis elegans genome annotation using machine learning. PLoS Computational Biology 3 e20 doi:10.1371/journal.pcbi.0030020.
113. RÉNYI, A. (1959). On measures of dependence. Acta Math. Acad. Sci. Hungar. 10 441-451. MR0115203
114. ROCKAFELLAR, R. T. (1970). Convex Analysis. Princeton Univ. Press. MR0274683
115. SCHOENBERG, I. J. (1938). Metric spaces and completely monotone functions. Ann. Math. 39 811-841. MR1503439
116. SCHÖLKOPF, B. (1997). Support Vector Learning. R. Oldenbourg Verlag, Munich. Available at http://www.kernel-machines.org.
117. SCHÖLKOPF, B., PLATT, J., SHAWE- TAYLOR, J., SMOLA, A. J. and WILLIAMSON, R. C. (2001). Estimating the support of a high-dimensional distribution. Neural Comput. 13 1443-1471.
118. SCHÖLKOPF, B. and SMOLA, A. (2002). Learning with Kernels. MIT Press, Cambridge, MA.
119. SCHÖLKOPF, B., SMOLA, A. J. and MÜLLER, K.-R. (1998). Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10 1299-1319.
120. SCHÖLKOPF, B., SMOLA, A. J., WILLIAMSON, R. C. and BARTLETT, P. L. (2000). New support vector algorithms. Neural Comput. 12 .1207-1245.
121. SCHÖLKOPF, B., TSUDA, K. and VERT, J.-P. (2004). Kernel Methods in Computational Biology. MIT Press, Cambridge, MA.
122. SHA, F. and PEREIRA, F. (2003). Shallow parsing with conditional random fields. In Proceedings of HLT-NAACL 213-220. Association for Computational Linguistics, Edmonton, Canada.
123. SHAWE-TAYLOR, J. and CRISTIANINI, N. (2004). Kernel Methods for Pattern Analysis. Cambridge Univ. Press.
124. SMOLA, A. J., BARTLETT, P. L., SCHÖLKOPF, B. and SCHUURMANS, D. (2000). Advances in Large Margin Classifiers. MIT Press, Cambridge, MA. MR1820960
125. SMOLA, A.J. and KONDOR, I. R. (2003). Kernels and regularization on graphs. Proc. Annual Conf. Computational Learning Theory (B. Schölkopf and M. K. Warmuth, eds.). Lecture Notes in Comput. Sci. 2726 .144-158. Springer, Heidelberg.
126. SMOLA, A. J. and SCHÖLKOPF, B. (1998). On a kernel-based method for pattern recognition, regression, approximation and operator inversion. Algorithmica 22 211-231. MR1637511
127. SMOLA, A. J., SCHÖLKOPF, B. and MÜLLER, K.-R. (1998). The connection between regularization operators and support vector kernels. Neural Networks 11 637-649.
128. STEINWART, I. (2002). On the influence of the kernel on the consistency of support vector machines. J. Mach. Learn. Res. 2 67-93. MR1883281
129. STEINWART, I. (2002). Support vector machines are universally consistent. J. Complexity 18 768-791. MR1928806
130. STEWART, J. (1976). Positive definite functions and generalizations, an historical survey. Rocky Mountain J. Math. 6 409-434. MR0430674
131. STITSON, M., GAMMERMAN, A., VAPNIK, V., VOVK, V., WATKINS, C. and WESTON, J. (1999). Support vector regression with ANOVA decomposition kernels. In Advances in Kernel Methods - Support Vector Learning (B. Schölkopf, C. J. C. Burges and A. J. Smola, eds.) 285-292. MIT Press, Cambridge, MA.
132. TASKAR, B., GUESTRIN, C. and KOLLER, D. (2004). Max-margin Markov networks. In Aavances in Neural Information Processing Systems 16 (S. Thrun, L. Saul and B. Schölkopf, eds.) 25-32. MIT Press, Cambridge, MA.
133. TASKAR, B., KLEIN, D., COLLINS, M., KOLLER, D. and MANNING, C. (2004). Max-margin parsing. In Empirical Methods in Natural Language Processing 1-8. Association for Computational Linguistics, Barcelona, Spain.
134. TAX, D. M. J. and DUIN, R. P. W. (1999). Data domain description by support vectors. In Proceedings ESANN (M. Verleysen, ed.) 251-256. D Facto, Brussels.
135. TIBSHIRANI, R. (1996). Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B Stat. Methodol. 58 267-288. MR1379242
136. TIKHONOV, A. N. (1963). Solution of incorrectly formulated problems and the regularization method. Soviet Math. Dokl. 4 1035-1038.
137. TSOCHANTARIDIS, I., JOACHIMS, T., HOFMANN, T. and ALTUN, Y. (2005). Large margin methods for structured and interdependent output variables. J. Mach. Learn. Res. 6 1453-1484. MR2249862
138. VAN RIJSBERGEN, C. (1979). Information Retrieval, 2nd ed. Butterworths, London.
139. VAPNIK, V. (1982). Estimation of Dependences Based on Empirical Data. Springer, Berlin. MR0672244
140. VAPNIK, V. (1995). The Nature of Statistical Learning Theory. Springer, New York. MR1367965
141. VAPNIK, V. (1998). Statistical Learning Theory. Wiley, New York. MR1641250
142. VAPNIK, V. and CHERVONENKIS, A. (1971). On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab. Appl. 16 264-281.
143. VAPNIK, V. and CHERVONENKIS, A. (1991). The necessary and sufficient conditions for consistency in the empirical risk minimization method. Pattern Recognition and Image Analysis 1 283-305.
144. VAPNIK, V., GOLOWICH, S. and SMOLA, A. J. (1997). Support vector method for function approximation, regression estimation, and signal processing. In Advances in Neural Information Processing Systems 9 (M. C. Mozer, M. I. Jordan and T. Petsche, eds.) 281-287. MIT Press, Cambridge, MA.
145. VAPNIK, V. and LERNER, A. (1963). Pattern recognition using generalized portrait method. Autom. Remote Control 24 774-780.
146. VISHWANATHAN, S. V. N. and SMOLA, A. J. (2004). Fast kernels for string and tree matching. In Kernel Methods in Computational Biology (B. Schölkopf, K. Tsuda and J. P. Vert, eds.) 113-130. MIT Press, Cambridge, MA.
147. VISHWANATHAN, S. V. N., SMOLA, A. J. and VIDAL, R. (2007). Binet-Cauchy kernels on dynamical systems and its application to the analysis of dynamic scenes. Internat. J. Computer Vision 73 95-119.
148. WAHBA, G. (1990). Spline Models for Observational Data. SIAM, Philadelphia. MR1045442
149. WAHBA, G., WANG, Y., GU, C., KLEIN, R. and KLEIN, B. (1995). Smoothing spline ANOVA for exponential families, with application to the Wisconsin Epidemiological Study of Diabetic Retinopathy. Ann. Statist. 23 1865-1895. MR1389856
150. WAINWRIGHT, M. J. and JORDAN, M. I. (2003). Graphical models, exponential families, and variational inference. Technical Report 649, Dept. Statistics, Univ. California, Berkeley.
151. WATKLNS, C. (2000). Dynamic alignment kernels. In Advances in Large Margin Classifiers (A. J. Smola, P. L. Bartlett, B. Schölkopf and. D. Schuurmans, eds.) 39-50. MIT Press, Cambridge, MA. MR1820960
152. WENDLAND, H. (2005). Scattered Data Approximation. Cambridge Univ. Press. MR2131724
153. WESTON, J., CHAPELLE, O., ELISSEEFF, A., SCHÖLKOPF, B. and VAPNIK, V. (2003). Kernel dependency estimation. In Advances in Neural Information Processing Systems 15 (S. T. S. Becker and K. Obermayer, eds.) 873-880. MIT Press, Cambridge, MA. MR1820960
154. WHITTAKER, J. (1990). Graphical Models in Applied Multivariate Statistics. Wiley, New York. MR1112133
155. YANG, H. H. and AMARI, S.-I. (1997). Adaptive on-line learning algorithms for blind separation - maximum entropy and minimum mutual information. Neural Comput. 9 1457-1482.
156. ZETTLEMOYER, L. S. and COLLINS, M. (2005). Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In Uncertainty in Artificial Intelligence UAI 658-666. AUAI Press, Arlington, Virginia.
157. ZIEN, A., RATSCH, G., MIKA, S., SCHÖLKOPF, B., LENGAUER, T. and MÜLLER, K.-R. (2000). Engineering support vector machine kernels that recognize translation initiation sites. Bioinformatics 16 799-807.