Statistical performance of support vector machines

Annals of Statistics

Volumn 36, Issue 2, 2008, Pages 489-531

  Blanchard, Gilles ^a,d   Bousquet, Olivier ^b,e   Massart, Pascal ^c  

^a Fraunhofer Insitute FIRST ^*

^b GOOGLE INC   (United States)

^c UNIVERSITÉ PARIS SACLAY   (France)

^d FRAUNHOFER FIRST   (Germany)

^e GOOGLE   (Switzerland)

Author keywords

Classification; Model selection; Oracle inequality; Support vector machine

Indexed keywords

EID: 48849115978     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/009053607000000839     Document Type: Article

References (42)

1. ALON, N., BEN-DAVID, S., CESA- BIANCHI, N. and HAUSSLER, D. (1997). Scale-sensitive dimensions, uniform convergence, and learnability. J. ACM 44 615-631. MR1481318
2. BARTLETT, P. (1998). The sample complexity of pattern classification with neural networks: The size of the weights is more important than the size of the network. IEEE Trans. Inform. Theory 44 525-536. MR1607706
3. BARTLETT, P., BOUSQUET, O. and MENDELSON, S. (2005). Local Rademacher complexities. Ann. Statist. 33 1497-1537. MR2166554
4. BARTLETT, P., JORDAN, M. and MCA ULIFFE, J. (2006). Convexity, classification, and risk bounds. J. Amer. Statist. Assoc. 101 138-156. MR2268032
5. BARTLETT, P. and MENDELSON, S. (2002). Rademacher and Gaussian complexities: Risk bounds and structural results. J. Machine Learning Research 3 463-482. MR1984026
6. BARTLETT, P. and MENDELSON, S. (2006). Empirical minimization. Probab. Theory Related Fields 135 311-334. MR2240689
7. α. Mat. USSR Sb. 73 295-317. MR0217487
8. BLANCHARD, G., BOUSQUET. O. and ZWALD, L. (2007). Statistical properties of kernel principal component analysis. Machine Learning 66 259-294.
9. BLANCHARD, G., LUGOSI, G. and VAYATIS, N. (2003). On the rate of convergence of regularized boosting classifiers. J. Machine Learning Research 4 861-894. MR2076000
10. BOUSQUET, O. (2002). A Bennett concentration inequality and its application to suprema of empirical processes. C. R. Acad. Sci. Paris Ser. I 334 495-500. MR1890640
11. BOUSQUET, O. and ELISSEEFF, A. (2002). Stability and generalization. J. Machine Learning Research 2 499-526. MR1929416
12. CHEN, D.-R., WU, Q., YING, Y. and ZHOU, D.-X. (2004). Support vector machine soft margin classifiers: Error analysis. J. Machine Learning Research 5 1143-1175. MR2248013
13. CRISTIANINI, N. and SHAWE-TAYLOR, J. (2000). An Introduction to Support Vector Machines. Cambridge Univ. Press.
14. EDMUNDS, D. E. and TRIEBEL, H. (1996). Function Spaces, Entropy Numbers, Differential Operators. Cambridge Univ. Press. MR 1410258
15. EVGENIOU, T., PONTIL, M. and POGGIO, T. (2000). Regularization networks and support vector machines. Adv. Comput. Math. 13 1-50. MR 1759187
16. KOLTCHINKSII, V. (2006). Local Rademacher complexities and oracle inequalities in risk minimization. Ann. Statist. 34 2593-2656. MR2329442
17. KOLTCHINSKII, V. and PANCHENKO, D. (2002). Empirical margin distributions and bounding the generalization error of combined classifiers. Ann. Statist. 30 1-50. MR 1892654
18. LECUÉ, G. (2007). Simultaneous adaptation to the margin and to complexity in classification. Ann. Statist. 35 1698-1721.
19. LIN, Y. (2002). Support vector machines and the Bayes rule in classification. Data Mining and Knowledge Discovery 6 259-275. MR1917926
20. LUGOSI, G. and WEGKAMP, M. (2004). Complexity regularization via localized random penalties. Ann. Statist. 32 1679-1697. MR2089138
21. MASSART, P. (2000). About the constants in Talagrand's inequality for empirical processes. Ann. Probab. 28 863-884. MR1782276
22. MASSART, P. (2000). Some applications of concentration inequalities in statistics. Ann. Fac. Sci. Toulouse Math. 9 245-303. MR 1813803
23. MASSART, P. and NÉDELEC, E. (2006). Risk bounds for statistical learning. Ann. Statist. 34 2326-2366. MR2291502
24. MASSART, P. (2007). Concentration Inequalities and Model Selection. Lectures on Probability Theory and Statistics. Ecole d' Été de Probabilités de Saint-Flour XXXIII - 2003. Lecture Notes in Math. 1896. Springer, Berlin. MR2319879
25. MENDELSON, S. (2003). Estimating the performance of kernel classes. J. Machine Learning Research 4 759-771. MR2075996
26. DE VITO, E., ROSASCO, L., CAPONNETTO, A., DE GIOVANNI, U. and ODONE, F. (2005). Learning from examples as an inverse problem. J. Machine Learning Research 6 883-904. MR2249842
27. SCHOLKOPF, B., SMOLA, A. J. and WILLIAMSON, R. C. (2001). Generalization performance of regularization networks and support vector machines via entropy numbers of compact operators. IEEE Trans. Inform. Theory 47 2516-2532. MR 1873936
28. SHAWE-TAYLOR, J., WILLIAMS. C., CRISTIANINI, N. and KANDOLA, J. (2005). On the eigenspectrum of the Gram matrix and the generalisation error of kernel PCA. IEEE Trans. Inform. Theory 51 2510-2522. MR2246374
29. SCHÖLKOPF, B. and SMOLA, A. (2002). Learning with Kernels. MIT Press.
30. SMOLA, A. and SCHÖLKOPF, B. (1998). From regularization operators to support vector kernels. In Advances in Neural Information Processings Systems 10 (M. I. Jordan. M. J. Kearns and S. A. Solla, eds.) 343-349. MIT Press.
31. STEINWART, I. (2002). Support vector machines are universally consistent. J. Complexity 18 768-791. MR 1928806
32. STEINWART, I. and SCOVEL, C. (2007). Fast rates for support vector machines using Gaussian kernels. Ann. Statist. 35 575-607. MR2336860
33. STEINWART. I., HUSH, D. and SCOVEL, C. (2006). A new concentration result for regularized risk minimizers. In High-Dimensional ProbabilityIV260-275.
34. IMS Lecture Notes Monograph Series 51.
35. 1 penalty. Bernoulli 12 1045-1076. MR2274857
36. TSYBAKOV, A. (2004). Optimal aggregation of classifiers in statistical learning. Ann. Statist. 32 135-166. MR2051002
37. TSYBAKOV, A. and VAN DE GEER, S. (2005). Square root penalty: Adaptation to the margin in classification and in edge estimation. Ann. Statist. 33 1203-1224. MR2195633
38. VAPNIK, V. (1998). Statistical Learning Theory: Wiley, New York. MR1641250
39. VAPNIK, V. and CHERVONENKIS, A. (1971). On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab. Appl. 16 264-280.
40. YANG, Y. (1999). Minimax nonparametric classification. I. Rates of convergence. IEEE Trans. Inform. Theory 45 2271-2284. MR1725115
41. ZHANG, T. (2004). Statistical behavior and consistency of classification methods based on convex risk minimization. Ann. Statist. 32 56-85. MR2051001
42. ZHOU. D.-X. (2003). Capacity of reproducing kernel spaces in learning theory. IEEE Trans. Inform. Theory491743-1752. MR1985575