Finite-dimensional projection for classification and statistical learning

IEEE Transactions on Information Theory

Volumn 54, Issue 9, 2008, Pages 4169-4182

  Blanchard, Gilles ^a   Zwald, Laurent ^b  

^a FRAUNHOFER FIRST   (Germany)

^b UNIV GRENOBLE ALPES   (France)

Author keywords

Classification; Dimension reduction; Kernel principal component analysis (KPCA); Regularization; Statistical learning; Support vector machine (SVM)

Indexed keywords

RISK ANALYSIS;

CLASSIFICATION; DIMENSION REDUCTION; KERNEL PRINCIPAL COMPONENT ANALYSIS (KPCA); REGULARIZATION; STATISTICAL LEARNING; SUPPORT VECTOR MACHINE (SVM);

CONVERGENCE OF NUMERICAL METHODS;

EID: 51349154782     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2008.926312     Document Type: Article

References (38)

1. V. N. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
2. G.Wahba, Spline Models for Observational Data, ser. CBMS-NSF Regional Conference Series in Applied Mathematics. Philadelphia, PA: SIAM, 1990, vol. 59.
3. A. Barron, L. Birgé, and P. Massart, "Risk bounds for model selection via penalization," Probab. Theory Relat. Fields, vol. 113, pp. 301-413, 1999.
4. Y. Baraud, "Model selection for regression on a random design," ESAIM Probab. Statist., vol. 6, pp. 127-146, 2002.
5. P. Massart, Concentration Inequalities and Model Selection, ser. Lecture Notes in Mathematics. Berlin, Germany: Springer-Verlag, 2007, vol. 1896.
6. P. Massart, "Some applications of concentration inequalities to statistics," Annales de la Faculté des Sciences de Toulouse, vol. IX, pp. 245-303, 2000.
7. P. Bartlett, O. Bousquet, and S. Mendelson, "Local Rademacher complexities," Ann. Statist., vol. 33, no. 4, pp. 1497-537, 2005.
8. G. Blanchard, P. Massart, R. Vert, and L. Zwald, L. K. Saul, Y.Weiss, and L. Bottou, Eds., "Kernel projection machine: A newtool for pattern recognition," in Advances in Neural Information Processing System 17 Cambridge, MA: MIT Press, 2005, pp. 1649-1656.
9. Y. Lin, "Support vector machines and the Bayes rule in classification," Data Mining Knowl. Disc., vol. 6, pp. 259-275, 2002.
10. T. Zhang, "Statistical behavior and consistency of classification methods based on convex risk minimization," Ann. Statist., vol. 32, no. 1, pp. 56-85, 2004.
11. L. Györfi, M. Kohler, A. Krzyÿzak, and H. Walk, A Distribution-Free Theory of Nonparametric Regression, ser. Statistics. New York: Springer Verlag, 2002.
12. P. L. Bartlett, "The sample complexity of pattern classification with neural networks: The size of the weights is more important than the size of the network," IEEE Trans. Inf. Theory, vol. 44, no. 2, pp. 525-536, Mar. 1998.
13. D.-R. Chen, Q.Wu, Y. Ying, and D.-X. Zhou, "Support vector machine soft margin classifiers: Error analysis," J. Mach. Learn. Res., vol. 5, pp. 1143-1175, 2004.
14. P. Massart and E. Nédélec, "Risk bounds for statistical learning," Ann. Statist., vol. 34, no. 5, pp. 2326-2366, 2006.
15. E. Mammen and A. B. Tsybakov, "Asymptotical minimax recovery of sets with smooth boundaries," Ann. Statist., vol. 23, no. 2, pp. 502-524, 1995.
16. A. Tsybakov, "Optimal aggregation of classifiers in statistical learning," Ann. Statist., vol. 32, no. 1, pp. 135-166, 2004.
17. A. Tsybakov and S. van de Geer, "Square root penalty: Adaptation to the margin in classification and in edge estimation," Ann. Statist., vol. 33, no. 3, pp. 1203-1224, 2005.
18. 1 penalty, " Bernoulli, vol. 12, pp. 1045-1076, 2006.
19. J.-Y. Audibert, "A randomized online learning algorithm for better variance control," in Proc. 19th Conf. Comput. Learn. Theory, ser. Lecture Notes in Computer Science, G. Lugosi and H. U. Simon, Eds. Berlin, Germany: Springer-Verlag, 2006, vol. 4005, pp. 392-407.
20. C. Scott and R. Nowak, "Minimax optimal classification with dyadic decision trees," IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1335-1353, Apr. 2006.
21. I. Steinwart and C. Scovel, "Fast rates for support vector machines using Gaussian kernels," Ann. Statist., vol. 35, pp. 575-607, 2007.
22. E. De Vito, L. Rosasco, A. Caponnetto, U. De Giovannini, and F. Odone, "Learning from examples as an inverse problem," J. Mach. Learn. Res., vol. 6, pp. 883-904, 2005.
23. G. Blanchard, O. Bousquet, and P. Massart, "Statistical performance of support vector machines," Ann. Statist., to be published.
24. T. Evgeniou, M. Pontil, and T. Poggio, "Regularization networks and support vector machines," in Advances in Large Margin Classifiers A. J. Smola, P. L. Bartlett, B. Schölkopf, and D. Schuurmans, Eds. Cambridge, MA: MIT Press, 2000, pp. 171-203.
25. A. J. Smola and B. Schölkopf, "On a kernel-based method for pattern recognition, regression, approximation and operator inversion," Algorithmica, vol. 22, pp. 211-231, 1998.
26. Baker, The Numerical Treatment of Integral Equations. Oxford, U.K.: Clarendon, 1977.
27. B. Schölkopf, A. J. Smola, and K.-R. Müller, "Nonlinear component analysis as a kernel eigenvalue problem," Neural Comput., vol. 10, pp. 1299-1319, 1998.
28. V. Koltchinskii, "Asymptotics of spectral projections of some random matrices approximating integral operators," Progr. Probab., vol. 43, pp. 191-227, 1998.
29. J. Dauxois, A. Pousse, and Y. Romain, "Asymptotic theory for the principal component analysis of a vector random function: Some applications to statistical inference," J. Multivariate Anal., vol. 12, pp. 136-154, 1982.
30. L. Zwald and G. Blanchard, "On the convergence of eigenspaces in kernel principal component analysis," in Advances in Neural Information Processing Systems 18, Y. Weiss, L. Bottou, and J. Platt, Eds. Cambridge, MA: MIT Press, 2006, pp. 1649-1656.
31. G. Raetsch, Benchmark repository used in several Boosting, KFD, and SVM papers, 1999 [Online]. Available: http://ida.first.gmd.de/~raetsch/ data/benchmarks.htm
32. N. Bissantz, T. Hohage, A. Munk, and F. Ruymgaart, "Convergence rates of general regularization methods for statistical inverse problems and applications," SIAM J. Numer. Anal., vol. 45, pp. 2610-2636, 2007.
33. V. Vapnik, Estimation of Dependences Based on Empirical Data, ser. Statistics. New York: Springer-Verlag, 1982.
34. O. Bousquet, "Concentration inequalities and empirical processes theory applied to the analysis of learning algorithms," Ph.D. dissertation, Dept. Appl. Math., École Polytechnique, Palaiseau, France, 2002.
35. A. W. van der Vaart and J. A. Wellner, Weak Convergence and Empirical Processes. New York: Springer-Verlag, 1996.
36. R. M. Dudley, Uniform Central Limit Theorems, ser. Cambridge Studies in Advanced Mathematics. Cambridge, U.K.: Cambridge Univ. Press, 1999.
37. H. Zhu, C. Williams, R. Rohwer, and M. Morciniec, "Gaussian regression and optimal finite dimensional linear models," in Neural Networks and Machine Learning, C. M. Bishop, Ed. New York: Springer-Verlag, 1998, pp. 167-184.
38. C. K. I. Williams and M. Seeger, P. Langley, Ed., "The effect of the input density distribution on kernel-based classifiers," in Proc. 17th Int. Conf. Mach. Learn., San Francisco, CA, 2000, pp. 1159-1166.