Multi-kernel regularized classifiers

Journal of Complexity

Volumn 23, Issue 1, 2007, Pages 108-134

  Wu, Qiang ^a   Ying, Yiming ^a   Zhou, Ding Xuan ^a  

^a CITY UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Classification algorithm; Convex loss function; Misclassification error; Multi kernel regularization scheme; Regularization error and sample error

Indexed keywords

CONVERGENCE OF NUMERICAL METHODS; ERROR ANALYSIS; LEAST SQUARES APPROXIMATIONS; OPTIMIZATION; PROBLEM SOLVING;

CLASSIFICATION ALGORITHMS; CONVEX LOSS FUNCTION; MISCLASSIFICATION ERROR; MULTI-KERNEL REGULARIZATION SCHEME; TIKHONOV REGULARIZATION SCHEMES;

ALGORITHMS;

EID: 33846804061     PISSN: 0885064X     EISSN: 10902708     Source Type: Journal    
DOI: 10.1016/j.jco.2006.06.007     Document Type: Article

References (47)

1. Aronszajn N. Theory of reproducing kernels. Trans. Amer. Math. Soc. 68 (1950) 337-404
2. Bartlett P.L., Bousquet O., and Mendelson S. Local Rademacher complexities. Ann. Statist. 33 (2005) 1497-1537
3. Bartlett P.L., Jordan M.I., and McAuliffe J.D. Convexity, classification, and risk bounds. J. Amer. Statist. Assoc. 101 (2006) 138-156
4. B. Blanchard, O. Bousquet, P. Massart, Statistical performance of support vector machines, preprint, 2003.
5. Blanchard B., Lugosi G., and Vayatis N. On the rate of convergence of regularized boosting classifiers. J. Mach. Learning Res. 4 (2003) 861-894
6. Boucheron S., Bousquet O., and Lugosi G. Theory of classification: a survey of some recent advances. ESAIM: Probab. Statist. 9 (2005) 323-375
7. Bousquet O., and Elisseeff A. Stability and generalization. J. Mach. Learning Res. 2 (2002) 499-526
8. Chapelle O., Vapnik V., Bousquet O., and Mukherjee S. Choosing multiple parameters for support vector machines. Mach. Learning 46 (2002) 131-159
9. Chen D.R., Wu Q., Ying Y., and Zhou D.X. Support vector machine soft margin classifiers: error analysis. J. Mach. Learning Res. 5 (2004) 1143-1175
10. Cortes C., and Vapnik V. Support-vector networks. Mach. Learning 20 (1995) 273-297
11. Cristianini N., and Shawe-Taylor J. An Introduction to Support Vector Machines (2000), Cambridge University Press, Cambridge, MA
12. Cucker F., and Smale S. On the mathematical foundations of learning. Bull. Amer. Math. Soc. 39 (2001) 1-49
13. Cucker F., and Smale S. Best choices for regularization parameters in learning theory: on the bias-variance problem. Found. Comput. Math. 2 (2002) 413-428
14. F. Cucker, D.X. Zhou, Learning Theory: An Approximation Theory Viewpoint, Cambridge University Press, Cambridge, MA, in press.
15. Devroye L., Györfi L., and Lugosi G. A Probabilistic Theory of Pattern Recognition (1997), Springer, New York
16. T. Evgeniou, M. Pontil, Regularized multi-task learning, Proceedings of the 17th SIGKDD Conference on Knowledge Discovery and Data Mining, 2004.
17. Evgeniou T., Pontil M., and Poggio T. Regularization networks and support vector machines. Adv. Comput. Math. 13 (2000) 1-50
18. Lanckriet G.R.G., Cristianini N., Bartlett P., El Ghaoui L., and Jordan M.I. Learning the kernel matrix with semidefinite programming. J. Mach. Learning Res. 5 (2004) 27-72
19. Lee W.S., Bartlett P.L., and Williamson R.C. Efficient agnostic learning of neural networks with bounded fan-in. IEEE Trans. Inform. Theory 42 (1996) 2118-2132
20. Li J., and Barron A. Mixture density estimation. In: Solla S.A., Leen T.K., and Muller K.R. (Eds). Advances in Neural Information Processing Systems vol. 12 (1999), Morgan Kaufmann Publishers, San Mateo
21. Lin Y. Support vector machines and the Bayes rule in classification. Data Mining Knowledge Discovery 6 (2002) 259-275
22. Lugosi G., and Vayatis N. On the Bayes-risk consistency of regularized boosting methods. Ann. Statist. 32 (2004) 30-55
23. Massart P. Some applications of concentration inequalities to statistics. Ann. Fac. Sci. Toulouse Ser. 6 9 (2000) 245-303
24. Mendelsen S. Improving the sample complexity using global data. IEEE Trans. Inform. Theory 48 (2002) 1977-1991
25. Micchelli C.A., and Pontil M. Learning the kernel function via regularization. J. Mach. Learning Res. 6 (2005) 1099-1125
26. Mukherjee S., Rifkin R., and Poggio T. Regression and classification with regularization. In: Denison D.D., Hansen M.H., Holmes C.C., Mallick B., and Yu B. (Eds). Lecture Notes in Statistics: Nonlinear Estimation and Classification (2002), Springer, New York 107-124
27. Rakhlin A., Panchenko D., and Mukherjee S. Risk bounds for mixture density estimation. ESAIM: Probab. Statist. 9 (2005) 220-229
28. Shawe-Taylor J., Bartlett P.L., Williamson R.C., and Anthony M. Structural risk minimization over data-dependent hierarchies. IEEE Trans. Inform. Theory 44 (1998) 1926-1940
29. Smale S., and Zhou D.X. Estimating the approximation error in learning theory. Anal. Appl. 1 (2003) 17-41
30. Smale S., and Zhou D.X. Shannon sampling and function reconstruction from point values. Bull. Amer. Math. Soc. 41 (2004) 279-305
31. S. Smale, D.X. Zhou, Learning theory estimates via integral operators and their applications, Constr. Approx., in press.
32. Steinwart I. On the influence of the kernel on the consistency of support vector machines. J. Mach. Learning Res. 2 (2001) 67-93
33. Steinwart I. Support vector machines are universally consistent. J. Complexity 18 (2002) 768-791
34. I. Steinwart, C. Scovel, Fast rates for support vector machines, in: Proceedings of the Conference on Learning Theory (COLT-2005), 2005, pp. 279-294.
35. Suykens J.A.K., and Vandewalle J. Least squares support vector machine classifiers. Neural Process. Lett. 9 (1999) 293-300
36. Tsybakov A.B. Optimal aggregation of classifiers in statistical learning. Ann. Statist. 32 (2004) 135-166
37. van der Vaart A.W., and Wellner J.A. Weak Convergence and Empirical Processes (1996), Springer, New York
38. Vapnik V. Statistical Learning Theory (1998), Wiley, New York
39. Wahba G. Spline Models for Observational Data (1990), SIAM, Philadelphia, PA
40. Wu Q., and Zhou D.X. SVM soft margin classifiers: linear programming versus quadratic programming. Neural Comput. 17 (2005) 1160-1187
41. Wu Q., and Zhou D.X. Analysis of support vector machine classification. J. Comput. Anal. Appl. 8 (2006) 99-119
42. Y. Ying, D.X. Zhou, Learnability of Gaussians with flexible variances, preprint, 2004.
43. Zhang T. On the dual formulation of regularized linear systems with convex risks. Mach. Learning 46 (2002) 91-129
44. Zhang T. Statistical behavior and consistency of classification methods based on convex risk minimization. Ann. Statist. 32 (2004) 56-85
45. Zhou D.X. The covering number in learning theory. J. Complexity 18 (2002) 739-767
46. Zhou D.X. Capacity of reproducing kernel spaces in learning theory. IEEE Trans. Inform. Theory 49 (2003) 1743-1752
47. D.X. Zhou, Density problem and approximation error in learning theory, preprint, 2003.