Learning kernel-based halfspaces with the 0-1 loss

SIAM Journal on Computing

Volumn 40, Issue 6, 2011, Pages 1623-1646

  Shalev Shwartz, Shai ^a   Shamir, Ohad ^a,b   Sridharan, Karthik ^c  

^a HEBREW UNIVERSITY OF JERUSALEM   (Israel)

^b MICROSOFT RESEARCH   (United States)

^c TOYOTA TECHNOLOGICAL INSTITUTE AT CHICAGO   (United States)

Author keywords

Kernel methods; Learning halfspaces; Learning theory

Indexed keywords

CRYPTOGRAPHIC ASSUMPTIONS; FINITE TIME; HALF SPACES; HALF-SPACE; HARDNESS RESULT; KERNEL METHODS; LEARNING THEORY; LIPSCHITZ CONSTANT; LOGISTIC REGRESSIONS; LOSS FUNCTIONS; TIME POLYNOMIALS;

SUPPORT VECTOR MACHINES;

LEARNING ALGORITHMS;

EID: 84855575451     PISSN: 00975397     EISSN: None     Source Type: Journal    
DOI: 10.1137/100806126     Document Type: Conference Paper

References (29)

1. C. THOMAS-AGNAN AND A. BERLINET, Reproducing Kernel Hilbert Spaces in Probability and Statistics, Springer, Berlin, New York, 2003.
2. N. I. ACHIESER, Theory of Approximation, Frederick Ungar Publishing, New York, 1956.
3. P. L. BARTLETT, For valid generalization, the size of the weights is more important than the size of the network, in Advances in Neural Information Processing Systems 9, MIT Press, Cambridge, MA, 1997, pp. 134-140.
4. P. L. BARTLETT, M. I. JORDAN, AND J. D. MCAULIFFE, Convexity, classification, and risk bounds, J. Amer. Statist. Assoc., 101(2006), pp. 138-156. (Pubitemid 43500031)
5. P. L. BARTLETT AND S. MENDELSON, Rademacher and Gaussian complexities: Risk bounds and structural results, J. Mach. Learn. Res., 3(2002), pp. 463-482.
6. S. BEN-DAVID, Alternative measures of computational complexity with applications to agnostic learning, in Theory and Applications of Models of Computation, Lecture Notes in Comput. Sci. 3959, Springer, Berlin, 2006, pp. 231-235. (Pubitemid 44030095)
7. S. BEN-DAVID AND H. SIMON, Efficient learning of linear perceptrons, in Advances in Neural Information Processing Systems 13, MIT Press, Cambridge, MA, 2000, pp. 189-195.
8. E. BLAIS, R. O'DONNELL, AND K. WIMMER, Polynomial regression under arbitrary product distributions, in Proceedings of the 21st Annual Conference on Learning Theory, R. Servedio and T. Zhang, eds., Omnipress, Madison, WI, 2008, pp. 193-204.
9. L. BOTTOU AND O. BOUSQUET, The tradeoffs of large scale learning, in Advances in Neural Information Processing Systems 20, MIT Press, Cambridge, MA, 2008, pp. 161-168.
10. O. BOUSQUET, Concentration Inequalities and Empirical Processes Theory Applied to the Analysis of Learning Algorithms, Ph. D. thesis, École Polytechnique, Palaiseau, France, 2002.
11. N. CRISTIANINI AND J. SHAWE-TAYLOR, Kernel Methods for Pattern Analysis, Cambridge University Press, Cambridge, UK, 2004.
12. D. ELLIOT, The evaluation and estimation of the coefficients in the Chebyshev series expansion of a function, Math. Comp., 18(1964), pp. 274-284.
13. V. FELDMAN, P. GOPALAN, S. KHOT, AND A. K. PONNUSWAMI, New results for learning noisy parities and halfspaces, in Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science, IEEE Press, Piscataway, NJ, 2006, pp. 563-574.
14. V. GURUSWAMI AND P. RAGHAVENDRA, Hardness of learning halfspaces with noise, in Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS), IEEE Press, Piscataway, NJ, 2006, pp. 543-552. (Pubitemid 351175542)
15. S. KAKADE, K. SRIDHARAN, AND A. TEWARI, On the complexity of linear prediction: Risk bounds, margin bounds, and regularization, in Advances in Neural Information Processing 21, MIT Press, Cambridge, MA, 2009, pp. 793-800.
16. A. KALAI, A. R. KLIVANS, Y. MANSOUR, AND R. SERVEDIO, Agnostically learning halfspaces, in Proceedings of the 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS), IEEE Press, Piscataway, NJ, 2005, pp. 11-20. (Pubitemid 44375715)
17. M. KEARNS, R. SCHAPIRE, AND L. SELLIE, Toward efficient agnostic learning, Machine Learning, 17(1994), pp. 115-141.
18. A. KLIVANS AND A. SHERSTOV, Cryptographic hardness for learning intersections of halfspaces, in Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science, IEEE Press, Piscataway, NJ, 2006, pp. 553-562. (Pubitemid 351175543)
19. N. N. LEBEDEV, Special Functions and Their Applications, Dover, New York, 1972.
20. J. MASON, Chebyshev Polynomials, CRC Press, Boca Raton, FL, 2003.
21. D. MCALLESTER, Simplified PAC-Bayesian margin bounds, in Proceedings of the 16th Annual Conference on Computational Learning Theory (COLT), Lecture Notes in Comput. Sci. 2777, Springer, Berlin, New York, 2003, pp. 203-215.
22. R. SCHAPIRE, The strength of weak learnability, Machine Learning, 5(1990), pp. 197-227.
23. B. SCHÖLKOPF AND A. SMOLA, Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond, MIT Press, Cambridge, MA, 2002.
24. S. SHALEV-SHWARTZ AND N. SREBRO, SVM optimization: Inverse dependence on training set size, in Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland, 2008, pp. 928-935.
25. I. STEINWART AND C. SCOVEL, Fast rates for support vector machines using Gaussian kernels, Ann. Statist., 35(2007), pp. 575-607.
26. A. TSYBAKOV, Optimal aggregation of classifiers in statistical learning, Ann. Statist., 32(2004), pp. 135-166. (Pubitemid 41449306)
27. V. N. VAPNIK, Statistical Learning Theory, John Wiley, New York, 1998.
28. G. WAHBA, Spline Models for Observational Data, CBMS-NSF Reg. Conf. Ser. Appl. Math. 59, SIAM, Philadelphia, 1990.
29. T. ZHANG, Statistical behavior and consistency of classification methods based on convex risk minimization, Ann. Statist., 32(2004), pp. 56-85.