Data-dependent margin-based generalization bounds for classification

Journal of Machine Learning Research

Volumn 3, Issue 1, 2003, Pages 73-98

  Antos, András ^a   Kégl, Balázs ^b   Linder, Tamás ^a   Lugosi, Gábor ^c  

^a QUEEN S UNIVERSITY   (Canada)

^b UNIVERSITÉ DE MONTRÉAL   (Canada)

^c UNIVERSITAT POMPEU FABRA   (Spain)

Author keywords

Classification; Error estimation; Fat shattering dimension; Margin based bounds

Indexed keywords

ERROR ANALYSIS; NEURAL NETWORKS; PROBABILITY;

ERROR ESTIMATION;

LEARNING SYSTEMS;

EID: 0042967689     PISSN: 15324435     EISSN: None     Source Type: Journal    
DOI: 10.1162/153244303768966111     Document Type: Article

References (20)

1. N. Alon, S. Ben-David, N. Cesa-Bianchi, and D. Haussler. Scale-sensitive dimensions, uniform convergence, and learnability. Journal of the ACM, 44:615-631, 1997.
2. M. Anthony and P. L. Bartlett. Neural Network Learning: Theoretical Foundations. Cambridge University Press, Cambridge, 1999.
3. P. Bartlett, S. Boucheron, and G. Lugosi. Model selection and error estimation. Machine Learning, 48:85-113, 2001.
4. P.L. Bartlett. The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. IEEE Transactions on Information Theory, 44:525-536, 1998.
5. P.L. Bartlett and S. Mendelson. Rademacher and gaussian complexities: risk bounds and structural results. Journal of Machine Learning Research, to appear, 2002.
6. C. Blake, E. Keogh, and C. Merz. Uci repository of machine learning databases, http://www.ics.uci.edu/̃mlearn/MLRepository.html, 1998.
7. S. Boucheron, G. Lugosi, and P. Massart. A sharp concentration inequality with applications. Random Structures and Algorithms, 16:277-292, 2000.
8. L. Devroye and G. Lugosi. Combinatorial Methods in Density Estimation. Springer-Verlag, New York, 2000.
9. T. Erlebach, K. Jansen, and E. Seidel. Polynomial-time approximation schemes for geometric graphs. In Proceedings of the 12th ACM-SIAM Symposium on Discrete Algorithms SODA'01, pages 671-679, 2001.
10. 1-ε. In Proceedings of the 37th Annual Symposium on Foundations of Computer Science FOCS'96, pages 627-636, 1996.
11. W. Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58:13-30, 1963.
12. M. Kearns and R.E. Schapire. Efficient distribution-free learning of probabilistic concepts. Journal of Computer Systems Sciences, 48:464-497, 1994.
13. V. Koltchinskii and D. Panchenko. Empirical margin distributions and bounding the generalization error of combined classifiers. Annals of Statistics, 30, 2002.
14. M. Ledoux and M. Talagrand. Probability in Banach Space. Springer-Verlag, New York, 1991.
15. C. McDiarmid. On the method of bounded differences. In Surveys in Combinatorics 1989, pages 148-188. Cambridge University Press, Cambridge, 1989.
16. J. Shawe-Taylor, P.L. Bartlett, R.C. Williamson, and M. Anthony. Structural risk minimization over data-dependent hierarchies. IEEE Transactions on Information Theory, 44:1926-1940, 1998.
17. J. Shawe-Taylor and R.C. Williamson. Generalization performance of classifiers in terms of observed covering numbers. In H. U. Simon P. Fischer, editor, Computational Learning Theory: Proceedings of the Fourth European Conference, EuroCOLT'99, pages 153-167. Springer, Berlin, 1999. Lecture Notes in Artificial Intelligence 1572.
18. E.V. Slud. Distribution inequalities for the binomial law. Annals of Probability, 5:404-412, 1977.
19. V.N. Vapnik. Statistical Learning Theory. John Wiley, New York, 1998.
20. V.N. Vapnik and A.Ya. Chervonenkis. Theory of Pattern Recognition. Nauka, Moscow, 1974. (in Russian); German translation: Theorie der Zeichenerkennung, Akademie Verlag, Berlin, 1979.