Rademacher and Gaussian complexities: Risk bounds and structural results

Journal of Machine Learning Research

Volumn 3, Issue 3, 2003, Pages 463-482

  Bartlett, Peter L ^a   Mendelson, Shahar ^a  

^a AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

Author keywords

Data Dependent Complexity; Error Bounds; Maximum Discrepancy; Rademacher Averages

Indexed keywords

DATA REDUCTION; ERROR ANALYSIS; NEURAL NETWORKS;

GAUSSIAN COMPLEXITIES;

LEARNING SYSTEMS;

EID: 0038453192     PISSN: 15324435     EISSN: None     Source Type: Journal    
DOI: 10.1162/153244303321897690     Document Type: Conference Paper

References (23)

1. Martin Anthony and Peter L. Bartlett. Neural Network Learning: Theoretical Foundations. Cambridge University Press, Cambridge, UK, 1999.
2. Peter L. Bartlett. The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. IEEE Transactions on Information Theory, 44(2):525-536, 1998.
3. Peter L. Bartlett, Stéphane Boucheron, and Gábor Lugosi. Model selection and error estimation. Machine Learning, 48:85-113, 2002.
4. Peter L. Bartlett and Shahar Mendelson. Rademacher and gaussian complexities: Risk bounds and structural results. In Proceedings of the Fourteenth Annual Conference on Computational Learning Theory, pages 224-240, 2001.
5. Nello Cristianini and John Shawe-Taylor. An Introduction to Support Vector Methods. Cambridge University Press, 2000.
6. Luc Devroye, Lázsló Györfi, and Gábor Lugosi. A probabilistic theory of pattern recognition. Applications of Mathematics: Stochastic Modelling and Applied Probability (31). Springer, 1996.
7. Mostefa Golea, Peter L. Bartlett, and Wee Sun Lee. Generalization in decision trees and DNF: Does size matter? In NIPS 10, pages 259-265, 1998.
8. Michael J. Kearns, Yishay Mansour, Andrew Y. Ng, and Dana Ron. An experimental and theoretical comparison of model selection methods. Machine Learning, 27:7-50, 1997.
9. V. Koltchinskii. Rademacher penalties and structural risk minimization. IEEE Transactions on Information Theory, 47(5):1902-1914, July 2001.
10. V. Koltchinskii and D. Panchenko. Empirical margin distributions and bounding the generalization error of combined classifiers. Technical report, Department of Mathematics and Statistics, University of New Mexico, 2000a.
11. V. I. Koltchinskii and D. Panchenko. Rademacher processes and bounding the risk of function learning. In E. Gine, D. Mason, and J. Wellner, editors, High Dimensional Probability II, pages 443-459. 2000b.
12. E. B. Kong and T. G. Dietterich. Error-correcting output coding corrects bias and variance. In Proc. 12th International Conference on Machine Learning, pages 313-321. Morgan Kaufmann, 1995.
13. M. Ledoux and M. Talagrand. Probability in Banach Spaces: isoperimetry and processes. Springer, 1991.
14. Llew Mason, Peter L. Bartlett, and Jonathan Baxter. Improved generalization through explicit optimization of margins. Machine Learning, 38(3):243-255, 2000.
15. G. McDiarmid. On the method of bounded differences. In Surveys in Combinatorics 1989, pages 148-188. Cambridge University Press, 1989.
16. Shahar Mendelson. l-norm and its application to learning theory. Positivity, 5:177-191, 2001.
17. Shahar Mendelson. Rademacher averages and phase transitions in Glivenko-Cantelli classes. IEEE Transactions on Information Theory, 48(1):251-263, 2002.
18. G. Pisier. The volume of convex bodies and Banach space geometry. Cambridge University Press, 1989.
19. Robert E. Schapire. Using output codes to boost multiclass learning problems. In Machine Learning: Proc. Fourteenth International Conference, pages 313-321, 1997.
20. Robert E. Schapire, Yoav Freund, Peter L. Bartlett, and Wee Sun Lee. Boosting the margin: a new explanation for the effectiveness of voting methods. Annals of Statistics, 26(5):1651-1686, October 1998.
21. John Shawe-Taylor, Peter L. Bartlett, Robert C. Williamson, and Martin Anthony. Structural risk minimisation over data-dependent hierarchies. IEEE Transactions on Information Theory, 44(5):1926-1940, 1998.
22. N. Tomczak-Jaegermann. Banach-Mazur distance and finite-dimensional operator ideals. Number 38 in Pitman Monographs and Surveys in Pure and Applied Mathematics. Pitman, 1989.
23. Vladimir N. Vapnik and A. Y. Chervonenkis. On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its Applications, 16(2):264-280, 1971