Estimates of covering numbers of convex sets with slowly decaying orthogonal subsets

Discrete Applied Mathematics

Volumn 155, Issue 15, 2007, Pages 1930-1942

  Kůrková, Věra ^a   Sanguineti, Marcello ^b  

^a INSTITUTE OF COMPUTER SCIENCE   (Czech Republic)

^b UNIVERSITY OF GENOA   (Italy)

Author keywords

Generalized Hadamard matrices; Lower bounds on covering numbers; Minkowski functional; Power type covering numbers; Symmetric convex hulls

Indexed keywords

APPROXIMATION THEORY; HILBERT SPACES; LEARNING SYSTEMS; MATRIX ALGEBRA;

LOWER BOUNDS ON COVERING NUMBERS; MINKOWSKI FUNCTIONAL; POWER-TYPE COVERING NUMBERS; SYMMETRIC CONVEX HULLS;

SET THEORY;

EID: 34547732358     PISSN: 0166218X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.dam.2007.04.007     Document Type: Article

References (46)

1. Alon N., Ben-David S., Cesa-Bianchi N., and Haussler D. Scale-sensitive dimensions, uniform convergence, and learnability. J. Assoc. Comput. Mach. 44 (1997) 615-631
2. Anthony M., and Bartlett P.L. Neural Network Learning: Theoretical Foundations (1999), Cambridge University Press, Cambridge, UK
3. Barron A.R. Neural net approximation. In: Narendra K. (Ed). Proceedings of the Seventh Yale Workshop on Adaptive and Learning Systems (1992), Yale University Press, New Haven, CT 69-72
4. Barron A.R. Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans. Inform. Theory 39 (1993) 930-945
5. Bartlett P.L. The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. IEEE Trans. Inform. Theory 44 (1998) 525-536
6. Birgé L. Estimating a density under order restrictions: nonasymptotic minimax risk Ann. Statist. 15 (1987) 995-1012
7. Bousquet O., Koltchinskii V., and Panchenko D. Some local measures of complexity of convex hulls and generalization bounds. Proceedings of the 15th Annual Conference on Computational Learning Theory (2002), Springer, London 59-73
8. Carl B. Entropy numbers of diagonal operators with an application to eigenvalue problems. J. Approx. Theory 32 (1981) 135-150
9. Carl B. Metric entropy of convex hulls in Hilbert spaces. Bull. London Math. Soc. 29 (1997) 452-458
10. Carl B., Kyrezi I., and Pajor A. Metric entropy of convex hulls in Banach spaces. J. London Math. Soc. 60 (1999) 871-896
11. Carl B., and Stephani I. Entropy, Compactness, and the Approximation of Operators (1990), Cambridge University Press, Cambridge, UK
12. Creutzig J., and Steinwart I. Metric entropy of convex hulls in type p spaces-the critical case. Proc. Amer. Math. Soc. 130 (2002) 733-743
13. Cucker F., and Smale S. On the mathematical foundations of learning. Bull. Amer. Math. Soc. 39 (2001) 1-49
14. Devroye L., and Lugosi G. Combinatorial Methods in Density Estimation (2001), Springer, Berlin
15. Dudley R.M. Uniform Central Limit Theorems, Cambridge Studies in Advanced Mathematics vol. 63 (1999), Cambridge University Press, Cambridge, UK
16. Edmunds D.E., and Triebel H. Function Spaces, Entropy Numbers, and Differential Operators (1996), Cambridge University Press, Cambridge, UK
17. Evgeniou T., Pontil M., and Poggio T. Regularization networks and support vector machines. Adv. Comput. Math. 13 (2000) 1-50
18. Fine T.L. Feedforward Neural Network Methodology (1999), Springer, New York
19. Gao F. Metric entropy of convex hulls. Israel J. Math. 123 (2001) 359-364
20. Gao F. Entropy of absolute convex hulls in Hilbert spaces. Bull. London Math. Soc. 36 (2004) 460-468
21. Gurvits L., and Koiran P. Approximation and learning of convex superpositions. J. Comput. System Sci. 55 (1997) 161-170
22. Jones L.K. A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training. Ann. Statist. 20 (1992) 608-613
23. Kainen P.C., and Kůrková V. Quasiorthogonal dimension of Euclidean spaces. Appl. Math. Lett. 6 (1993) 7-10
24. Kolmogorov A.N. Asymptotic characteristics of some completely bounded metric spaces. Dokl. Akad. Nauk. SSSR 108 (1956) 585-589
25. V. Kůrková, Dimension-independent rates of approximation by neural networks, in: K. Warwick, M. Kárný (Eds.), Computer-Intensive Methods in Control and Signal Processing. The Curse of Dimensionality, Birkhauser, Boston, 1997, pp. 261-270.
26. Kůrková V., Kainen P.C., and Kreinovich V. Estimates of the number of hidden units and variation with respect to half-spaces. Neural Networks 10 (1997) 1061-1068
27. Kůrková V., and Sanguineti M. Comparison of worst case errors in linear and neural network approximation. IEEE Trans. Inform. Theory 48 (2002) 264-275
28. Li W.V., and Linde W. Approximation, metric entropy and small ball estimates for Gaussian measures. Ann. Probab. 27 (1999) 1556-1578
29. Li W.V., and Linde W. Metric entropy of convex hulls in Hilbert spaces. Studia Math. 139 (2000) 29-45
30. Lorentz G.G. Metric entropy and approximation. Bull. Amer. Math. Soc. 72 (1966) 903-937
31. Luschgy H., and Pagés G. Sharp asymptotics of the Kolmogorov entropy for Gaussian measures. J. Funct. Anal. 212 (2004) 89-120
32. Makovoz Y. Random approximants and neural networks. J. Approx. Theory 85 (1996) 98-109
33. Mendelson S. On the size of convex hulls of small sets. J. Mach. Learning Res. 2 (2001) 1-18
34. Mendelson S., and Vershynin R. Entropy and the combinatorial dimension. Inven. Math. 152 (2003) 37-55
35. G. Pisier, Remarques sur un résultat non publié de B. Maurey. Séminaire d'Analyse Fonctionnelle 1980-81, Exposé no. V, École Polytechnique, Centre de Mathématiques, Palaiseau, France, pp. V.1-V.12.
36. Pollard D. Convergence of Stochastic Processes (1984), Springer, New York
37. Rudin W. Functional Analysis. 2nd ed. (1991), McGraw-Hill, USA
38. Smola A.J., Elisseeff A., Schölkopf B., and Williamson R.C. Entropy numbers for convex combinations and MLPs. In: Smola A.J., Bartlett P.L., Schölkopf B., and Schuurmans D. (Eds). Advances in Large Margin Classifiers (2000), MIT Press, Cambridge, MA 369-387
39. A.J. Smola, R.C. Williamson, B. Schölkopf, Generalization bounds for convex combinations of kernel functions, NeuroCOLT Technical Report NC-TR-98-022, 1998.
40. Steinwart I. Entropy numbers of convex hulls and an application to learning algorithms. Arch. Math. 80 (2003) 310-318
41. Steinwart I. Entropy of convex hulls-some Lorentz norm results. J. Approx. Theory 128 (2004) 42-52
42. I. Steinwart, C. Scovel, Fast rates for support vector machines using Gaussian kernels, Ann. Statist. 35 (2) (2007).
43. Vapnik V.N. Statistical Learning Theory (1998), Wiley, USA
44. Vidyasagar M. A Theory of Learning and Generalization (1997), Springer, Berlin
45. Williamson R.C., Smola A.J., and Schölkopf B. Generalization performance of regularization networks and support vector machines via entropy numbers of compact operators. IEEE Trans. Inform. Theory 47 (2001) 2516-2532
46. Zhou D.-X. The covering number in learning theory. J. Complexity 18 (2002) 739-767