Approximation error bounds via rademacher's complexity

Applied Mathematical Sciences

Volumn 2, Issue 4, 2008, Pages 153-176

  Gnecco, Giorgio ^a   Sanguineti, Marcello ^a  

^a UNIVERSITY OF GENOA   (Italy)

Author keywords

Approximation error; Curse of dimension ality; Model complexity; Rademacher's complexity; Talagrand's inequality; Union bounds; VC di mension

Indexed keywords

EID: 57649240046     PISSN: 1312885X     EISSN: 13147552     Source Type: Journal    
DOI: None     Document Type: Article

References (35)

1. A. R Barron, Universal approximation bounds for superpositions of a sigmoidal function, IEEE Trans. on Information Theory, 39 (1993), 930-945.
2. L. Breiman, Hinging hyperplanes for regression, classification, and func-tion approximation, IEEE Trans. on Information Theory, 39 (1993), 993-1013.
3. L. K. Jones, A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training, Annals of Statistics, 20 (1992), 608-613.
4. F. Girosi and G. Anzellotti, Rates of convergence for Radial Basis Func-tions and neural networks, in Artificial Neural Networks for Speech and Vision, R. J. Mammone, editor, 97-113, Chapman & Hall, London, 1993.
5. V. Kůrková and M. Sanguineti, Comparison of worst case errors in linear and neural network approximation, IEEE Trans. on Information Theory, 48 (2002), 264-275.
6. V. Kůrková and M. Sanguineti, Learning with generalization capability by kernel methods of bounded complexity, J. of Complexity, 21 (2005), 350-367.
7. R. Zoppoli, M. Sanguineti, and T. Parisini, Approximating networks and extended Ritz method for the solution of functional optimization prob-lems, J. of Optimization Theory and Applications, 112 (2002), 403-440.
8. V. Kůrková and M. Sanguineti, Error estimates for approximate optimiza-tion by the extended Ritz method, SIAM J. on Optimization, 15 (2005), 461-487.
9. R. Bellman, Dynamic Programming, Princeton University Press, 1957.
10. F. Girosi, Approximation error bounds that use VC-bounds, in Proc. Int. Conf. on Artificial Neural Networks (Paris, 1995), 295-302.
11. M. A. Kon, L. A. Raphael, and D. A Williams, Extending Girosi's approx-imation estimates for functions in Sobolev spaces via Statistical Learning Theory, J. of Analysis and Applications, 3(2) (2005), 67-90.
12. M. A. Kon and L. A Raphael, Approximating functions in reproduc-ing kernel Hilbert spaces via Statistical Learning Theory, in Wavelets and Splines, G. Chen and M. J. Lai, editors, 271-286, Nashboro Press, Nashville, TN, 2006.
13. N. Aronszajn, Theory of reproducing kernels, Trans. of AMS, 68 (1950), 337-404.
14. F. Cucker and S. Smale, On the mathematical foundations of learning, Bulletin of AMS, 39 (2001), 1-49.
15. V. P. Vapnik, The Nature of Statistical Learning Theory, Springer-Verlag, New York, 1995.
16. S. Mendelson, A few notes on Statistical Learning Theory, in Advanced Lectures on Machine Learning-LNCS 2600, Mendelson, S. and Smola, A., editors, 1-40, Springer, 2003.
17. M. Talagrand, Sharper bounds for Gaussian and empirical processes, An-nals of Probability, 22 (1994), 20-76.
18. M. Anthony and P. L. Bartlett, Neural Network Learning: Theoretical Foundations, Cambridge University Press, Cambridge, UK, 1999.
19. B. Carl, I. Kyrezi, and A. Pajor, Metric entropy of convex hulls in Banach spaces, J. London Math. Soc., 60 (1999), 871-896.
20. M. Vidyasagar, A Theory of Learning and Generalization, Springer, Berlin, 1997.
21. W. Hoefding, Probability inequalities for sums of bounded random vari-ables, J. of the American Statistical Association, 53 (1963), 13-30.
22. N. Alon, S. Ben-David, N. Cesa-Bianchi, and D. Haussler, Scale sensitive dimensions, uniform convergence and learnability, J. of the ACM, 4 (1997), 615-631.
23. D. Pollard, Empirical Processes: Theory and Applications, Volume 2 of NSF-CBMS Regional Conf. Series in Probability and Statistics, Institute of Mathematical Statistics and American Statistical Association, 1990.
24. N. Aronszajn and K. T. Smith, Theory of Bessel potentials-Part I, Ann. Inst. Fourier, 11 (1961), 385-475 (Also: Technical Report 22-O±ce of Naval Research, Lawrence, Kansas, 1959).
25. E. M. Stein, Singular Integrals and Di®erentiability Properties of Func-tions, Princeton University Press, Princeton, NJ, 1970.
26. L. P. Castro, Strongly elliptic operators for a plane wave di®raction prob-lem in Bessel potential spaces, J. of Inequalities in Pure and Applied Mathematics, 3 (2002), 2, Article 25.
27. H. Brezis, Analyse Fonctionnelle-Théorie et Applications. Masson, Paris, 1983.
28. E. Sontag, VC dimension of neural networks, in Neural Networks and Ma-chine Learning, C. Bishop, editor, 69-95, Springer-Verlag, Berlin, 1998.
29. H. N. Mhaskar and C. A. Micchelli, Dimension independent bounds on the degree of approximation by neural networks, IBM J. of Research and Development, 38 (1994), 277-284.
30. A. R. Barron, Neural net approximation, in Proc. 7th Yale Workshop on Adaptive and Learning Systems, K. S. Narendra, editor, 69-72, Yale University Press, 1992.
31. H. N. Mhaskar, Neural networks for optimal approximation of smooth and analytic functions, Neural Computation, 8 (1996), 164-177.
32. W. Rudin, Real and Complex Analysis, McGraw-Hill, Singapore, 1987 (III Ed.).
33. http://www.cs.berkeley.edu/~bartlett/courses/281b-sp06/lecture24.ps.
34. http://ocw.mit.edu/OcwWeb/Mathematics/18-465Spring-2004/LectureNotes/index.htm.
35. S. Mendelson and R. Vershynin, Entropy and the combinatorial dimen-sion, Inventiones Mathematicae, 152 (2003), 37-55.