On a variational norm tailored to variable-basis approximation schemes

IEEE Transactions on Information Theory

Volumn 57, Issue 1, 2011, Pages 549-558

  Gnecco, Giorgio ^a   Sanguineti, Marcello ^a  

^a UNIVERSITY OF GENOA   (Italy)

Author keywords

Approximation schemes; convex hulls; infinite dimensionaloptimization; L1 norm; upper and lower bounds; variation with respect to a set

Indexed keywords

APPROXIMATION SCHEME; CONVEX HULL; INFINITE-DIMENSIONALOPTIMIZATION; L1-NORM; UPPER AND LOWER BOUNDS; VARIATION WITH RESPECT TO A SET;

COMPUTATIONAL GEOMETRY;

EID: 78650896759     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2010.2090198     Document Type: Article

References (49)

1. I. Singer, Best Approximation in Normed Linear Spaces by Elements of Linear Subspaces. Berlin, Heidelberg: Springer, 1970.
2. V. Kurková and M. Sanguineti, "Comparison of worst-case errors in linear and neural network approximation," IEEE Trans. Inf. Theory, vol. 48, pp. 264-275, 2002.
3. A. R. Barron, "Neural net approximation," in Proc. 7th Yale Workshop on Adapt. Learn. Syst., K. S. Narendra, Ed., 1992, pp. 69-72, Yale Univ. Press.
4. R. A. DeVore and G. G. Lorentz, Constructive Approximation. Berlin: Springer, 1993.
5. R. A. DeVore, B. Jawerth, and V. Popov, "Compression of wavelet decompositions," Amer. J. Math., vol. 114, pp. 737-785, 1992.
6. R. A. DeVore and V. N. Temlyakov, "Nonlinear approximation by trigonometric sums," The J. Fourier Anal. Appl., vol. 2, pp. 29-48, 1995.
7. S. Haykin, Neural Networks. A Comprehensive Foundation. New York: MacMillan, 1994.
8. A. R. Barron, "Universal approximation bounds for superpositions of a sigmoidal function," IEEE Trans. Inf. Theory, vol. 39, pp. 930-945, 1993.
9. R. Bellman, Dynamic Programming. Princeton, NJ: Princeton Univ. Press, 1957.
10. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, 1996.
11. R. Zoppoli, M. Sanguineti, and T. Parisini, "Approximating networks and extended Ritz method for the solution of functional optimization problems," J. Optimiz. Theory Appl., vol. 112, pp. 403-439, 2002.
12. T. J. Sejnowski and C. Rosenberg, "Parallel networks that learn to pronounce English text," Complex Syst., vol. 1, pp. 145-168, 1987.
13. V. Kurková and M. Sanguineti, "Error estimates for approximate optimization by the extended Ritz method," SIAM J. Optimiz., vol. 15, pp. 261-287, 2005.
14. V. Kurková, , K. Warwick and M. Kárný, Eds., "Dimension-independent rates of approximation by neural networks," in Computer-Intensive Methods in Control and Signal Processing. The Curse of Dimensionality. Boston, MA: Birkhäuser, 1997, pp. 261-270.
15. V. Kurková, P. C. Kainen, and V. Kreinovich, "Estimates of the number of hidden units and variation with respect to half-spaces," Neural Netw., vol. 10, pp. 1061-1068, 1997.
16. V. Kurková, P. Savický, and K. Hlaváč ková, "Representations and rates of approximation of real-valued boolean functions by neural networks," Neural Netw., vol. 11, pp. 651-659, 1998.
17. V. Kurková and M. Sanguineti, "Bounds on rates of variable-basis and neural-network approximation," IEEE Trans. Inf. Theory, vol. 47, pp. 2659-2665, 2001. (Pubitemid 32949922)
18. V. Kürková and M. Sanguineti, "Learning with generalization capability by kernel methods of bounded complexity," J. Complex., vol. 21, pp. 350-367, 2005.
19. V. Kürková and M. Sanguineti, "Approximate minimization of the regularized expected error over kernel models," Math. Operat. Res., vol. 33, pp. 747-756, 2008.
20. P. C. Kainen, V. Kürková, and M. Sanguineti, "Minimization of error functionals over variable-basis functions," SIAM J. Optimiz., vol. 14, pp. 732-742, 2003.
21. S. Giulini and M. Sanguineti, "On dimension-independent approximation by neural networks and linear approximators," in Proc. Int. Joint Conf. Neural Netw., 2000, pp. I283-I288.
22. S. Giulini and M. Sanguineti, "Approximation schemes for functional optimization problems," J. Optimiz. Theory Appl., vol. 140, pp. 33-54, 2009.
23. V. Kürková and M. Sanguineti, "Geometric upper bounds on rates of variable-basis approximation," IEEE Trans. Inf. Theory, vol. 54, pp. 5681-5688, 2008.
24. L. K. Jones, "A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training," Ann. Statist., vol. 20, pp. 608-613, 1992.
25. L. Breiman, "Hinging hyperplanes for regression, classification and function approximation," IEEE Trans. Inf. Theory, vol. 39, pp. 999-1013, 1993.
26. P. C. Kainen and V. Kürková, "An integral upper bound for neural network approximation," Neural Computat., vol. 21, pp. 2970-2989, 2009.
27. V. Kürková, "Model complexity of neural networks and integral transforms," in Lecture Notes in Comput. Sci. (Proc. ICANN 2009),M. Polycarpou, C. Panayiotou, C. Alippi, and G. Ellinas, Eds., Berlin, Heidelberg, 2009, vol. 5768 , pp. 708-718, Springer.
28. G. Gnecco and M. Sanguineti, "Estimates of variation with respect to a set and applications to optimization problems," J. Optimiz. Theory Appl., vol. 145, pp. 53-75, 2010.
29. D. Hong, J.Wang, and R. Gardner, Real Analysis With an Introduction to Wavelets and Applications. New York: Elsevier Academic , 2005.
30. G. Pisier, "Remarques sur un résultat non publié de B. Maurey," Séminaire d'Analyse Fonctionnelle 1980-81, vol. I, no. 12, 1981, École Polytechnique, Centre de Mathématiques, Palaiseau, France.
31. R. A. Adams and J. J. F. Fournier, Sobolev Spaces. New York: Elsevier Academic , 2003.
32. A. Pinkus, n-Widths in Approximation Theory. Berlin, Heidelberg: Springer, 1985.
33. D. Gottlieb and S. A. Orszag, Numerical Analysis of Spectral Methods: Theory and Applications. Philadelphia, PA: SIAM, 1977.
34. N. Aronszajn, "Theory of reproducing kernels," Trans. AMS, vol. 68, pp. 337-404, 1950.
35. C. Berg, J. P. R. Christensen, and P. Ressel, Harmonic Analysis on Semigroups. New York: Springer, 1984.
36. G. Wahba, Spline Models for Observational Data. Philadelphia, PA: SIAM, 1990.
37. A. Berlinet and C. Thomas-Agnan, Reproducing Kernel Hilbert Spaces in Probability and Statistics. Boston, MA: Kluwer, 2004.
38. E. Parzen, "An approach to time series analysis," Ann. Math. Statist., vol. 32, pp. 951-989, 1961.
39. C. Cortes and V. Vapnik, "Support vector networks," Mach. Learn., vol. 20, pp. 1-25, 1995.
40. T. Poggio and F. Girosi, "Networks for approximation and learning," Proc. IEEE, vol. 78, pp. 1481-1497, 1990.
41. F. Cucker and S. Smale, "On the mathematical foundations of learning," Bull. AMS, vol. 39, pp. 1-49, 2001.
42. F. Cucker and D. X. Zhou, Learning Theory-An Approximation Theory Viewpoint. Cambridge, U.K.: Cambridge Univ. Press, 2007.
43. S. Smale and D. X. Zhou, "Learning theory estimates via integral operators and their approximations," Construct. Approx., vol. 26, pp. 153-172, 2007.
44. G. Gnecco and M. Sanguineti, "Accuracy of suboptimal solutions to kernel principal component analysis," Computat. Optimiz. Appl., vol. 42, pp. 265-287, 2009.
45. G. Gnecco and M. Sanguineti, "The weight-decay technique in learning from data: An optimization point of view," Computat. Manage. Sci., vol. 6, pp. 53-79, 2009.
46. G. Gnecco and M. Sanguineti, "Regularization and suboptimal solutions in learning from data," in Innovations in Neural Information Paradigms and Applications, M. Bianchini, M. Maggini, and F. Scarselli, Eds. New York: Springer, 2009, pp. 113-154.
47. G. Gnecco and M. Sanguineti, "Regularization techniques and suboptimal solutions to optimization problems in learning from data," Neural Computat., vol. 22, pp. 793-829, 2010.
48. G. Gnecco and M. Sanguineti, "Error bounds for suboptimal solutions to kernel principal component analysis," Optimiz. Lett., vol. 4, pp. 197-210, 2010.
49. H. Sun, "Mercer theorem for RKHS on noncompact sets," J. Complex., vol. 21, pp. 337-349, 2005.