Minimization of error functionals over variable-basis functions

SIAM Journal on Optimization

Volumn 14, Issue 3, 2004, Pages 732-742

  Kainen, Paul C ^a   Kůrková, Věra ^b   Sanguineti, Marcello ^c  

^a GEORGETOWN UNIVERSITY   (United States)

^b INSTITUTE OF COMPUTER SCIENCE   (Czech Republic)

^c UNIVERSITY OF GENOA   (Italy)

Author keywords

Approximate optimization; Complexity of admissible functions; Error functionals; Generalized Tikhonov well posedness; Rate of decrease of infima

Indexed keywords

EID: 2542498732     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/S1052623402401233     Document Type: Article

References (29)

1. R. A. ADAMS, Sobolev Spaces, Academic Press, New York, 1975.
2. A. R. BARRON, Universal approximation bounds for superpositions of a sigmoidal function, IEEE Trans. Inform. Theory, 39 (1993), pp. 930-945.
3. R. BELLMAN, Dynamic Programming, Princeton University Press, Princeton, NJ, 1957.
4. D. P. BERTSEKAS AND J. N. TSITSIKLIS, Neuro-Dynamic Programming, Athena Scientific, Belmont, MA, 1996.
5. D. BRAESS, Nonlinear Approximation Theory, Springer-Verlag, Berlin, 1986.
6. R. COURANT, Differential and Integral Calculus, Vol. II, Wiley, New York, 1988.
7. C. DARKEN, M. DONAHUE, L. GURVITS, AND E. SONTAG, Rates of approximation results motivated by robust neural network learning, in Proceedings of the 6th Annual ACM Conference on Computational Learning Theory, ACM, New York, 1993, pp. 303-309.
8. A. L. DONTCHEV AND T. ZOLEZZI, Well-Posed Optimization Problems, Lecture Notes in Math. 1543, Springer-Verlag, Berlin, 1993.
9. L. GURVITS AND P. KOIRAN, Approximation and learning of convex superpositions, J. Comput. System Sci., 55 (1997), pp. 161-170.
10. L. K. JONES, A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training, Ann. Statist., 20 (1992), pp. 608-613.
11. P. C. KAINEN, V. KŮRKOVÁ, AND A. VOGT, Best approximation by linear combinations of characteristic functions of half-spaces, J. Approx. Theory, 122 (2003), pp. 151-159.
12. V. KŮRKOVÁ, Approximation of functions by perceptron networks with bounded number of hidden units, Neural Networks, 8 (1995), pp. 745-750.
13. V. KŮRKOVÁ, Dimension-independent rates of approximation by neural networks, in Computer-Intensive Methods in Control and Signal Processing. The Curse of Dimensionality, K. Warwick and M. Kárný, eds., Birkhäuser Boston, Cambridge, MA, 1997, pp. 261-270.
14. V. KŮRKOVÁ, High-dimensional approximation by neural networks, in Advances in Learning Theory: Methods, Models and Applications, J. Suykens et al., eds., IOS Press, Amsterdam, 2003, pp. 69-88.
15. V. KŮRKOVÁ, P. C. KAINEN, AND V. KREINOVICH, Estimates of the number of hidden units and variation with respect to half-spaces, Neural Networks, 10 (1997), pp. 1061-1068.
16. V. KŮRKOVÁ AND M. SANGUINETI, Bounds on rates of variable-basis and neural-network approximation, IEEE Trans. Inform. Theory, 47 (2001), pp. 2659-2665.
17. V. KŮRKOVÁ AND M. SANGUINETI, Comparison of worst case errors in linear and neural network approximation, IEEE Trans. Inform. Theory, 48 (2002), pp. 264-275.
18. V. KŮRKOVÁ, P. SAVICKÝ, AND K. HLAVÁČKOVÁ, Representations and rates of approximation of real-valued Boolean functions by neural networks, Neural Networks, 11 (1998), pp. 651-659.
19. E. KUSHILEVITZ AND Y. MANSOUR, Learning decision trees using the Fourier spectrum, SIAM J. Comput., 22 (1993), pp. 1331-1348.
20. M. LESHNO, V. Y. LIN, A. PINKUS, AND S. SCHOCKEN, Multilayer feedforward networks with a nonpolynomial activation can approximate any function, Neural Networks, 6 (1993), pp. 861-867.
21. T. PARISINI AND R. ZOPPOLI, Neural networks for feedback feedforward nonlinear control systems, IEEE Trans. Neural Networks, 5 (1994), pp. 436-449.
22. T. PARISINI AND R. ZOPPOLI, Neural approximations for multistage optimal control of nonlinear stochastic systems, IEEE Trans. Automat. Control, 41 (1996), pp. 889-895.
23. A. PINKUS, n-Widths in Approximation Theory, Springer-Verlag, Berlin, Heidelberg, 1985.
24. G. PISIER, Remarques sur un résultat non publié de B. Maurey, in Séminaire d'Analyse Fonctionnelle 1980-81, Exposé V, École Polytechnique, Centre de Mathématiques, Palaiseau, France, 1981, pp. V.1-V.12.
25. T. J. SEJNOWSKI AND C. R. ROSENBERG, Parallel networks that learn to pronounce English text, Complex Systems, 1 (1987), pp. 145-168.
26. I. SINGER, Best Approximation in Normed Linear Spaces by Elements of Linear Subspaces, Springer-Verlag, Berlin, 1970.
27. H. J. WEAVER, Applications of Discrete and Continuous Fourier Analysis, Wiley, New York, 1983.
28. R. ZOPPOLI AND T. PARISINI, Learning techniques and neural networks for the solution of N-stage nonlinear nonquadratic optimal control problems, in Systems, Models and Feedback: Theory and Applications, A. Isidori and T. J. Tarn, eds., Birkhäuser Boston, Cambridge, MA, 1992, pp. 193-210.
29. R. ZOPPOLI, M. SANGUINETI, AND T. PARISINI, Approximating networks and extended Ritz method for the solution of functional optimization problems, J. Optim. Theory Appl., 112 (2002), pp. 403-440.