Sup-Norm Approximation Bounds for Networks Through Probabilistic Methods

IEEE Transactions on Information Theory

Volumn 41, Issue 4, 1995, Pages 1021-1027

  Yukich, Joseph E ^a   Stinchcombe, Maxwell B ^b   White, Halbert ^c  

^a LEHIGH UNIVERSITY   (United States)

^b UNIVERSITY OF TEXAS AT AUSTIN   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Artificial neural networks; central limit theorems; degree of approximation bounds; empirical processes; hidden layer feedforward networks; radial basis networks; wavelet networks

Indexed keywords

APPROXIMATION THEORY; FEEDFORWARD NEURAL NETWORKS; FOURIER TRANSFORMS; LYAPUNOV METHODS; NEURAL NETWORKS; PROBABILITY; ROBOTICS; THEOREM PROVING;

CENTRAL LIMIT THEOREMS; DEGREE OF APPROXIMATION BOUNDS; EMPIRICAL PROCESSES; HIDDEN LAYER FEEDFORWARD NETWORKS; RADIAL BASIS NETWORKS; WAVELET NETWORKS;

INFORMATION THEORY;

EID: 0029341524     PISSN: 00189448     EISSN: 15579654     Source Type: Journal    
DOI: 10.1109/18.391247     Document Type: Article

References (27)

1. A. R. Barron, “Neural net approximation,” in Proc. 7th Yale Workshop on Adaptive and Learning Systems, May, 1992.
2. —, “Universal approximation bounds for superpositions of a sigmoidal function,” IEEE Trans. Inform. Theory, vol. 39, pp. 930–945, 1993.
3. G. Cybenko, “Approximation by superpositions of a sigmoidal function,” Math. Contr., Sig., Syst., vol. 2, pp. 303–14, 1989.
4. I. Daubechies, “Orthonormal bases of compactly supported wavelets,” Commun. Pure Appl. Math., vol. 41, pp. 909–996, 1988.
5. I. Daubechies, Ten Lectures on Wavelets, Philadelphia, PA: SIAM, 1992.
6. C. Darken, M. Donahue, L. Gurvits, and E. Sontag, “Rate of approximation results motivated by robust neural network learning,” in Proc. 6th Annual ACM Conf. On Computational Learning Theory. ACM Press, 1993, pp. 303–309.
7. B. Delyon, A. Juditsky, and A. Benveniste, “Accuracy analysis for wavelet approximations,” preprint, 1992.
8. R. M. Dudley, A Course on Empirical Processes (Ecole d'été de probabilités de St. Flour, Lecture Notes in Math.) New York: Springer, 1997, pp. 1–142.
9. —, “Universal Donsker classes and metric entropy,” Annals Probab., vol. 15, pp. 1306–1326, 1987.
10. K. Funahashi, “On the approximate realization of continuous mappings by neural networks,” Neural Net., vol. 2, pp. 183–192, 1989.
11. R. Gencay and W. D. Dechert, “An algorithm for the n Lyapunov exponents of an n-dimensional unknown dynamical system,” Physica D, pp. 142–157, 1992.
12. K. Hornik, “Some new results on neural network approximation,” Neural Net., vol. 6, pp. 1069–1072, 1993.
13. K. Hornik, M. Stinchcombe, and H. White, “Multi-layer feedforward networks are universal approximators,” Neural Net., vol. 2, pp. 359–368, 1989.
14. —, “Universal approximation of an unknown mapping and its derivatives using multi-layer feedforward networks,” Neural Net., vol. 3, pp. 551–560, 1990; reprinted in Artificial Neural Networks: Approximation and Learning Theory, H. White, Ed. Oxford, UK: Blackwell, 1992.
15. K. Hornik, M. Stinchcombe, H. White, and P. Auer, “Degree of approximation results for feedforward networks approximating an unknown mapping and their derivatives,” Neural Comput., vol. 6, pp. 1262–1275, 1994.
16. B. Irie and S. Miyake, “Capabilities of three-layered perceptrons,” in Proc. Int. Conf. on Neural Networks, vol. I, 1988, pp. 641–648.
17. W. Joerding and Y. Li, “Approximating production functions with feedforward networks,” Washington State Univ., Dept. of Economics Discussion Paper, 1992.
18. M. Jordan, “Generic constraints on underspecified target trajectories,” in Proc. 1989 Int. Joint Conf. on Neural Networks. New York: IEEE Press, pt. I, pp. 217–225, 1989.
19. S. Kelly, M. Kon, and L. Raphael, “Pointwise convergence of wavelet expansions,” Bull. AMS, vol. 30, pp. 87–94, 1994.
20. A. N. Kolmogorov, “On the representation of continuous functions of many variables by superpositions of continuous functions of one variable and addition,” Dokl. Akad. Nauk. SSSR, vol. 114, pp. 953–956, 1957.
21. M. Ledoux and M. Talagrand, Probability on Banach Spaces. New York: Springer-Verlag, 1991.
22. D. McCaffrey, S. Ellner, A. R. Gallant, and D. Nychka, “Estimating the Lyapunov exponent of a chaotic system with nonparametric regression,” J. Amer. Statist. Assoc., vol. 87, pp. 682–695, 1992.
23. J. Park and I. Sandberg, “Approximation and radial basis function networks,” Neural Comput., vol. 5, pp. 305–316, 1993.
24. G. Pisier, “Remarques sur un resultat non-publié de B. Maurey,” Seminaire d'analyse fonctionelle 1980–1981, Ecole Polytechnique, Centre de Mathématiques, Palaiseau, France, 1981.
25. M. Stinchcombe and H. White, “Consistent specification testing with unidentified nuisance parameters using duality and Banach space limit theory,” preprint, 1994.
26. A. van der Waart and J. Wellner, Weak Convergence and Empirical Processes. New York: Springer-Verlag, to appear.
27. Q. Zhang and A. Benveniste, “Wavelet networks,” IEEE Trans. Neural Net., vol. 3, no. 6, pp. 889–898, 1992.