On the optimality of neural-network approximation using incremental algorithms

IEEE Transactions on Neural Networks

Volumn 11, Issue 2, 2000, Pages 323-337

  Meir, Ron ^a,b   Maiorov, Vitaly E ^b  

^a IEEE

^b TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; ERROR ANALYSIS;

INCREMENTAL ALGORITHMS;

NEURAL NETWORKS;

EID: 0033732457     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.839004     Document Type: Article

References (43)

1. R. A. Adams, Sobolev Spaces. New York: Academic, 1975.
2. M. Anthony and P. L. Bartlett, A Theory of Learning in Artificial Neural Networks. Cambridge, U.K.: Cambridge Univ. Press, 1999.
3. P. Auer, M. Herbster, and M. Warmuth, "Exponentially many local minima for single neurons," in Advances in Neural Information Processing Systems 8, D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo, Eds. Cambodge, MA: MIT Press, 1996, pp. 316-322.
4. A. R. Barron, "Universal approximation bound for superpositions of a sigmoidal function," IEEE Trans. Inform. Theory, vol. 39, pp. 930-945, 1993.
5. A. R. Barron and R. L. Barron, "Statistical learning networks: a unifying view," in Comput. Sci. Statist.: Proc. 20th Symp. Interface, E. Wegman, Ed.. Washington, D.C., 1988, pp. 192-203.
6. P. L. Bartlett, V. Maiorov, and R. Meir, "Almost linear vc dimension bounds for piecwise polynomial networks," Neural Comput., vol. 10, pp. 2159-2173, 1998.
7. J. Bergh and J. Lofstrom, Interpolation Spaces. Berlin: Springer-Verlag, 1976.
8. M. S. Birman and M. Z. Solomjak, Quantitative Analysis in Sobolev Imbedding Theorems and Applications to Spectral Theory. Providence, RI: Amer. Math. Soc., 1980, vol. 144.
9. C. de Boor and G. Fix, "Spline approximation by quasiinterpolation," J. Approx. Theory, vol. 7, pp. 19-45, 1973.
10. n) by multilayer feedforward networks and related problems," IEEE Trans. Neural Networks, vol. 6, no. 1, pp. 25-30, 1995.
11. G. Cybenko, "Approximation by superpositions of a sigmoidal function," Math. Contr., Signals, Syst., vol. 2, pp. 303-314, 1989.
12. B. Delyon, A. Juditsky, and A. Benveniste, "Accuracy analysis for wavelet approximations," IEEE Trans. Neural Networks, vol. 6, pp. 332-348, 1995.
13. R. DeVore, "Nonlinear approximation," Acta Numerica, vol. 7, pp. 51-151, 1998.
14. R. A. Devore and G. G. Lorentz, Constructive Approximation. New York: Springer-Verlag, 1993.
15. R. A. DeVore, K. I. Oskolkov, and P. P. Petrushev, "Approximation by feed-forward neural networks, "Ann. Numer. Math., vol. 4, pp. 261-287, 1997.
16. A. T. Dingankar and I. W. Sandberg, "A note on error bounds for approximation in innerproduct spaces," Circuits, Syst., Signal Processing, vol. 15, no. 4, pp. 519-522, 1996.
17. I. W. Dingankar and A. T. Sandberg, "A note on errorbounds for function approximation using nonlinear networks," Circuits, Syst. Signal Processing, vol. 17, no. 4, pp. 449-457, 1998.
18. M. J. Donahue, L. Gurvits, C. Darken, and E. Sontag, "Rates of convex approximation in nonhilbert spaces," Constructive Approx., vol. 13, pp. 187-220, 1997.
19. D. L. Donoho and I. M. Johnstone, "Wavelet shrinkage: Asymptopia?," J. Royal Statist. Soc. B, vol. 57, no. 2, pp. 301-369, 1995.
20. S. E. Fahlman and C. Labiere, "The cascade-correlation learning architecture," in Advances in Neural Information Processing Systems, D. S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1990.
21. J. Fan and I. Gijbels, Local Polynomial Modeling and Its Applications. London, U.K.: Chapman and Hall, 1996.
22. K. Funahashi, "On the approximate realization of continuous mappings by neural networks," Neural Networks, vol. 2, pp. 183-192, 1989.
23. G. Hardy, J. E. Littlewood, and G. Polya, Inequalities, 2nd ed. Cambridge, U.K.: Cambridge Univ. Press, 1952.
24. K. Hornik, M. Stinchcombe, and H. White, "Multilayer feedforward networks are universal function approximators," Neural Networks, vol. 2, pp. 359-366, 1989.
25. L. Jones, "A simple lemma on greedy approximation in Hilbert space and convergence rate for projection pursuit regression and neural network training," Ann. Statist., vol. 20, pp. 608-613, 1992.
26. W. S. Lee, P. S. Bartlett, and R. C. Williamson, "Efficient Agnostic learning of neural networks with bounded fan-in," IEEE Trans. Inform. Theory, vol. 42, no. 6, pp. 2118-2132, 1996.
27. M. Leshno, V. Lin, A. Pinkus, and S. Schocken, "Multilayer feedforward networks with a nonpolynomial activation function can approximate any function," Neural Networks, vol. 6, pp. 861-867, 1993.
28. G. G. Lorentz, M. V. Golitscheck, and Y. Makovoz, Constructive Appoximation: Advanced Problems. Berlin, U.K.: Springer-Verlag, 1996.
29. V. E. Maiorov, "On best approximation by ridge functions," J. Approx. Theory, vol. 99, pp. 68-94, 1999.
30. V. E. Maiorov and R. Meir, "On the near optimality of the stochastic approximation of smooth functions by neural network," Advances Computa. Math., 2000, to be published.
31. V. E. Maiorov and J. Ratsaby, "On the degree of approximation using manifolds of finite pseudo-dimension," J. Constr. Approx., vol. 15, pp. 291-300, 1999.
32. S. Mallat, A Wavelet Tour of Signal Processing. New York: Academic, 1998.
33. H. Mhaskar, "Neural networks for optimal approximation of smooth and analytic functions," Neural Computa., vol. 8, no. 1, pp. 164-177, 1996.
34. O. P. Misra and J. L. Lavoine, "Transform analysis of generalized functions," in North-Holland Math Studies. Amsterdam, The Netherlands: North-Holland, 1986.
35. S. M. Nikolskii, Approximation of Functions of Several Variables and Imbedding Theorems. Berlin, Germany: Springer-Verlag, 1975.
36. P. P. Petrushev, "Aproximation by ridge functions and neural networks," SIAM J. Math. Anal., vol. 30, pp. 155-189, 1998.
37. A. Pinkus, N-Widths in Approximation Theory. New York: Springer-Verlag, 1985.
38. A. Pinkus, "Approximation theory of the mlp model in neural networks," Acta Numerica, vol. 8, pp. 143-195, 1999.
39. V N. Temlyakov, "The best m-term approximation and greedy algorithms," Advances in Computa. Math., vol. 8, no. 3, pp. 249-265, 1998.
40. H. Triebel, Interpolation Theory Function Spaces and Differential Operators. Berlin, Germany: Veb Deutcher Verlag, 1978.
41. H. Triebel, Theory of Function Spaces. Basel: Birkhauser, 1983.
42. M. Vidyasagar, A Theory of Learning and Generalization. New York: Springer-Verlag, 1996.
43. G. Wahba, Spline Models for Observational Data, ser. Regional Conference Series in Aplied Mathematics. Philadelphia, PA: SIAM, 1990.