Approximation properties of local bases assembled from neural network transfer functions

Mathematical and Computer Modelling

Volumn 28, Issue 9, 1998, Pages 43-62

  Meade Jr , A J ^a   Zeldin, B A ^a  

^a RICE UNIVERSITY   (United States)

Author keywords

(Artificial) neural networks; Function approximation; Local bases; Transfer functions

Indexed keywords

EID: 0032213552     PISSN: 08957177     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0895-7177(98)00144-7     Document Type: Article

References (34)

1. 1. G. Cybenko, Approximation by superposition of a sigmoidal function, Mathematics of Control, Signals, and Systems 2, 303-314 (1989).
2. 2. F. Girosi and T. Poggio, Networks and the best approximation property, Biological Cybernetics 63, 169-176 (1990).
3. 3. K. Hornik, Some new results on neural network approximation, Neural Networks 6, 1069-1072 (1993).
4. 4. K. Hornik, M. Stinchcombe and H. White, Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks, Neural Networks 3, 551-560 (1990).
5. 5. Y. Ito, Representation of functions by superpositions of a step or sigmoidal function and their applications to neural network theory, Neural Networks 4, 385-394 (1991).
6. 6. D. Rumelhart, G.E. Hinton and R.J. Williams, Learning internal representations by error propagation, In Parallel Distributed Processing: Exploration in the Microstructure of Cognition, (Edited by D. Rumelhart, J.L. McClelland and the PDP Research Group), Ch. 8, pp. 318-362, MIT Press, Cambridge, MA, (1986).
7. 7. S. Saarinen, R. Bramley and G. Cybenko, Ill-conditioning in neural network training problems, SIAM J. Sci. Comput. 14 (3), 693-714 (1993).
8. 8. T. Poggio and F. Girosi, Regularization algorithms for learning that are equivalent to multilayer networks, Science 247, 978-982 (1990).
9. 9. T. Poggio and F. Girosi, Networks for approximation and learning, Proceedings of the IEEE 78 (9), 1481-1497 (1990).
10. 10. A.N. Tikhonov and V.Y. Arsenin, Solution of Ill-Posed Problems, W.H. Winston, Washington, DC, (1977).
11. 11. C.M. Bishop, Curvature-driven smoothing: A learning algorithm for feedforward networks, IEEE Transactions on Neural Networks 4 (5), 882-884 (1993).
12. 12. C.M. Bishop, Training with noise is equivalent to Tikhonov regularization, Neural Computation 7, 108-116 (1995).
13. 13. A.R. Webb, Functional approximation by feed-forward networks: A least square approach to generalization, IEEE Transactions on Neural Networks 5 (3), 363-371 (1994).
14. 14. R. Reed, R.J. Marks and S. Oh, Similarity of error regularization, sigmoid gain scaling, target smoothing, and training with jitter, IEEE Transactions on Neural Networks 6 (3), 529-538 (1995).
15. 15. B. Igelnik and Y.-H. Pao, Stochastic choice of basis functions in adaptive function approximation and the functional-link net, IEEE Transactions on Neural Networks 6 (6), 1320-1329 (1995).
16. 16. S. Chen, C.F.N. Cowan and P.M. Grant, Orthogonal least square algorithm for radial basis function networks, IEEE Transactions on Neural Networks 2 (2), 302-309 (1991).
17. 17. M. Lehtokangas, J. Saarinen, K. Kaski and P. Huuntanen, Initializing weights of a multilayer perception network by using the orthogonal least squares algorithm, Neural Computation 7, 982-999 (1995).
18. 18. M. Bertero, T. Poggio and V. Torre, Ill-posed problems in early vision, Proceedings of the IEEE 76, 869-889 (1988).
19. 19. D.D. Cox, Multivariate smoothing spline functions, SIAM J. Numer. Anal. 21 (4), 789-813 (1984).
20. 20. G. Wahba, Spline Models for Observational Data, SIAM, Philadelphia, PA, (1990).
21. 21. P. Cardaliaguet and G. Euvrard, Approximation of a function and its derivative with a neural network, Neural Networks 5, 207-220 (1992).
22. 22. K. Jetter, Multivariate approximation from the cardinal interpolation point of view, Approximation Theory VII, (Edited by E.W. Cheney, C.K.Chui and L.L. Schumaker), pp. 131-161, (1992).
23. d), Transaction of the American Mathematical Society 341 (2), 787-806 (1994).
24. 24. M. Leshno, V.Y. Lin, A. Pinkus and S. Schocken, Multilayer feedforward networks with a nonpolynomial activation function can approximate any function, Neural Networks 6, 861-867 (1993).
25. 25. H.N. Mhaskar, Neural networks for optimal approximation of smooth and analytic functions, Neural Computations, 8, 164-177 (1995).
26. 26. P. Grisvard, Elliptic Problems in Nonsmooth Domains, Pitman Advanced Publishing Program, Boston, MA, (1985).
27. 27. R.-Q. Jia and J. Lei, Approximation by multi-integer translates of functions having global support, Journal of Approximation Theory 72, 2-23 (1993).
28. 28. A.R. Barron, Universal approximation bounds for superpositions of a sigmoidal function, IEEE Transactions on Information Theory 39 (3), 930-945 (1993).
29. 29. L.K. Jones, Constructive approximations for neural networks by sigmoidal functions, Proceedings of the IEEE 78 (10), 1586-1589 (1990).
30. 30. L.K. Jones, A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training, The Annals of Statistics 20 (1), 608-613 (1992).
31. 31. F. Girosi and G. Anzellotti, Convergence rates of approximation by translates, Report AI 1288, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, (1992).
32. 32. H.N. Mhaskar, Neural network for optimal approximation of smooth and analytic functions, Neural Computation 8, 164-177 (1996).
33. 33. G.G. Lorentz, Approximation of Functions, Chelsea Publishing, New York, (1986).
34. 34. A.J. Meade and A.A. Fernandez, Solution of nonlinear ordinary differential equations by feedforward neural networks, Mathl. Comput., Modelling 20 (9), 19-44 (1994).