Nonlinearity creates linear independence

Advances in Computational Mathematics

Volumn 5, Issue 1, 1996, Pages 189-203

  Ito, Yoshifusa ^a  

^a TOYOHASHI UNIVERSITY OF TECHNOLOGY   (Japan)

Author keywords

Activation function; Complete linear independence; Fourier transform; Linear independence; Neural network; Nonlinear mapping; Plane wave; Strict linear independence

Indexed keywords

EID: 21344441889     PISSN: 10197168     EISSN: None     Source Type: Journal    
DOI: 10.1007/bf02124743     Document Type: Article

References (18)

1. E.K. Blum and L.K. Li, Approximation theory and feedforward networks, Neural Networks 4, 1991, 511-515.
2. G.Cybenko, Approximation by superpositions of a sigmoidal function, Math. Control Signal Systems 2, 1989, 303-314.
3. K. Funahashi, On the approximate realization of continuous mappings by neural networks, Neural Networks 2, 1989, 183-192.
4. L.M. Gel'fand and G.E. Shilov, Generalized Functions 2 (English translation), Academic Press, 1968.
5. K. Hornik, M. Stinchcombe and H. White, Multilayer feedforward networks are universal approximators, Neural Networks 2, 1989, 359-366.
6. K. Hornik, M. Stinchcombe and H. White, Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks, Neural Networks 3, 1990, 551-560.
7. Y. Ito, Representation of functions by superpositions of a step or sigmoid function and their applications to neural network theory, Neural Networks 4, 1991, 385-394.
8. Y. Ito, Approximation of functions on a compact set by finite sums of a sigmoid function without scaling, Neural Networks 4, 1991, 817-826.
9. Y. Ito, Finite mapping by neural networks and truth functions, Math. Sci. 17, 1992, 69-77.
10. d by linear combinations of shifted rotations of a sigmoid function with and without scaling, Neural Networks 5, 1992, 105-115.
11. Y. Ito, Approximations of differentiable functions and their derivatives on compact sets by neural networks, Math. Sci. 18, 1993, 11-19.
12. Y. Ito, Approximation capability of layered neural networks with sigmoid units on two layers, Neural Comp. 6, 1994, 1233-1243.
13. Y. Ito, Differentiable approximation by means of the Radon transform and its applications to neural networks, J. Comput. Appl. Math. 55, 1994, 31-50.
14. Y. Ito and K. Saito, Superposition of linearly independent functions and finite mappings by neural networks, Math. Sci. 19, 1995, to appear.
15. W.S. McCulloch and W. Pitts, A logical calculus of the ideas immanent in nervous activity, Bull. Math. Biophys. 5, 1943, 115-133.
16. D.E. Rumelhart, G.E. Hinton and R.J. William, Learning representations by back-propagating errors, Nature 323, 1986, 533-536.
17. L. Schwartz, Theorie des Distributions, Hermann, Paris, 1966.
18. H.J. Sussmann, Uniqueness of the weights for minimal feedforward nets with a given input-output map, Neural Networks 5, 1992, 589-593.