Regularising neural networks using flexible multivariate activation function

Neural Networks

Volumn 17, Issue 2, 2004, Pages 247-260

  Solazzi, Mirko ^a   Uncini, Aurelio ^b  

^a UNIVERSITÀ POLITECNICA DELLE MARCHE   (Italy)

^b SAPIENZA UNIVERSITY OF ROME   (Italy)

Author keywords

Adaptive activation functions; Generalisation; Generalised sigmoidal functions; Multilayer perceptron; Neural networks; Regularisation; Spline; Spline neural networks

Indexed keywords

COMPUTATIONAL COMPLEXITY; LEARNING ALGORITHMS; LEARNING SYSTEMS; NONLINEAR SYSTEMS; PROBLEM SOLVING;

MULTIVARIATE ACTIVATION FUNCTIONS; STRUCTURAL COMPLEXITY;

NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; COMPUTER SIMULATION; INFORMATION PROCESSING; LEARNING; MATHEMATICAL ANALYSIS; MULTIVARIATE ANALYSIS; NERVE CELL; PRIORITY JOURNAL;

EID: 1242338186     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(03)00189-8     Document Type: Article

References (34)

1. Amari S. A universal theorem on learning curves. Neural Networks. 6:(2):1993;161-166.
2. Catmull E., Rom R. A class of local interpolating splines. Barnhill R.E., Riesenfeld R.F. Computer aided geometric design. 1974;317-326 Academic Press, New York.
3. Cibenko G. Approximation by Superposition of a sigmoidal function. Mathemathics of Control, Signals and Systems. 2:1989;303-314.
4. Funahashi K.I. On the approximate realisation of continuous mappings by neural networks. Neural Networks. 2:1989;183-192.
5. Girosi F., Jones M., Poggio T (1993). Priors, stabilizers and basis functions: from regularisation to radial, tensor and additive splines. Artificial Intelligence Laboratory, Massachusetts Inst. Technol., A.I. Memo No.1430.
6. Girosi F., Jones M., Poggio T. Regularisation theory and neural networks architectures. Neural Computations. 7:1995;219-269.
7. Guarnieri S., Piazza F., Uncini A. Multilayer feedforward networks with adaptive spline activation function. IEEE Transactions on Neural Network. 10:(3):1999;672-683.
8. Haykin S. Adaptive filter theory. 3rd ed:1996;Prentice Hall, Englewood Cliffs, NJ.
9. Haykin S. Neural networks - A comprehensive foundation. 2nd ed:1999;Prentice Hall, Englewood Cliffs, NJ.
10. Hecht-Nielsen R. Kolmogorov mapping neural network existence theorem. IEEE First International Conference on Neural Networks. vol. 3:1987;. pp. 11-13.
11. Hinton G.E. Learning distributed representation of concepts. Proceedings of the Eighth Annual Conference of the Cognitive Science Society. 1986;Lawrence Erlbaum Associates, Hillsdale, NJ.
12. Hornik K. Multilayer feedforward networks are universal approximators. Neural Networks. 2:1989;359-366.
13. Hwang J.-N., Lay S.-R., Maechler M., Martin R.D., Schimert J. Regression modeling in backpropagation and projection pursuit learning. IEEE Transactions on Neural Networks. 5:(2):1994;342-353.
14. Ito Y. Representation of functions by superposition of a step or sigmoid function nd their application to neural network theory. Neural Networks. 4:1991;385-394.
15. Ito Y. Approximation of functions on a compact set by finite sums sigmoid function without scaling. Neural Networks. 4:1991;817-826.
16. Karam G., Sari H. A data predistortion technique with memory for QAM radio systems. IEEE Transactions on Communications. 39:(2):1991.
17. Kolmogorov A.N. On the representation of continuous functions of several variables by superposition of continuous functions of one variable and addition. Doklady Akademii Nauk SSSR. 114:1957;953-956.
18. Kurkova V. Kolmogorov's theorem and multilayer neural networks. Neural Networks. 5:1992;501-506.
19. Lang K., Witbrock. Learning to tell two spiral apart. Proceedings of the 1988 Connectionist Models Summer School. 1988;Morgan Kaufmann, San Mateo, CA. pp. 52-59.
20. Piazza F., Uncini A., Zenobi M. Neural networks with adaptive polynomial activation function. Proceedings of the IJCNN International Joint Conference on Neural Networks, Beijing, China. vol. 2:1992;. pp. 343-349.
21. Piazza F., Uncini A., Zenobi M. Neural networks with digital LUT activation function. Proceedings of IJCNN, Nagoya, Japan. 1993;II-1401-II-1404.
22. Poggio T. Visual algorithms. Braddick O.J., Sleigh A.C. Physical and biological processing of images. 1983;128-135 Springer, New YorK.
23. Proakis J.G. Digital communications. 3rd ed:1995;McGraw-Hill, New York.
24. Rumelhart D., Hinton E., Williams J. Learning integral representation by error propagation. Parallel Distributed Proceedings. Vol. 1:1986;MIT Press, Cambridge, MA. pp. 318-364.
25. Solazzi M., Piazza F., Uncini A. An adaptive spline nonlinear function for blind signal processing. Proceedings of the Workshop on Neural Networks for Signal Processing X. 2000;. pp. 396-404.
26. Solazzi M., Uncini A., Di Claudio E.D., Parisi R. Complex discriminative learning bayesian neural equalizer. Signal Processing. 2003;. in press.
27. Sprecher D.A. On the structure of continuous functions of several variables. Transactions of Mathematical Society. 115:1964;340-355.
28. Tikhonov A.N. Solution of incorrectly formulated problems and the regularisation method. Soviet Mathematical Doklady. 4:1963;1035-1038.
29. Tikhonov A.N., Arsenin V.Y. Solution of ill-posed problems. 1977;W.H. Winston, Washington, DC.
30. Unser M. A perfect fit for signal and image processing. IEEE, Signal Processing Magazine. 1999;22-38.
31. Uncini A., Vecci V., Campolucci P., Piazza F. Complex-valued neural networks with adaptive spline activation function for digital radio links nonlinear equalization. IEEE Transactions on Signal Processing. 47:(2):1999.
32. Van Dam F. Fundamentals of interactive computer graphics. 1982;Addison-Wesley, Reading, MA. ISBN 0-201-14468-9.
33. Vecci L., Piazza F., Uncini A. Learning and approximation capabilities of adaptive spline activation function neural networks. Neural Networks. 11:(2):1998;259-270.
34. Widrow B., Lehr M.A. 30 Years of adaptive neural networks: perceptron, Madaline, and backpropagation. Proceedings of the IEEE. 78:(9):1990;1415-1442.