Learning and approximation capabilities of adaptive spline activation function neural networks

Neural Networks

Volumn 11, Issue 2, 1998, Pages 259-270

  Vecci, Lorenzo ^a   Piazza, Francesco ^a   Uncini, Aurelio ^a  

^a Dipto Elettron Automatica U   (Italy)

Author keywords

Adaptive activation functions; Function shape autotuning; Generalization; Generalized sigmoidal functions; Multilayer perceptron; Neural networks; Spline neural networks

Indexed keywords

APPROXIMATION THEORY; COMPUTATIONAL COMPLEXITY; COMPUTER ARCHITECTURE; COMPUTER SIMULATION; LEARNING SYSTEMS; MATHEMATICAL MODELS;

ADAPTIVE ACTIVATION FUNCTIONS; CATMULL-ROM CUBIC SPLINE; FUNCTION SHAPE AUTOTUNING; GENERALIZED SIGMOIDAL FUNCTIONS; SIGMOIDAL ACTIVATION FUNCTION;

NEURAL NETWORKS;

ADAPTATION; ARTICLE; BEHAVIOR; CONTROL SYSTEM; CYTOARCHITECTURE; LEARNING; MATHEMATICAL ANALYSIS; NERVE CELL NETWORK; NONHUMAN; OSCILLATION; PATTERN RECOGNITION; PRIORITY JOURNAL;

EID: 0032029155     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(97)00118-4     Document Type: Article

References (25)

1. Amari S. A universal theorem on learning curves. Neural Networks. 6:1993;161-166.
2. Amari S., Murata N. Statistical theory of learning curves under entropic loss criterion. Neural Computation. 5:1993;140-153.
3. Back A.D., Tsoi A.C. A simplified gradient algorithm for IIR synapse Multilayer Perceptron. Neural Networks. 5:1993;456-462.
4. Campolucci, P., Capparelli, F., Guarnieri, S., Piazza, F., & Uncini, A., Neural networks with adaptive spline activation function. Proceedings of IEEE MELECON 96, Bari Italy, pp. 1442-1445.
5. Chen C. T., Chang W. D. A feedforward neural network with function shape autotuning. Neural Networks. 9:1996;627-641.
6. Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals, and Systems, New York: Springer Verlag, pp. 303-314.
7. Geman S., Bienenstock E., Doursat R. Neural networks and the bias/variance dilemma. Neural Computation. 4:1992;1-58.
8. Girosi F., Jones M., Poggio T. Regularization theory and neural networks architectures. Neural Computation. 7:1995;219-269.
9. Guarnieri, S., Piazza, F., & Uncini, A. (1995). Multilayer neural networks with adaptive spline-based activation functions. Proceedings of the International Neural Network Society Annual Meeting WCNN 95, Washington D.C., USA, pp. I695-I699.
10. Guyon, I., Vapnik, V., Boser, B., Bottou, L., & Solla, S. A. (1992). Structural Risk Minimization for Character Recognition. In J. E. Moody, S. J. Hanson, and R. P. Lippmann, eds, Advances in Neural Information Processing Systems, vol. 4, Morgan Kauffmann, pp. 471-479.
11. Hassoun, M. H. (1996). Fundamentals of Artificial Neural Networks. Cambridge, MA: MIT.
12. Holden S.B., Rayner P.J.W. Generalization and PAC learning: some new results for the class of generalized single-layer networks. IEEE Transactions on Neural Networks. 6(2):1995;368-380.
13. Hwang J.-N., Lay S.-R., Maechler M., Martin R.D., Schimert J. Regression modeling in back-propagation and projection pursuit learning. IEEE Transactions on Neural Networks. 5(2):1994;342-353.
14. Moody, J.E. (1992). The effective number of parameters: an analysis of generalization and regularization in nonlinear learning systems. In J.E. Moody, S.J. Hanson, & R.P. Lippmann, eds, Advances in Neural Information Processing Systems, vol. 4, Morgan Kauffmann, pp. 847-854.
15. Moody, J.E., & Utans, J. (1994). Architecture selection startegies for neural networks: application to corporate bond rating prediction. In A.N. Refenes, ed., Neural Networks in the Capital Markets, John Wiley.
16. Niyogi P., Girosi F. On the relationship between generalization error hypothesis complexity and sample complexity for radial basis functions. Neural Computation. 8:1996;819-842.
17. Piazza, F., Smerilli, S., Uncini, A., Griffo, M., & Zunino, R. (1996). Fast spline neural networks for image compression. In WIRN96, Vietri sul Mare, Italy.
18. Piazza, F., Uncini, A., & Zenobi, M. (1992). Artificial neural networks with adaptive polynomial activation function. Proc. of the IJCNN, vol. 2, Bejing, China, pp. 343-349.
19. Piazza, F., Uncini, A., & Zenobi, M. (1993). Neural networks with digital LUT activation function, Proc. of the IJCNN, vol. 2, Nagoya, Japan, pp. 1401-1404.
20. Poggio T., Girosi F. Networks for approximation and learning. Proceedings of the IEEE. 78(9):1990;1481-1497.
21. Poggio T., Girosi F. Regularization algorithms for learning that are equivalent to multilayer networks. Science. 247:1990;978-982.
22. Reed R., Marks II R.J., Oh O. Similarities of error regularization, sigmoid gain scaling, target smoothing, and training with jitters. IEEE Transactions on Neural Networks. 6(3):1995;529-538.
23. Stinchcombe, M., & White, H. (1989). Universal approximation using feedforward networks with non-sigmoid hidden layer activation functions. Proc. of the IJCNN, vol. 1, Washington D.C., USA, pp. 613-617.
24. Widrow B., Lehr M.A. 30 Years of adaptive neural networks: perceptron, madaline, and backpropagation. Proceedings of the IEEE. 78(9):1990;1415-1442.
25. Wolpert, D.H. (1995). On bias plus variance. Technical Report SFI TR 95-08-074, Santa Fe Institute.