Influences of variable scales and activation functions on the performances of multilayer feedforward neural networks

Pattern Recognition

Volumn 36, Issue 4, 2003, Pages 869-878

  Daqi, Gao ^a,b   Genxing, Yang ^a  

^a EAST CHINA UNIVERSITY OF SCIENCE AND TECHNOLOGY   (China)

^b EAST CHINA UNIVERSITY OF SCIENCE AND TECHNOLOGY   (China)

Author keywords

Activation functions; Neural networks; Support vector machine; Variable scales

Indexed keywords

NEURAL NETWORKS;

MULTILAYER FEEDFORWARD NETWORKS; SUPPORT VECTOR MACHINE (SVM);

PATTERN RECOGNITION;

ARTIFICIAL NEURAL NETWORK;

EID: 0036555952     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0031-3203(02)00120-6     Document Type: Article

References (33)

1. R.P. Lippmann, An introduction to computing with neural nets, IEEE ASSP Mag. 4 (1987) 4-22.
2. D.E. Rumelhart, J.L. McClelland, Parallel Distributed Processing, MIT Press, Cambridge, MA, 1986.
3. C.M. Bishop, Neural Networks for Pattern Recognition, Clarendon Press, Oxford, 1998.
4. F. Stager, M. Agarwal, Three methods to speed up the training of feedforward and feedback perceptrons, Neural Networks 10 (8) (1997) 1435-1443.
5. S.E. Fahlman, Faster-learning variations on back-propagation: an empirical study, Proceedings of the 1988 Connectionist Model Summer School, Carnegie Mellon University, 1989, pp. 38-51.
6. D. Nguyen, B. Widrow, Improving the learning speed of 2-layer neural networks by choosing initial values of adaptive weights, Proc. IJCNN 3 (1990) 21-26.
7. V.V. Phansalkar, P.S. Sastry, Analysis of the back-propagation algorithm with momentum, IEEE Trans. Neural Networks 5 (3) (1994) 505-507.
8. E.M. Corwin, A.M. Logar, W.J.E. Oldham, An iterative method for training multilayer networks with threshold functions, IEEE Trans. Neural Networks 5 (3) (1999) 507-508.
9. G.D. Magoulas, M.N. Varhatis, G.S. Androulakis, Effective backpropagation training with variable stepsize, Neural Networks 10 (1) (1997) 69-82.
10. A.G. Parlos, F. Fernandez, A.F. Atiya, et al., An accelerated learning algorithm for multilayer perceptron networks, IEEE Trans. Neural Networks 5 (3) (1994) 493-497.
11. R.D. Leone, R. Capparuccia, E. Merelli, A successive overrelaxative backpropagation algorithm for neural-network training, IEEE Trans. Neural Networks 9 (3) (1998) 381-387.
12. S. Ergezinger, E. Thomsen, An accelerated learning algorithm for multilayer perceptrons: optimization layer by layer, IEEE Trans. Neural Networks 6 (1) (1995) 31-42.
13. N.S. Rubanov, The layer-wise method and the backpropagation hybrid approach to learning a feedforward neural network, IEEE Trans. Neural Networks 11 (2) (2000) 295-305.
14. Gao Daqi, Wu Shou yi, An optimization method for the topological structures of feed-forward multi-layer neural networks, Pattern Recognition 31 (9) (1998) 1337-1342.
15. D.C. Psichogios, L.H. Ungar, SVD-NET: an algorithm that automatically selects network structure, IEEE Trans. Neural Networks 5 (3) (1994) 513-515.
16. A. Menon, K. Mehrotra, C.K. Mohan, et al., Characterization of a class of sigmoid functions with applications to neural networks, Neural Networks 9 (5) (1996) 819-835.
17. G. Thimm, P. Moerland, E. Fiesler, The interchangeability of learning rate and gain in backpropagation neural networks, Neural Comput. 8 (2) (1996) 451-460.
18. C.-T. Chen, W.-D. Chang, A feedforward neural network with function shape autotuning, Neural Networks 9 (4) (1996) 627-641.
19. D. Christian, M. John, Note on learning rate schedules for stochastic optimization, in: R.P. Lippmann et al. (Eds.), Neural Information Processing Systems, 1991, pp. 832-838.
20. S. Guarnieri, F. Piazza, A. Uncini, Multilayer feedforward networks with adaptive spline activation function, IEEE Trans. Neural Networks 10 (3) (1999) 672-684.
21. W.L. Buntine, A.S. Weigend, Computing second derivatives in feed-forward neural networks: a review, IEEE Trans. Neural Networks 5 (3) (1994) 480-488.
22. G.-B. Huang, Y.-Q. Chen, H.A. Babri, Classification ability of single hidden layer feedforward neural networks, IEEE Trans. Neural Networks 11 (3) (2000) 799-801.
23. G.-B. Huang, H.A. Babri, Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation function, IEEE Trans. Neural Networks 9 (1) (1998) 224-228.
24. G. Dundar, K. Rose, The effects of quantization on multilayer neural networks, IEEE Trans. Neural Networks 6 (6) (1995) 1445-1451.
25. D. Anguita, S. Ridella, S. Rovetta, Worst case analysis of weight inaccuracy effects in multi perceptrons, IEEE Trans. Neural Networks 10 (2) (1999) 415-419.
26. V.N. Vapnik, An overview of statistical learning theory, IEEE Trans. Neural Networks 10 (5) (1999) 988-999.
27. M. Lehtokangas, Modified cascade-correlation learning for classification, IEEE Trans. Neural Networks 11 (3) (2000) 795-798.
28. J.A.K. Suykens, J. Vandewalle, Training multilayer perceptron classifiers based on a modified support vector method, IEEE Trans. Neural Networks 10 (4) (1999) 907-911.
29. N.B. Karayiannis, Reformulated radial basis neural networks trained by gradient descent, IEEE Trans. Neural Networks 10 (3) (1999) 1121-1138.
30. ftp://ftp.ics.uci.edu/pub/mechine-learning-databases
31. http://www.ics.uci.edu/~mlearn.
32. R.P. Gormann, T.J. Sejnowski, Analysis of hidden units in a layered network trained to classify sonar targets, Neural Networks 1 (1) (1998) 75-89.
33. A. Roy, S. Govil, P. Miranda, An algorithm to generate radial basis function (RBF)-like nets for classification problems, Neural Networks 8 (2) (1995) 179-201.