Bi-modal derivative activation function for sigmoidal feedforward networks

Neurocomputing

Volumn 143, Issue , 2014, Pages 182-196

  Singh Sodhi, Sartaj ^a   Chandra, Pravin ^a  

^a GURU GOBIND SINGH INDRAPRASTHA UNIVERSITY   (India)

Author keywords

Activation function; Bi modal activation function; Sigmoidal feed forward artificial neural network; Sigmoidal function class

Indexed keywords

APPROXIMATION ALGORITHMS; BACKPROPAGATION ALGORITHMS; BISMUTH COMPOUNDS; CHEMICAL ACTIVATION; NEURAL NETWORKS;

ACTIVATION FUNCTIONS; FEED-FORWARD ARTIFICIAL NEURAL NETWORKS; FEED-FORWARD NETWORK; FUNCTION APPROXIMATION; LEVENBERG-MARQUARDT ALGORITHM; RESILIENT BACKPROPAGATION; SIGMOIDAL FUNCTIONS; UCI MACHINE LEARNING REPOSITORY;

RADIOACTIVITY LOGGING;

ARTICLE; ARTIFICIAL NEURAL NETWORK; BIMODAL DERIVATIVE ACTIVATION FUNCTION; BROYDEN FLETCHER GOLDFARB SHANNO ALGORITHM; CLASSIFICATION ALGORITHM; DATA PROCESSING; LEVENBERG MARQUARDT ALGORITHM; MATHEMATICAL COMPUTING; MATHEMATICAL PHENOMENA; PRIORITY JOURNAL; REGRESSION ANALYSIS; RESILIENT BACKPROPAGATION ALGORITHM; SCALED CONJUGATE GRADIENT ALGORITHM; SIGMOIDAL FEEDFORWARD NETWORK;

EID: 84904803668     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2014.06.007     Document Type: Article

References (41)

1. Anastasiadis A.D., Magoulas G.D., Vrahatis M.N. New globally convergent training scheme based on the resilient propagation algorithm. Neurocomputing 2005, 64:253-270. Trends in Neurocomputing: 12th European Symposium on Artificial Neural Networks 2004.
2. K. Bache, M. Lichman, UCI machine learning repository, 2013.
3. Barron A. Universal approximation bounds for superposition of a sigmoidal function. IEEE Trans. Inf. Theory 1993, 3:930-945.
4. Blum E.K., Li L.K. Approximation theory and feedforward networks. Neural Netw. 1991, 4:511-515.
5. Breiman L. The PI method for estimating multivariate functions from noisy data. Technometrics 1991, 3:125-160.
6. S.M. Caroll, D.W. Dickinson, Construction of neural nets using the radon transform, in: International Joint Conference on Neural Networks, IEEE, 1989, pp. 607-611.
7. Chandra P. Sigmoidal function classes for feedforward artificial neural networks. Neural Process. Lett. 2003, 18:205-215.
8. Chandra P., Singh Y. A case for the self-adaptation of activation functions in FFANNs. Neurocomputing 2004, 56:447-454.
9. Chandra P., Singh Y. Feedforward sigmoidal networks-equicontinuity and fault-tolerance. IEEE Trans. Neural Netw. 2004, 15:1350-1366.
10. Chen T., Chen H. Universal approximation to nonlinear operators by neural networks with arbitrary activation function and its application to dynamical systems. IEEE Trans. Neural Netw. 1995, 6:911-917.
11. Cherkassky V., Gehring D., Mülier F. Comparison of adaptive methods for function estimation from samples. IEEE Trans. Neural Netw. 1996, 7:969-984.
12. Cherkassky V., Mülier F. Learning from Data-Concepts, Theory and Methods 1998, John Wiley, New York.
13. Cortez P., Cerdeira A., Almeida F., Matos T., Reis J. Modeling wine preferences by data mining from physicochemical properties. Decis. Support Syst. 2009, 47:547-553.
14. Cybenko G. Approximation by superpositions of a sigmoidal function. Math. Control Signals Syst. 1989, 2:303-314.
15. Duch W., Jankowski N. Survey of neural transfer functions. Neural Comput. Surv. 1999, 2:163-212.
16. Funahashi K. On the approximate realization of continuous mapping by neural networks. Neural Netw. 1989, 2:183-192.
17. Gill P.E., Murray W., Wright M.H. Practical Optimization 1997, Academic Press Inc. [Harcourt Brace Jovanovich Publishers], London.
18. Goh S.L., Mandic D.P. Recurrent neural networks with trainable amplitude of activation functions. Neural Netw. 2003, 16:1095-1100.
19. Hagan M., Menhaj M.B. Training feedforward networks with the Marquardt algorithm. IEEE Trans. Neural Netw. 1994, 5:989-993.
20. Haykin S. Neural Networks. A Comprehensive Foundations 1999, Prentice-Hall, Englewood Cliffs, NJ.
21. Hornik K., Stinchcombe M., White H. Multilayer feedforward networks are universal approximators. Neural Netw. 1989, 2:359-366.
22. Huber P.J., Ronchetti E.M. Robust Statistics 2009, John Wiley & Sons, Inc., New York.
23. Igel C., Hüsken M. Empirical evaluation of the improved RPROP learning algorithms. Neurocomputing 2003, 50:105-123.
24. Jones L.K. Constructive approximations for neural networks by sigmoidal functions. Proc. IEEE 1990, 78:1586-1589. Corrections and additions, Proc. IEEE (1991) 79, pp. 243.
25. Dennis J.E., Schnabel R.B. Numerical Methods for Unconstrained Optimization and Nonlinear Equations 1983, Prentice Hall, Englewood Cliffs, NJ.
26. LeCun Y., Bottou L., Orr G.B., Müller K.R. Efficient backprop. Neural Networks: Tricks of the trade, Lecture Notes in Computer Science, vol. 1524 1998, 9-50. Springer, Berlin. G.B. Orr, K.R. Müller (Eds.).
27. M. Maechler, D. Martin, J. Schimert, M. Csoppenszky, J. Hwang, Projection pursuit learning networks for regression, in: Proceedings of the 2nd International IEEE Conference on Tools for Artificial Intelligence, 1990, pp. 350-358.
28. Mhaskar H., Micchelli C. Degree of approximation by neural and translation networks with a single hidden layer. Adv. Appl. Math. 1995, 16:151-183.
29. Mhaskar H., Micchelli C.A. Approximation by superposition of sigmoidal and radial basis functions. Adv. Appl. Math. 1992, 13:350-373.
30. Moller M.F. A scaled conjugate gradient algorithm for fast supervised learning. Neural Netw. 1993, 6:525-533.
31. D. Nguyen, B. Widrow, Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights, in: 1990 IJCNN International Joint Conference on Neural Networks, 1990, vol. 3, pp. 21-26.
32. Pinkus A. Approximation theory of the MLP model in neural networks. Acta Numer. 1999, 143-195.
33. Riedmiller M. Advanced supervised learning in multi-layer perceptrons-from backpropagation to adaptive learning algorithms. Comput. Stand. Interfaces 1994, 16:265-278.
34. M. Riedmiller, H. Braun, A direct adaptive method for faster backpropagation learning: the RPROP algorithm, in: IEEE International Conference on Neural Networks, 1993, vol. 1, pp. 586-591.
35. Sanguineti M. Universal approximation by ridge computational models and neural networks. a survey. Open Appl. Math. J. 2008, 2:31-58.
36. Scarselli F., Tsoi A.C. Universal approximation using feedforward neural networks. a survey of some existing methods, and some new results. Neural Netw. 1998, 11:15-37.
37. Trentin E. Networks with trainable amplitude of activation functions. Neural Netw. 2001, 14:471-493.
38. Tsanas A., Xifara A. Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools. Energy Build. 2012, 49:560-567.
39. Wang J., Xu Z. New study on neural networks. the essential order of approximation. Neural Netw. 2010, 23:618-624.
40. Yeh I.C. Modeling of strength of high-performance concrete using artificial neural networks. Cement Concr. Res. 1998, 28:1797-1808.
41. Yu D.S. Approximation by neural networks with sigmoidal functions. Acta Math. Sin. Engl. Ser. 2013, 29:2013-2026.