A non-sigmoidal activation function for feedforward artificial neural networks

Proceedings of the International Joint Conference on Neural Networks

Volumn 2015-September, Issue , 2015, Pages

  Chandra, Pravin ^a   Ghose, Udayan ^a   Sood, Apoorvi ^a  

^a GURU GOBIND SINGH INDRAPRASTHA UNIVERSITY   (India)

Author keywords

[No Author keywords available]

Indexed keywords

CHEMICAL ACTIVATION; FEEDFORWARD NEURAL NETWORKS; FUNCTIONS; HYPERBOLIC FUNCTIONS; NEURAL NETWORKS; POLYNOMIAL APPROXIMATION; RADIOACTIVITY LOGGING;

ACTIVATION FUNCTIONS; FEED-FORWARD ARTIFICIAL NEURAL NETWORKS; FUNCTION APPROXIMATION; HYPERBOLIC TANGENT FUNCTION; POLYNOMIAL FUNCTIONS; SIGMOIDAL ACTIVATION FUNCTIONS; SIGMOIDAL FUNCTIONS; UNIVERSAL APPROXIMATION PROPERTIES;

EXPONENTIAL FUNCTIONS;

EID: 84951100009     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2015.7280440     Document Type: Conference Paper

References (34)

1. W. McCulloch and W. Pitts," A logical calculus of the ideas immanent in nervous activity," Bulletin of Mathematical Biophysics, vol. 5, pp. 115-133, 1943.
2. P. Chandra and Y Singh, "Feedforward sigmoidal networks-equicontinuity and fault-tolerance," IEEE Transactions on Neural Networks, vol. 15, no. 6, pp. 1350-1366,2004.
3. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning representations by back-propagating errors," Nature, vol. 323, pp. 533-536, Oct. 1986.
4. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning internal representations by error propagation," in Parallel Distributed Processing: Volume I: Foundations, D. E. Rumelhart, J. L. McClelland, and The PDP Research Group, Eds. Cambridge: MIT Press, 1987, pp. 318-362.
5. -, "Learning representations by back-propagating errors," in Neurocomputing: foundations of research, J. A. Anderson and E. Rosenfeld, Eds. Cambridge, MA, USA: MIT Press, 1988, pp. 696-699.
6. M. Riedmiller and H. Braun, "A direct adaptive method for faster backpropagation learning: The RPROP algorithm," in Proc. of IEEE conference on Neural Networks, vol. I, San Francisco, 2010, pp. 586-591.
7. M. Riedmiller, "Advanced supervised learning in multi-layer perceptrons from backpropagation to adaptive learning algorithms," Computer Standards & Interfaces, vol. 16, no. 3, pp. 265-278, 1994.
8. C. Igel and M. Husken, "Empirical evaluation of the improved rprop learning algorithms," Neurocomputing, vol. 50, no. 0, pp. 105-123, 2003.
9. M. T. Hagan and M. B. Menhaj, "Training feed forward networks with the Marquardt algorithm," IEEE Transactions on Neural Networks, vol. 5, pp. 989-993, 1994.
10. M. Moller, "A scaled conjugate gradient algorithm for fast supervised learning," Aarhus University Computer Science Department, Aarhus, Denmark, Tech. Rep. Technical Report PB-339" 1990.
11. R. Battiti, "First and second order methods for learning: btetween steepest descent and Newton's method," Neural Computation, vol. 4, no. 2, pp. 141-166, 1992.
12. A. R. Gallant and H. White, "There exists a neural network that does not make avoidable mistakes," in Proceedings of the Second International Joint Conference on Neural Networks, vol. 1, 1988, pp. 593-606.
13. S. M. Carroll and B. W. Dickinson, "Construction of neural networks using the Radon transform," in Proc. of the IJCNN, vol. 1, 1989, pp. 607-611.
14. G. Cybenko, "Approximation by superposition of a sigmoidal function," Mathematics of Control, Signal and Systems, vol. 5, pp. 233-243, 1989.
15. K. Funahashi, "On the approximate realization of continuous mappings by neural networks," Neural Networks, vol. 2, pp. 183-192, 1989.
16. K. Hornik, M. Stinchcombe, and H. White, "Multilayer feedforward networks are universal approximators," Neural Networks, vol. 2, pp. 359-366, 1989.
17. M. Stinchcombe and H. White, "Universal approximation using feedforward networks with non-sigmoid hidden layer activation functions," in Neural Networks, 1989. IJCNN., International Joint Conference on, 1989, pp. 613-617 vol. I.
18. K. Hornik, "Approximation capabilities of multilayer feed forward networks," Neural Networks, vol. 4, no. 2, pp. 251-257, 1991.
19. M. Leshno, V. Y Lin, A. Pinkus, and S. Schocken, "Multilayer feedforward networks with a non-polynomial activation function can approximate any function," Neural Networks, vol. 6, pp. 861-867, 1993.
20. A. Pinkus, "Approximation theory of the MLP model in neural networks," Acta NlImerica, vol. 8, pp. 143-195, 1999.
21. S. Guarnieri, F. Piazza, and A. Uncini, "Multilayer feed forward networks with adaptive spline activation function," Nellral Networks, IEEE Transactions on, vol. 10, no. 3, pp. 672-683, May 1999.
22. M. Solazzi and A. Uncini, "Artificial neural networks with adaptive multidimensional spline activation functions," in Nellral Networks, 2000. UCNN 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on, vol. 3, 2000, pp. 471-476 vol. 3.
23. J.-N. Hwang, S.-R. Lay, M. Maechler, R. Martin, and J. Schimert, "Regression modeling in back-propagation and projection pursuit learning," Nellral Networks, IEEE Transactions on, vol. 5, no. 3, pp. 342-353, May 1994.
24. W. Ouch and N. Jankowski, "Survey of neural network transfer functions," Neural Computing Surveys, vol. 2, pp. 163-212, 1999.
25. P. Chandra, "Sigmoidal function classes for feedforward artificial neural networks," Nellral Processing Letters, vol. 18, no. 3, pp. 205-215,2003.
26. S. S. Sodhi and P. Chandra, "Bi-modal derivative activation function for sigmoidal feed forward networks," Nellrocompllting, vol. 143, no. 0, pp. 182-196,2014.
27. Y. LeCun, L. Bottou, G. B. Orr, and K.-R. Muller, "Efficient backprop," in Nellral Networks: Tricks of the trade, ser. LNCS:1524, G. B. Orr and K.-R. MUlier, Eds. Berlin: Springer, 1998, pp. 9-50.
28. S. Lee and R. M. Kil, "A Gaussian potential function network with hierarchically self-organizing learning," Nellral Networks, vol. 4, no. 2, pp. 207-224, 1991.
29. S. Haykin, Nellral Networks: A Comprehensive Foundation. New Jersey: Prentice Hall, Inc. , 1999.
30. L. Breiman, "The PI method for estimating multivariate functions from noisy data," Technometrics, vol. 3, no. 2, pp. 125-160, 1991.
31. V. Cherkassky, D. Gehring, and F. MUlier, "Comparison of adaptive methods for function estimation from samples," IEEE Transactions on Nellral Networks, vol. 7, no. 4, pp. 969-984, 1996.
32. V. Cherkassky and F. MUlier,L earning from Data-Concepts, Theory and Methods. New York: John Wiley, 1998.
33. M. Maechler, O. Martin, J. Schimert, M. Csoppenszky, and J. Hwang, "Projection pursuit learning networks for regression," in Proc. of the 2nd International IEEE Conference on Tools for Artificial Intelligence, 1990, pp. 350-358.
34. J. O. Gibbons and S. Chakraborti, Nonparametric Statistical Inference. New York: Marcel Dekker, Inc., 2003.