Faster training using fusion of activation functions for feed forward neural networks

International Journal of Neural Systems

Volumn 19, Issue 6, 2009, Pages 437-448

  Asaduzzaman, M D ^a   Shahjahan, M D ^a   Murase, Kazuyuki ^b  

^a KHULNA UNIVERSITY OF ENGINEERING AND TECHNOLOGY   (Bangladesh)

^b UNIVERSITY OF FUKUI   (Japan)

Author keywords

Activation function; Combination of activation functions; Convergence; Neural network; Training

Indexed keywords

ACTIVATION FUNCTIONS; BENCHMARK CLASSIFICATION; BREAST CANCER; CHAOTIC TIME SERIES PREDICTION; ERROR FUNCTION; HEART DISEASE; MULTI-LAYER NETWORK; MULTILAYER FEEDFORWARD NEURAL NETWORKS; NEURAL NETWORK TRAINING; SUBGOALS; TIME SERIES PREDICTION; TRAINING METHODS;

ARTIFICIAL INTELLIGENCE; BACKPROPAGATION; GLASS; MULTILAYERS; TIME SERIES; WINE;

MULTILAYER NEURAL NETWORKS;

ALGORITHM; ANIMAL; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; COMPUTER SIMULATION; HUMAN; INFORMATION PROCESSING; MATHEMATICAL PHENOMENA; MATHEMATICS; METHODOLOGY; STATISTICAL MODEL; TEACHING; TIME;

ALGORITHMS; ANIMALS; ARTIFICIAL INTELLIGENCE; AUTOMATIC DATA PROCESSING; COMPUTER SIMULATION; HUMANS; LINEAR MODELS; MATHEMATICAL CONCEPTS; MATHEMATICS; NEURAL NETWORKS (COMPUTER); TEACHING; TIME FACTORS;

EID: 73949099341     PISSN: 01290657     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0129065709002130     Document Type: Article

References (23)

1. S. L. Hung and H. Adeli, Parallel backpropagation learning algorithms on Cray Y-MP8/864 Supercomputer, Neurocomputing 5(6) (1993) 287-302.
2. S. L. Hung and H. Adeli, Object-oriented back propagation and its application to structural design, Neurocomputing 6(1) (1994) 45-55.
3. A. Dharia and H. Adeli, Neural network model for rapid forecasting of freeway link travel time, Engineering Applications of Artificial Intelligence 16(7- 8) (2003) 607-613.
4. S. Hooshdar and H. Adeli, Toward intelligent variable message signs in freeway work zones: A neural network approach, Journal of Transportation Engineering 130(1) (2004) 83-93.
5. X. H. Yu and G. A. Chen, Efficient backpropagation learning using optimal learning rate and momentum, Neural Networks 10(3) (1997) 517-527.
6. X. H. Yu, G. A. Chen and S. X. Cheng, Dynamic learning rate optimization of the backpropagation algorithm, IEEE Transactions on Neural Networks (1995) 669-677.
7. S.-C. Ng, C.-C. Cheung and S.-H. Leung, Magnified gradient function with deterministic weight modification in adaptive learning, IEEE Trans. Neural Network 15(6) (2004) 1411-1423.
8. A. Cichocki and R. Zdunek, Multilayer nonnegative matrix factorization using projected gradient approaches, Int. J. Neural Systems 17(6) (2007) 431-446.
9. J. Menke and T. Martinez, Improving supervised learning by adapting the problem to the learner, Int. J. Neural Systems 19(1) (2009) 1-9.
10. C. C. Yu and B. D. Liu, A backpropagation algorithm with adaptive learning rate and momentum coefficient, (WCCI 2002). IEEE, (2002) 1218-1223.
11. G. D. Magoulas, V. P. Plagianakos and M. N. Vrahatis, Globally, convergent algorithms with local learning rates, IEEE Trans. Neural Network 13(3) (2002) 774-779.
12. Y. H. Zweiri, J. F. Whidborne, K. Althoefer and L. D. Seneviratne, A three-term backpropagation algorithm, Neurocomputing 50 (2003) 305-318.
13. S. Abid and F. Fnaiech, Fast training of Multilayer perceptrons with Least mean Fourth (LMF) Algorithm, Int. J. Soft Computing 3(5) (2008) 359-367.
14. J. Kamruzzaman, Improving convergence of backpropagation algorithm using exponential cost function, IEEJ Trans. EIS. 123(5) (2003) 999-1003.
15. R. Tawel, Does the neuron learn like synapse? in Advance in Neural Information Processing System, ed. D. Touretzy, Vol.1 (Morgan Kaufmann, 1989), pp. 169-176.
16. J. Kamruzzaman, Arctangent activation function to accelerate backpropagation learning, IEICE Transactions of Fundamentals of Electronics, Communications and Computer Sciences, (2002) 2373-2376.
17. J. Bilski, The backpropagation learning with logarithmic transfer function, Proc. Fifth Conf. on Neural Networks and Soft Computing, (Poland) 2000 71-76.
18. S. E. Fahlman and C. Lebiere, The cascadecorrelation learning architecture, in Advances in Neural Information Processing Systems, ed. D. S. Touretzky, Vol.2 (San Mateo, CA: Morgan Kaufmann 1990) 524-532.
19. L. L. Prechelt, PROBEN1-A set of neural network benchmark problems and benchmarking rules, Technical Report 21/94, Faculty of Informatics, (University of Karlsruhe, 1994).
20. S. E. Fahlman, Fast learning variations on backpropagation: An empirical study, in Proc. Connectionist Models Summer School, D. Touretzky, G. Hinton and T. Sejnowski, (eds.) (San Mateo, CA, 1989) 38-51.
21. M. Riedmiller and H. Braun, A direct adaptive method for faster backpropagation learning: The RPROP algorithm, in Proc. Int. Conf. Neural Networks, Vol.1, (1993) 586-591.
22. N. K. Treadgold and T. D. Gedeon, Simulated annealing and weight decay in adaptive learning: The SARPROP algorithm, IEEE Trans. Neural Networks 9 (1998) 662-668.
23. M. T. Hagan and M. B. Menhaj, Training feedforward networks with the Marquardt algorithm, IEEE Trans. Neural Networks 5 (1994) 989-993.