A method to improve the performance of multilayer perceptron by utilizing various activation functions in the last hidden layer and the least squares method

Neural Processing Letters

Volumn 34, Issue 3, 2011, Pages 293-303

  Halawa, Krzysztof ^a  

^a WROCLAW UNIVERSITY OF TECHNOLOGY   (Poland)

Author keywords

Activation functions; Least squares method; Multilayer perceptron

Indexed keywords

ACTIVATION FUNCTIONS; HIDDEN LAYERS; LEARNING PROCESS; LEAST SQUARES METHODS; MULTILAYER PERCEPTRON; NEURON ACTIVATION FUNCTION; SINGLE-STEP; SQUARED ERRORS;

ELECTRIC LOADS; MULTILAYERS; PATTERN RECOGNITION SYSTEMS;

LEAST SQUARES APPROXIMATIONS;

EID: 84855563455     PISSN: 13704621     EISSN: 1573773X     Source Type: Journal    
DOI: 10.1007/s11063-011-9199-4     Document Type: Article

References (17)

1. Al-Batah MS, Isa NAM, Zamli KZ, Azizli KA (2010) Modified recursive least squares algorithm to train the hybrid multilayered perceptron (HMLP) network. Appl Soft Comput 10:236-244
2. Azimi-Sadjadi MR, Liou RJ (1992) Fast learning process of multilayer neural networks using recursive least squares method. IEEE Trans Signal Process 40(2):446-450
3. Nelles O (2001) Nonlinear system identification: from classical approaches to neural networks and fuzzy models. Springer, Berlin, pp 253-255
4. Tseng CS, Chen CS (2004) Performance comparison between the training method and the numerical method of the orthogonal neural network in function approximation. Int J Intell Syst 19(12):1257-1275
5. Zhu C, Shukla D, Paul FW (2002) Orthogonal functions for system identification and control. In: Leondes CT (ed) Neural network systems techniques and applications. Control and dynamic systems, vol 7. Academic Press, San Diego, pp 1-73
6. Rafajowicz E, Pawlak M (1997) On function recovery by neural networks based on orthogonal expansions. In: Proc. 2nd world congress of nonlinear analysis. Nonlinear analysis, theory and applications, vol 30, no 3. Pergamon Press, Oxford, pp 1343-1354 (Pubitemid 127423749)
7. Halawa K (2008) Fast and robust way of learning the Fourier series neural networks on the basis of multidimensional discrete Fourier transform. In: Lecture notes in computer science. Lecture notes in artificial intelligence, vol 5097. Springer, New York, pp 62-70
8. Barron AR (1993) Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans Inform Theory 39(3):930-945
9. Ebert T, Bänfer O, Nelles O (2010) Multilayer perceptron network with modified sigmoid activation functions. In: Artificial intelligence and computational intelligence. Lecture notes in computer science, vol 6319. Springer, New York, pp 414-421
10. Chen C, Chang W (1996) A feedforward neural network with function shape autotuning. Neural Netw 9(4):627-641 (Pubitemid 26232898)
11. Guarnieri S, Piazza F (1999) Multilayer feedforward networks with adaptive spline activation function. IEEE Trans Neural Netw 10(3):672-683
12. Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(5):359-366 (Pubitemid 20609008)
13. Björck A (1996) Numerical methods for least squares problems. SIAM: Society for Industrial andApplied Mathematics, Philadelphia
14. Frank A, Asuncion A (2011) UCI machine learning repository. University of California, School of Information and Computer Science, Irvine. http://archive.ics.uci.edu/ml
15. Cortez P, Cerdeira A, Almeida F, Matos T, Reis J (2009) Modeling wine preferences by data mining from physicochemical properties. Decis Support syst 47(4):547-553
16. Redmond MA, Baveja A (2002) Adata-driven software tool for enabling cooperative information sharing among police departments. Eur J Oper Res 141:660-678
17. Computer Files: Department of Commerce, Bureau of the Census, Census of Population and Housing 1990. United States: Summary Tape File 1a and 3a, U.S. Department of Justice, Federal Bureau of Investigation (1995) Crime in the United States, U.S. Department of Justice, Bureau of Justice Statistics, Law EnforcementManagement and Administrative Statistics. U.S. Department Of Commerce, Bureau Of The Census Producer, Washington and Inter-university Consortium for Political and Social Research Ann Arbor