A growing and pruning sequential learning algorithm of hyper basis function neural network for function approximation

Neural Networks

Volumn 46, Issue , 2013, Pages 210-226

  Vuković, Najdan ^a   Miljković, Zoran ^a  

^a UNIVERSITY OF BELGRADE   (Serbia)

Author keywords

Extended Kalman filter; Feedforward neural networks; Hyper basis function; Neuron's significance; Sequential learning

Indexed keywords

ACTIVATION FUNCTIONS; ENGINEERING PRACTICES; FUNCTION APPROXIMATION; GENERALIZATION PROPERTIES; HYPER BASIS FUNCTIONS; RADIAL BASIS FUNCTION NEURAL NETWORKS; SEQUENTIAL LEARNING; SEQUENTIAL LEARNING ALGORITHM;

COMPLEX NETWORKS; EXTENDED KALMAN FILTERS; FEEDFORWARD NEURAL NETWORKS; LEARNING ALGORITHMS; NETWORK PERFORMANCE; NEURAL NETWORKS; NEURONS; OPTIMIZATION;

RADIAL BASIS FUNCTION NETWORKS;

ARTICLE; COMPARATIVE STUDY; ENGINEERING; EXTENDED KALMAN FILTER; HYPER BASIS FUNCTION; HYPER BASIS FUNCTION WITH GROWING AND PRUNING; LEARNING ALGORITHM; MEASUREMENT ACCURACY; NERVE CELL; NERVE CELL NETWORK; PREDICTION; PRIORITY JOURNAL; PROCESS OPTIMIZATION; RADIAL BASED FUNCTION;

EXTENDED KALMAN FILTER; FEEDFORWARD NEURAL NETWORKS; HYPER BASIS FUNCTION; NEURON'S SIGNIFICANCE; SEQUENTIAL LEARNING;

ALGORITHMS; COMPUTER SIMULATION; GENERALIZATION (PSYCHOLOGY); MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER); NEURONS;

EID: 84879753900     PISSN: 08936080     EISSN: 18792782     Source Type: Journal    
DOI: 10.1016/j.neunet.2013.06.004     Document Type: Article

References (36)

1. Alexandridis A., Sarimveis H., Bafas G. A new algorithm for online structure and parameter adaptation of RBF networks. Neural Networks 2003, 16:1003-1017.
2. Asuncion, A., & Newman, D. (2010). UCI machine learning repository. Irvine, CA: University of California, School of Information and Computer Science. [] (last date of access: September 10, 2012). http://archive.ics.uci.edu/ml/.
3. Billings S.A., Wei H.L., Balikhin M.A. Generalized multiscale radial basis function networks. Neural Networks 2007, 20(10):1081-1094.
4. Bortman M., Aladjem M. A growing and pruning method for radial basis function networks. IEEE Transactions on Neural Networks 2009, 20(6):1039-1045.
5. Crassidis J.L., Junkins J.L. Optimal estimation of dynamic systems (2nd edition) 2011, CRC Press, Taylor and Francis, New York, (Chapter 3).
6. Figueiredo M.A.T., Jain A.K. Unsupervised learning of finite mixture models. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002, 24(3):381-396.
7. Gonzalez J., Rojas I., Ortega J., Pomares H., Fernandez F.J., Diaz A.F. Multi-objective evolutionary optimization of the size, shape, and position parameters of radial basis function networks for function approximation. IEEE Transactions on Neural Networks 2003, 14(6):1478-1495.
8. Guvenir, H. A., & Uysal, I. (2000). Bilkent university function approximation repository. [ ] (last date of access: September 10, 2012). http://funapp.cs.bilkent.edu.tr.
9. Han H.G., Chen Q.L., Qiao J.F. An efficient self-organizing RBF neural network for water quality prediction. Neural Networks 2011, 24(7):717-725.
10. Han H.G., Qiao J.F., Chen Q.L. Model predictive control of dissolved oxygen concentration based on a self-organizing RBF neural network. Control Engineering Practice 2012, 20(4):465-476.
11. Huang G.B., Saratchandran P., Sundararajan N. An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks. IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics 2004, 34(6):2284-2292.
12. Huang G.B., Saratchandran P., Sundararajan N. A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation. IEEE Transactions on Neural Networks 2005, 16(1):57-67.
13. Huang G.B., Zhou H., Ding X., Zhang R. Extreme learning machine for regression and multiclass classification. IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics 2012, 42(2):513-529.
14. Huang G.B., Zhu Q.Y., Siew C.K. Extreme learning machine: theory and applications. Neurocomputing 2006, 70:489-501.
15. Islam M.M., Sattar M.A., Amin M.F., Yao X., Murase K. A new adaptive merging and growing algorithm for designing artificial neural networks. IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics 2009, 39(3):705-722.
16. Jazwinski A. Stochastic processes and filtering theory 1970, Academic Press, New York, (Chapter 7).
17. Kadirkamanathan V., Niranjan M. A function estimation approach to sequential learning with neural networks. Neural Computation 1993, 5:954-975.
18. Karayiannis N.B. Reformulated radial basis neural networks trained by gradient descent. IEEE Transactions on Neural Networks 1999, 10(3):657-671.
19. Lazaro M., Santamari I., Pantaleo C. A new EM-based training algorithm for RBF networks. Neural Networks 2003, 16:69-77.
20. Lu Y., Sundararajan N., Saratchandran P. A sequental learning scheme for function approximation using minimal radial basis function (RBF) neural networks. Neural Computation 1997, 9:461-478.
21. Lu Y., Sundararajan N., Saratchandran P. Performance evaluation of a sequential minimal radial basis function (RBF) neural network learning algorithm. IEEE Transactions on Neural Networks 1998, 9(2):308-318.
22. Mahdi R.N., Rouchka E.C. Reduced hyperBF networks: regularization by explicit complexity reduction and scaled rprop based training. IEEE Transactions on Neural Networks 2011, 22(5):673-686.
23. Miljković, Z., & Aleksendrić, D. (2009). Artificial neural networks-solved examples with theoretical background (in Serbian). University of Belgrade-Faculty of Mechanical Engineering, Belgrade.
24. Nishida K., Yamauchi K., Omori T. An online learning algorithm with dimension selection using minimal hyper basis function networks. Systems and Computers in Japan 2006, 37(11):11-21.
25. Orr M., Hallam J., Takezawa K., Murray A., Ninomiya S., Oide M., et al. Combining regression trees and radial basis function networks. International Journal of Neural Systems 2000, 10(6):453-465.
26. Platt J. A resource allocating network for function interpolation. Neural Computation 1991, 3:213-225.
27. Poggio T., Girosi F. A theory of networks for approximation and learning. A. I. Memo 1140 1989, MIT.
28. Poggio T., Girosi F. Networks for approximation and learning. Proceedings of the IEEE 1990, 1990:1481-1497.
29. Qiao J.F., Han H.G. Identification and modeling of nonlinear dynamical systems using a novel self-organizing RBF-based approach. Automatica 2012, 48(8):1729-1734.
30. Salmeron M., Ortega J., Puntonet C.G., Prieto A. Improved RAN sequential prediction using orthogonal techniques. Neurocomputing 2001, 41:153-172.
31. Schwenker F., Kestler H.A., Palm G. Three learning phases for radial-basis-function networks. Neural Networks 2001, 14(4-5):439-458.
32. Simon D. Training radial basis function neural networks with the extended Kalman filter. Neurocomputing 2001, 48:455-475.
33. Todorović, B., Stanković, M. S., & Moraga, C. (2002). Extended Kalman filter trained recurrent radial basis function network in nonlinear system identification. In ICANN(2002) (pp. 819-824).
34. Vuković, N. (2012). Machine learning of intelligent mobile robot based on artificial neural networks. Ph.D. dissertation (in Serbian). University of Belgrade-Faculty of Mechanical Engineering.
35. Wu Y., Wang H., Zhang B., Du K.-L. Using radial basis function networks for function approximation and classification. ISRN Applied Mathematics 2012, Article ID 324194. 10.5402/2012/324194.
36. Zhang R., Huang G.B., Sundararajan N., Saratchandran P. Improved GAP-RBF network for classification problems. Neurocomputing 2007, 70(16-18):3011-3018.