Radial basis function network configuration using mutual information and the orthogonal least squares algorithm

Neural Networks

Volumn 9, Issue 9, 1996, Pages 1619-1637

  Zheng, Guang L ^a   Billings, Steve A ^b  

^a UNIVERSITY OF SHEFFIELD   (United Kingdom)

^b NONE

Author keywords

hidden node selection; input node selection; mutual information; network structure; pattern recognition; radial basis function; system identification

Indexed keywords

ALGORITHMS; DATA STRUCTURES; ERROR COMPENSATION; HEURISTIC METHODS; KNOWLEDGE REPRESENTATION; LEARNING SYSTEMS; MATHEMATICAL MODELS; PATTERN RECOGNITION;

RADIAL BASIS FUNCTIONS (RBF);

NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; PATTERN RECOGNITION; PRIORITY JOURNAL;

EID: 0030462726     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/0893-6080(95)00139-5     Document Type: Article

References (31)

1. Ashrafi S., Conway D., Rokni M., Sperling R., Roszman L., Cooley J. Solar flux forecasting using mutual information with an optimal delay. Advances in the Astronautical Sciences. 84(1-2):1993;901-913.
2. Battiti R. Using mutual information for selecting features in supervised neural net learning. IEEE Transactions on Neural Networks. 5(4):1994;537-550.
3. Billings S.A., Chen S. Extended model set, global data and threshold model identification of severely non-linear systems. International Journal of Control. 50:1989;1897.
4. Billings S.A., Korenberg M.J., Chen S. Identification of nonlinear output-affine systems using an orthogonal least squares algorithm. International Journal of Control. 19:1988;1559-1568.
5. Broomhead D.S., Lowe D. Multivariable functional interpolation and adaptive networks. Complex Systems. 2:1988;321-355.
6. Chen S., Billings S.A., Cowan C.F.N., Grant P.M. Non-linear systems identification using radial basis functions. International Journal of Systems Science. 21(12):1990;2513-2539.
7. Chen S., Billings S.A., Cowan C.F.N., Grant P.W. Practical identification of narmax models using radial basis functions. International Journal of Control. 52(6):1990;1327-1350.
8. Chen S., Billings S.A., Grant P.W. Recursive hybrid algorithm for non-linear system identification using radial basis function network. International Journal of Control. 55(5):1992;1051-1070.
9. Chen S., Billings S.A., Luo W. Orthogonal least squares methods and their application to nonlinear system identification. International Journal of Control. 50:1989;1873-1896.
10. Conese C., Maselli R. Selection of optimal bands from tm scenes through mutual information analysis. ISPRS Journal of Photogrammetry and Remote Sensing. 48(3):1993;2-11.
11. Deco G., Finnoff W., Zimmerman H.G. Unsupervised mutual information criterion for elimination of overtraining in supervised multilayer networks. Neural Computation. 7:1995;86-107.
12. Fraser A.M., Swinney H.L. Independent coordinates for strange attractors from mutual information. Physical Review A. 33(2):1986;1134-1140.
13. Haber R., Unbehauen H. Structure identification of nonlinear dynamic systems - a survey on input/output approaches. Automatica. 26(4):1990;651-677.
14. Holcomb, T., & Morari, M. (1991). Local training for radial basis function networks: Towards solving the hidden unit problem. Proceedings of American Control Conference (pp. 2331-2336). Evanston, IL: American Automatic Control Council.
15. Jones, D.S. (1979). Elementary information theory. Oxford: Clarendon Press.
16. Kaviori S.N., Venkata Subramanian V. Using fuzzy clustering with ellipsoidal units in neural networks for robust fault classification. Computers in Chemical Engineering. 17(8):1993;765-784.
17. Lee S., Rhee M.K. A Gaussian potential function network with hierarchically self-organizing learning. Neural Networks. 4:1991;207-224.
18. Li W. Mutual information functions versus correlation functions. Journal of Statistical Physics. 60(5-6):1990;823-837.
19. Light W.A. Some aspects of radial basis function approximation. Approximation Theory, Spline Functions and Applications. 356:1992;163-190.
20. Linsker R. How to generate ordered maps by maximizing the mutual information between input and output signals. Neural Computation. 1:1989;402-411.
21. Martinrie J.M., Albano A.M., Mees A.I., Rapp P.E. Mutual information, strong attractors, and the optimal estimation of dimension. Physical Review A. 45(10):1992;7058-7064.
22. Moody J., Darken C. Fast learning in networks of locally-tuned processing units. Neural Computation. 1:1989;281-294.
23. Musavi M.T., Ahmed W., Chan K.H., Faris K.B., Hummels D.M. On the training of radial basis function classifiers. Neural Networks. 5:1992;595-603.
24. Noonan J.P., Marcus J.R. Minimum mutual information in image restoration. Kybernetes. 19(6):1990;34-41.
25. Poggio T., Girosi F. Network for approximation and learning. Proceedings of IEEE. 78(9):1990;1481-1497.
26. Powell, M.J.D. (1992). The theory of radial basis function approximation in 1990. In W.A. Light, (Ed.) Advances in Numerical analysis, Vol. 2, (pp. 105-210). Oxford: Oxford University Press.
27. Silverman, B.W. (1986). Density estimation for statistics and data analysis. London: Chapman and Hall.
28. Treves A., Panzeri S. The upward bias in measures of information derived from limited data samples. Neural Computation. 7:1995;399-407.
29. Vogt M. Combination of radial basis function neural networks with optimized vector quantization. Proceedings of the IEEE International Conference on Neural Networks. 3:1993;1841-1846.
30. Xu L., Krzyzak A., Oja E. Rival penalized competitive learning for clustering analysis, rbf net, and curve detection. IEEE Transactions on Neural Networks. 4(4):1993;636-649.
31. Zheng, G.L., & Billings, S.A. (1994). Radial basis function network training using a fuzzy clustering scheme (Research Report No. 505). Department of Automatic Control and Systems Engineering, University of Sheffield.