GOLS-Genetic orthogonal least squares algorithm for training RBF networks

Neurocomputing

Volumn 69, Issue 16-18, 2006, Pages 2041-2064

  Barreto, A M S ^a   Barbosa, H J C ^b   Ebecken, N F F ^a  

^a FEDERAL UNIVERSITY OF RIO DE JANEIRO   (Brazil)

^b LABORATÓRIO NACIONAL DE COMPUTAÇÃO CIENTÍFICA   (Brazil)

Author keywords

Genetic algorithms; Orthogonal least squares algorithm; Radial basis function networks

Indexed keywords

COMPUTATIONAL METHODS; COMPUTER SIMULATION; GENETIC ALGORITHMS; NUMERICAL METHODS;

GENETIC ORTHOGONAL LEAST SQUARES APPROACH; HYBRID ALGORITHM; ORTHOGONAL LEAST SQUARES ALGORITHM;

RADIAL BASIS FUNCTION NETWORKS;

ACCURACY; ANALYTICAL ERROR; ARTICLE; ARTIFICIAL NEURAL NETWORK; BIOINFORMATICS; BREAST CANCER; CANCER CLASSIFICATION; CHROMOSOME ANALYSIS; DNA SEQUENCE; GENE INSERTION; GENE MUTATION; GENETIC ALGORITHM; GENETIC RECOMBINATION; GENETIC SELECTION; HEART DISEASE; HUMAN; INFORMATION PROCESSING; NORMAL DISTRIBUTION; PERFORMANCE; PREDICTION; PRIORITY JOURNAL; PROCESS OPTIMIZATION; QUALITY CONTROL; REGRESSION ANALYSIS; SENSITIVITY ANALYSIS;

EID: 33748625457     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2005.10.004     Document Type: Article

References (70)

1. Barreto A., Barbosa H., and Ebecken N. Growing compact RBF networks using a genetic algorithm. Proceedings of the VII Brazilian Symposium on Neural Networks. Recife, PE (2002) 61-66
2. Benoudjit N., Archambeau C., Lendasse A., and Lee M.V.J.A. Width optimization of the gaussian kernels in radial basis function networks. Proceedings of the European Symposium on Artificial Neural Networks (2002), Bruges, Belgium 425-432
3. Billings S.A., and Zheng G.L. Radial basis function network configuration using genetic algorithms. Neural Networks 8 6 (1995) 877-890
4. C. Blake, C. Merz, UCI repository of machine learning databases, URL: 〈http://www.ics.uci.edu/∼mlearn/MLRepository.html〉, 1998.
5. E. Blanzieri, Learning algorithms for radial basis function networks: synthesis, experiments and cognitive modelling, Ph.D. Thesis, Center of Cognitive Science-University and Polytechnic of Turin, 1998.
6. Branke J. Evolutionary approaches to dynamic optimization problems. In: Branke J., and Bäck T. (Eds). Evolutionary Algorithms for Dynamic Optimization Problems (2001), San Francisco, CA, USA 27-30
7. Broomhead D.S., and Lowe D. Multivariable functional interpolation and adaptive networks. Complex Systems 2 (1988) 321-355
8. Burdsall B., and Giraud-Carrier C. GA-RBF. a self-optimising RBF network. Proceedings of the Third International Conference on Artificial Neural Networks and Genetic Algorithms. Springer (1997) 348-351
9. Carse B., Pipe A.G., Fogarty T.C., and Hill T. Evolving radial basis function neural networks using a genetic algorithm. Proceedings of the IEEE International Conference on Evolutionary Computation vol. 1 (1995), IEEE, Silverspring, MD 300-305
10. S. Chen, Multi-output regression using a locally regularised orthogonal least square algorithm, Technical Report, Department of Electronics and Computer Science-University of Southhampton, Highfield, Southhampton SO17 1BJ, UK, 2002.
11. Chen S., Billings S.A., and Luo W. Orthogonal least squares methods and their application to non-linear system identification. Int. J. Control 50 5 (1989) 1873-1896
12. Chen S., Cowan C.F.N., and Grant P.M. Orthogonal least squares learning algorithm for radial basis function networks. IEEE Trans. Neural Networks 2 2 (1991) 302-309
13. S. Chen, C.F.N. Cowan, P.M. Grant, Orthogonal least squares algorithm for training multi-output radial basis function networks, in: IEEE Proceedings, vol. Part F 139, 1992, pp. 378-384.
14. Chen S., Wu Y., and Luk B.L. Combined genetic algorithm optimization and regularized orthogonal least squares learning for radial basis function networks. IEEE-NN 10 5 (1999) 1239
15. Chen S., Wun Y., and Alkadhimi K. A two-layer learning method for radial basis function networks using combined genetic and regularised OLS algorithms. Proceedings of Genetic Algorithms in Engineering Systems. Innovations and Applications vol. 414 (1995), IEEE, Silverspring, MD 245-249
16. H.G. Cobb, An investigation into the use of hypermutation as an adaptive operator in genetic algorithms having continuous, time-dependent nonstationary environments, Technical Report AIC-90-001, Navy Center for Applied Research in Artificial Intelligence, 1991.
17. Davis L. Handbook of Genetic Algorithms (1991), Van Nostrand Reinhold, New York
18. de Lacerda E.G.M., de Carvalho A.C.P.L.F., and Ludermir T.B. Evolutionary optimization of RBF networks. VIth Brazilian Symposium on Neural Networks, Sociedade Brasileira de Computação (SBC) vol. I (2000), IEEE, Silverspring, MD 219-224
19. Duda R.O., and Hart P.E. Pattern Classification and Scene Analysis (1973), Wiley, New York
20. Eshelman L., Mathias K., and Schaffer J. Crossover operator biases. exploring the population distribution. In: Baeck T. (Ed). Proceedings of the Seventh International Conference on Genetic Algorithms and their Applications. East Lansing, MI (1997) 354-361
21. Eshelman L., and Schaffer J. Real coded genetic algorithms and interval schemata. In: Whitley D. (Ed). Foundations of Genetic Algorithms 2 (1993), Morgan Kaufmann, San Mateo, CA 187-202
22. Fritzke B. Growing cell structures-a self-organizing network in k dimensions. In: Aleksander I., and Taylor J. (Eds). Artificial Neural Networks, 2 vol. II (1992), North-Holland, Amsterdam, Netherlands 1051-1056
23. Girosi F. Some extensions of radial basis functions and their applications in artificial intelligence. Compute. Math. Appl. 24 12 (1992) 61-80
24. F. Girosi, T. Poggio, Networks and the best approximation property, Technical Report AIM-1164, Massachusetts Institute of Technology Artificial Intelligence Laboratory and Center for Biological Information Processing Whitaker College, 1989.
25. Goldberg D. Genetic Algorithms in Search, Optimization, and Machine Learning (1989), Addison-Wesley, Reading, MA
26. D. Goldberg, Real-coded genetic algorithms, virtual alphabets, and blocking, Technical Report IlliGAL Report 90001, Illinois Genetic Algorithms Laboratory, Department of General Engineering, University of Illinois, Urbana, IL, USA, 1990.
27. G.H. Golub, C.F.V. Loan, Matrix Computations, second ed., Johns Hopkins Series in Mathematical Sciences, The Johns Hopkins University Press, Baltimore, MD, 1993.
28. Grefenstette J.J. Genetic algorithms for changing environments. In: Männer R., and Manderick B. (Eds). Parallel Problem Solving from Nature 2 (Proceedings of the Second International Conference on Parallel Problem Solving from Nature, Brussels, 1992) (1992), Elsevier, Amsterdam 137-144
29. Haykin S. Neural Networks: a Comprehensive Foundation (1999), Prentice-Hall, NJ, USA
30. J. Hefferon, Linear Algebra, Mathematics, Saint Michael's College, Colchester, VT, USA (05439), 2001.
31. Herrera F., Lozano M., and Verdegay J.L. Tackling real-coded genetic algorithms: operators and tools for behavioural analysis. Artif. Intell. Rev. 12 4 (1998) 265-319
32. Hinton G.E. Connectionist learning procedures. Artif. Intell. 40 (1989) 185-234
33. Holland J.H. Adaptation in Natural and Artificial Systems (1975), University of Michigan Press, Ann Arbor, MI
34. Hong X., and Billings S.A. Givens rotation based fast backward elimination algorithm for RBF neural network pruning. IEE Proc. D, Control Theory and Applications 5 144 (1997) 381-384
35. Hwang Y., and Bang S. An efficient method to construct a radial basis function neural network classifier. Neural Networks 10 8 (1997) 1495-1503
36. V. Kadirkamanathan, M. Niranjan, A function estimation approach to sequential learning with neural networks, Technical Report, CUED/F-INFENG/TR.111, Cambridge University Engineering Department, Cambridge, England, September 1992.
37. A. Lapedes, R. Farber, How neural nets work, Neural Information Processing Systems, American Institute of Physics, New York, 1998, pp. 442-456.
38. G.P. Liu, V. Kadirkamanathan, Multiobjective criteria for neural network structure selection and identification of nonlinear systems using genetic algorithms, in: IEE Proceedings Part P: Control Theory and Applications, vol. 146(5), 1999, pp. 373-382.
39. Mackey M.C., and Glass L. Oscillation and chaos in physiological control systems. Science 197 (1977) 287-289
40. Maillard E.P., and Gueriot D. RBF neural network, basis functions and genetic algorithm. International Conference on Neural Networks vol. 4 (1997), IEEE 2187-2192
41. M.W. Mak, K.W. Cho, Genetic evolution of radial basis function centers for pattern classification, in: Proceedings of the 1998 IEEE International Joint Conference on Neural Networks, vol. 1, 1998, pp. 669-673.
42. O.L. Mangasarian, W.N. Street, W.H. Wolberg, Breast cancer diagnosis and prognosis via linear programming, Technical Report MP-TR-1994-10, University of Wisconsin, Madison, December 1994.
43. O.L. Mangasarian, W. H. Wolberg, Cancer diagnosis via linear programming, Technical Report CS-TR-1990-958, University of Wisconsin, Madison, August 1990.
44. Michalewicz Z. Genetic Algorithms + Data Structures = Evolution Programs (1992), Springer, New York
45. Moody J., and Darken C. Fast learning in networks of locally-tuned processing units. Neural Comput. 1 2 (1989) 281-294
46. Orr M. Regularisation in the selection of radial basis function centres. Neural Comput. 7 3 (1995) 606-623
47. M.J.L. Orr, Introduction to radial basis function networks, Technical Report, Centre for Cognitive Science, University of Edinburgh, April 1996.
48. Orr M.J.L. Optimising the widths of radial basis functions. Vth Brazilian Symposium on Neural Networks (1998), IEEE, Belo Horizonte, Brazil 26-29
49. Platt J.C. A resource-allocating network for function interpolation. Neural Comput. 3 2 (1991) 213-225
50. Poggio T., and Girosi F. Network for approximation and learning. Proceedings of the IEEE 78 9 (1990) 1481-1497
51. Powell M.J.D. Radial basis functions for multivariable interpolation: a review. In: Mason J.C., and Cox M.G. (Eds). Algorithms for Approximation (1987), Clarendon Press, Oxford 143-167
52. M.J.D. Powell, Radial basis function methods for interpolation to functions of many variables, NA2001/11, DAMTP, University of Cambridge, Numerical Analysis Group, 2001.
53. L. Prechelt, Proben1: A set of neural network benchmark problems and benchmarking rules, Technical Report 21/94, Fakultat fur Informatik, Universitat Karlsruhe, 76128 Karlsruhe, Germany, uRL: 〈citeseer.nj.nec.com/prechelt94proben.html〉, September 1994.
54. Prechelt L. A quantitative study of experimental evaluations of neural network learning algorithms: current research practice. Neural Networks 9 3 (1996) 457-462
55. Press W.H., Teukolsky S.A., Vetterling W.T., and Flannery B.P. Numerical Recipes in C. second ed. (1992), Cambridge University Press, Cambridge
56. Rawlings J.O. Applied Regression Analysis (1988), Wadsworth & Brooks, Cole, Pacific Grove, CA, USA
57. Riedmiller M., and Braun H. A direct adaptive method for faster backpropagation learning. the RPROP algorithm. Proceedings of the IEEE International Conference on Neural Networks. San Francisco, CA (1993) 586-591
58. Rumelhart D.E., Hinton G.E., and Williams R.J. Learning representations by backpropagating errors. Nature 323 (1986) 533-536
59. W. Sarle, Stopped training and other remedies for overfitting, in: Proceedings of the 27th Symposium on Interface, 1995, pp. 352-360.
60. Schwarz G. Estimating the dimension of a model. Ann. Stat. 6 2 (1978) 461-464
61. Sherstinsky A., and Picard R.W. On the efficiency of the orthogonal least squares training method for radial basis function networks. IEEE Trans. Neural Networks 7 1 (1996) 195-200
62. A.F. Sheta, K.D. Jong, Time-series forecasting using GA-tuned radial basis functions, Information Sciences (Special issue).
63. Wettschereck D., and Dietterich T. Improving the performance of radial basis function networks by learning center locations. In: Moody J.E., Hanson S.J., and Lippmann R.P. (Eds). Advances in Neural Information Processing Systems vol. 4 (1992), Morgan Kaufmann, Loas Altos, CA 1133-1140
64. Whitehead B.A. Genetic evolution of radial basis function coverage using orthogonal niches. IEEE Trans. Neural Networks 7 6 (1996) 1525-1528
65. Whitehead B.A., and Choate T.D. Evolving space-filling curves to distribute radial basis functions over an input space. IEEE Trans. Neural Networks 5 1 (1993) 15-23
66. Whitehead B.A., and Choate T.D. Cooperative-competitive genetic evolution of radial basis function centers and widths for time series prediction. IEEE Trans. Neural Networks 7 4 (1995) 869-880
67. Whitley D. The GENITOR algorithm and selective pressure. In: Schaffer J. (Ed). Proceedings of the Third International Conference on Genetic Algorithms and their Applications (1989), Morgan Kaufmann, San Mateo, CA 116-121
68. Wright A.H. Genetic algorithms for real parameter optimization. In: Rawlins G.J. (Ed). Foundations of Genetic Algorithms (1991), Morgan Kaufmann, San Mateo, CA 205-218
69. Yingwei L., Sundararajan N., and Saratchandran P. A sequential learning scheme for function approximation using minimal radial basis function neural networks. Neural Comput. 9 (1997) 461-478
70. Q.F. Zhao, O. Hammami, K. Kuroda, K. Saito, Cooperative co-evolutionary algorithm-how to evaluate a module?, in: IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks, 2000, pp. 150-157.