Cooperative-competitive genetic evolution of radial basis function centers and widths for time series prediction

IEEE Transactions on Neural Networks

Volumn 7, Issue 4, 1996, Pages 869-880

  Whitehead, Bruce A ^a,b   Choate, Timothy D ^c  

^a IEEE

^b UNIVERSITY OF TENNESSEE SPACE INSTITUTE   (United States)

^c Edge Internet Services   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATION THEORY; FORECASTING; FUNCTIONS; INVERSE PROBLEMS; LEAST SQUARES APPROXIMATIONS; MATHEMATICAL TECHNIQUES; NEURAL NETWORKS; OPTIMIZATION; TIME SERIES ANALYSIS;

COMPUTATION TIMES; GENETIC FITNESS FUNCTION; INVERSE MULTIQUADRIC; MACKEY GLASS TIME; NICHE CREATION; PREDICTION ERROR; RADIAL BASIS FUNCTIONS CENTERS AND WIDTHS; TIME SERIES PREDICTION;

GENETIC ALGORITHMS;

EID: 0030197198     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.508930     Document Type: Article

References (76)

1. D. S. Broomhead and D. Lowe, "Multivariable functional interpolation and adaptive networks," Complex Syst., vol. 2, pp. 321-355, 1988.
2. M. J. D. Powell, "Radial basis functions for multivariable interpolation: A review," in IMA Conf. Algorithms for the Approximation of Functions and Data, J. C. Mason and M. G. Cox, Eds. Oxford, U.K.: Oxford Univ. Press, 1987, pp. 143-167.
3. _, "The theory of radial basis function approximation in 1990," in Advances in Numerical Analysis, vol. 3, W. Light, Ed. Oxford: Clarendon, 1992, pp. 105-210.
4. T. Poggio and F. Girosi, "Networks for approximation and learning," Proc. IEEE, vol. 78, no. 9, pp. 1481-1497, 1990.
5. F. Girosi, "Some extensions of radial basis functions and their applications in artificial intelligence," Computers Math. Applic., vol. 24, no. 12, pp. 61-80, 1992.
6. M. T. Musavi, W. Ahmed, K. H. Chan, K. B. Faris, and D. M. Hummels, "On the training of radial basis function classifiers," Neural Networks, vol. 5, pp. 595-603, 1992.
7. S. Chen, C. F. N. Cowan, and P. M. Grant, "Orthogonal least squares learning algorithm for radial basis function networks," IEEE Trans. Neural Networks, vol. 2, no. 2, pp. 302-309, 1991.
8. S. Renals and R. Rohwer, "Phoneme classification experiments using radial basis functions," in IJCNN: Int. Joint Conf. Neural Networks, IEEE, Piscataway, NJ, June 1989, pp. I-461-I-467.
9. K. J. Cios, R. E. Tjia, N. Liu, and R. A. Langenderfer, "Study of continuous ID3 and radial basis function algorithms for the recognition of glass defects," in IJCNN-91-Seattle: Int. Joint Conf. Neural Networks, IEEE, Piscataway, NJ, July 1991, pp. I-49-I-54.
10. T. A. Foley, "The map and blend scattered data interpolant on a sphere," Computers Math. Applic., vol. 24, no. 12, pp. 49-60, 1992.
11. C. Bishop, "Improving the generalization properties of radial basis function neural networks," Neural Computation, vol. 3, no. 4, pp. 579-588, 1991.
12. J. Moody and C. J. Darken, "Fast learning in networks of locally-tuned processing units," Neural Computa., vol. 1, pp. 281-294, 1989.
13. T. Kohonen, "An introduction to neural computing," Neural Networks, vol. 1, no. 1, pp. 3-16, 1988.
14. B. Widrow and M. E. Hoff, "Adaptive switching circuits," in WESCON Convention, Institute of Radio Engineers, New York, 1960, pp. 96-104.
15. T. Kohonen, "The self-organizing map," Proc. IEEE, vol. 78, no. 9, pp. 1464-1480, 1990.
16. G. F. Miller, P. M, Todd, and S. U. Hegde, "Designing neural networks using genetic algorithms," in Proc. 3rd Int. Conf. Genetic Algorithms, J. D. Schaffer, Ed. San Francisco: Morgan Kaufmann, 1989, pp. 379-384.
17. S. A. Harp, T. Samad, and A. Guha, "Designing application-specific neural networks using the genetic algorithm," in Advances in Neural Information Processing Systems 2, D. S. Touretzky, Ed. San Francisco: Morgan Kaufmann, 1990, pp. 447-454.
18. H. Kitano, "Designing neural networks using genetic algorithms with graph generation system," Complex Syst., vol. 4, pp. 461-476, 1990.
19. J. D. Schaffer, R. A. Caruana, and L. J. Eshelman, "Using genetic search to exploit the emergent behavior of neural networks," in Emergent Computation: Self-Organizing, Collective, and Cooperative Phenomena in Natural and Artificial Computing Networks, S. Forrest, Ed. Cambridge, MA: MIT Press, 1990, pp. 244-248.
20. S. W. Wilson, "Perceptron redux: Emergence of structure," in Emergent Computation: Self-Organizing, Collective, and Cooperative Phenomena in Natural and Artificial Computing Networks, S. Forrest, Ed. Cambridge, MA: MIT Press, 1990, pp. 249-256.
21. S. A. Harp and T. Samad, "Genetic optimization of self-organizing feature maps," in IJCNN-91-Seattle: Int. Joint Conf. Neural Networks, IEEE, Piscataway, NJ, July 1991, pp. I-341-I-346.
22. J. W. L. Merrill and R. F. Port, "Fractally configured neural networks," Neural Networks, vol. 4, pp. 53-60, 1991.
23. K. U. Hoffgen, H. P. Siemon, and A. Ultsch, "Genetic improvements of feedforward nets for approximating functions," in Lecture Notes in Computer Science 496: Parallel Problem Solving from Nature, G. Goos and J. Hartmanis, Eds. Berlin: Springer-Verlag, 1991, pp. 302-306.
24. W. Schiffmann, M. Joost, and R. Werner, "Performance evaluation of evolutionary created neural network topologies," in Lecture Notes in Computer Science 496: Parallel Problem Solving from Nature. Berlin: Springer-Verlag, 1991, pp. 274-283.
25. S. Bornholdt and D. Graudenz, "General asymmetric neural networks and structure design by genetic algorithms," Neural Networks, vol. 5. no. 1, pp. 327-334, 1992.
26. F. Z. Brill, D. E. Brown, and W. N. Martin, "Fast genetic selection of features for neural network classifiers," IEEE Trans. Neural Networks, vol. 3, no. 2, pp. 324-328, 1992.
27. J. G. Elias, "Genetic generation of connection patterns for a dynamic artificial neural network," in Combinations of Genetic Algorithms and Neural Networks, L. D. Whitley and J. D. Schaffer, Eds. Los Alamitos, CA: IEEE Computer Soc. Press, June 1992, pp. 38-54.
28. P. J. Hancock, "Genetic algorithms and permutation problems: A comparison of recombination operators for neural net structure specification," in Combinations of Genetic Algorithms and Neural Networks, L. D. Whitley and J. D. Schaffer, Eds. Los Alamitos, CA: IEEE Computer Soc. Press, June 1992, pp. 108-122.
29. V. Maniezzo, "Genetic evolution of the topology and weight distribution of neural networks," IEEE Trans. Neural Networks, vol. 5, no. 1, pp. 39-53, 1994.
30. X. Yao, "A review of evolutionary artificial neural networks," Int. J. Intelligent Syst., vol. 8, no. 4, pp. 539-567, 1993.
31. J. D. Schaffer, D. Whitley, and L. J. Eshelman, "Combinations of genetic algorithms and neural networks: A survey of the state of the art," in Combinations of Genetic Algorithms and Neural Networks, L. D. Whitley and J. D. Schaffer, Eds. Los Alamitos, CA: IEEE Computer Soc. Press, June 1992, pp. 1-37.
32. T. P. Caudell and C. P. Dolan, "Parametric connectivity: Training of constrained networks using genetic algorithms," in Proceedings of the Third International Conference on Genetic Algorithms, J. D. Schaffer, Ed. San Francisco: Morgan Kaufmann, June 1989, pp. 370-374.
33. D. J. Montana and L. Davis, "Training feedforward neural networks using genetic algorithms," in Proc. 11th Int. Joint Conf. Artificial Intell., San Francisco: Morgan Kaufmann, Aug. 1989, pp. 762-767.
34. D. Whitley and T. Hanson, "Optimizing neural networks using a faster, more accurate genetic search," in Proc. 3rd Int. Conf. Genetic Algorithms, J. D. Schaffer, Ed. San Francisco: Morgan Kaufmann, June 1989, pp. 391-396.
35. D. B. Fogel, L. J. Fogel, and V. W. Porto, "Evolving neural networks," Biological Cybernetics, vol. 63, pp. 487-493, 1990.
36. H. de Garis, "Genetic programming: Building artificial nervous systems using genetically programmed neural network modules," in Machine Learning: Proc. 7th Int. Conf. B. Porter and R. Mooney, Eds. San Francisco: Morgan Kaufmann, June 1990, pp. 132-139.
37. H. Kitano, "Empirical studies on the speed of convergence of neural network training using genetic algorithms,"' in Proc. 8th Nat. Conf. AI (AAAI-90), Cambridge, MA, July-Aug. 1990. Cambridge: MIT Press, pp. 789-795.
38. J. R. Koza and J. P. Rice, "Genetic generation of both the weights and architecture for a neural network," in IJCNN-91-Seattle: Int. Joint Conf. Neural Networks, IEEE, Piscataway, NJ, July 1991, pp. II-397-II-404.
39. S. Dominic, R. Das, D. Whitley, and C. Anderson, "Genetic reinforcement learning for neural networks," in IJNN-91-Seattle: Proc. Int. Joint Conf. Neural Networks, IEEE, Piscataway, NJ, July 1991, pp. II-71-II-76.
40. H. Braun and J. Weisbrod, "Evolving neural networks for application oriented problems," in Prof. 2nd Ann. Conf. Evolutionary Programming, D. B. Fogel and W. Atmar, Eds. La Jolla, CA: Evolutionary Programming Soc., 1993, pp. 62-71.
41. D. B. Fogel, "Using evolutionary programming to create neural networks that are capable of playing tic-lac-toe," in Proc. Int. Conf. Neural Networks, IEEE, Piscataway, NJ, 1993, pp. 875-880.
42. J. R. McDonnell and D. Waagen, "Neural network structure design by evolutionary programming," in Proc. 2nd Ann. Conf. Evolutionary Programming D. B. Fogel and W. Atmar, Eds. La Jolla, CA: Evolutionary Programming Soc.,1993, pp. 79-89.
43. S. Oliker and M. Furst, "A distributed genetic algorithm for neural network design and training," Complex Syst., vol. 6, no. 5, pp. 459-477, 1992.
44. D. Dasgupta and D. R. McGregor, "Designing application-specific neural networks using the structured genetic algorithm," in Combinations of Genetic Algorithms and Neural Networks, L. D. Whitley and J. D. Schaffer, Eds. Los Alamitos, CA: IEEE Computer Soc. Press, June 1992. pp. 87-107.
45. J. McDonnell and D. Waagen, "Evolving recurrent perceptrons for time-series modeling," IEEE Trans. Neural Networks, vol. 5, no. 1, pp. 24-38, 1994.
46. P. J. Angeline, G. M. Saunders, and J. B. Pollack, "An evolutionary algorithm that constructs recurrent neural networks," IEEE Trans. Neural Networks, vol. 5, no. 1, pp. 54-65, 1994.
47. B. A. Whitehead and T. D. Choate, "Evolving space-filling curves to distribute radial basis functions over an input space," IEEE Trans. Neural Networks, vol. 5, no. 1, pp. 15-23, 1994.
48. J. H. Holland, Adaptation in Natural and Artificial Systems. Ann Arbor, MI: Univ. Michigan Press, 1975.
49. S. E. Fahlman and C. Lebiere, "The cascade-correlation learning architecture," in Advances in Neural Information Processing Systems 2, R. P. Lippmann, J. E. Moody, and D. S. Touretzky, Eds. San Francisco: Morgan Kaufmann, 1991, pp. 524-532.
50. J.-N. Hwang, S.-R. Lay, M. Maechler, R. D. Martin, and J. Schimert, "Regression modeling in backpropagation and projection pursuit learning," IEEE Trans. Neural Networks, vol. 5, no. 3, pp. 342-353, 1994.
51. N. Karunanithi, R. Das, and D. Whitley, "Genetic cascade learning for neural networks," in Combinations of Genetic Algorithms and Neural Networks, L. D. Whitley and J. D. Schaffer, Eds. Los Alamitos, CA: IEEE Computer Soc. Press, June 1992, pp. 134-145.
52. M. A. Potter, "A genetic cascade-correlation learning algorithm," in Combinations of Genetic Algorithms and Neural Networks, L. D. Whitley and J. D. Schaffer, Eds. Los Alamitos, CA: IEEE Computer Soc. Press, June 1992, pp. 123-133.
53. J. McDonnell and D. Waagen, "Evolving cascade-correlation networks for time-series forcasting," IEEE Trans. Syst., Man, Cybern., 1995, to be published.
54. S. V. Odri, D. P. Petrovacki, and G. A. Krstonosic, "Evolutional development of a multilevel neural network," Neural Networks, vol. 6. no. 4, pp. 583-595, 1993.
55. G. F. Hinton, T. J. Sejnowski, and D. H. Ackley, "Boltzmann machines: Constraint satisfaction networks that learn," Carnegie-Mellon Univ., Pittsburgh, PA, Tech. Rep. 1-37, 1984.
56. R. Smalz and M. Conrad, "Combining evolution with credit apportionment: A new learning algorithm for neural nets," Neural Networks, vol. 7. no. 2, pp. 341-351, 1994.
57. K. A. De Jong, "An analysis of the behavior of a class of genetic adaptive algorithms." Ph.D. dissertation, Univ. Michigan, Ann Arbor, 1975.
58. K. Deb and D. E. Goldberg, "An investigation of niche and species formation in genetic function optimization," in Proceedings of the Third International Conference on Genetic Algorithms. San Francisco: Morgan Kaufmann, June 1989, pp. 42-50.
59. J. Horn, D. E. Goldberg, and K. Deb, "Implicit niching in a learning classifier system: Nature's way," Evolutionary Computation, vol. 2, no. 1, pp. 37-66, 1994.
60. R. E. Smith, S. Forrest, and A. S. Perelson, "Searching for diverse, cooperative populations with genetic algorithms," Evolutionary Computa., vol. 1, no. 1 pp. 127-149, 1993.
61. D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning. Reading, MA: Addison-Wesley, 1989.
62. D. E. Goldberg, K. Deb, and J. Horn, "Massive multimodality, deception, and genetic algorithms," in Parallel Problem Solving from Nature, 2, R. Manner and B. Manderick, Eds. New York: Elsevier Sci., 1992, pp. 37-50.
63. M. E. Palmer and S. J. Smith, "Improved evolutionary optimization of difficult landscapes: Control of premature convergence through scheduled sharing," Complex Syst., vol. 5, no. 5, pp. 443-458, 1991.
64. R. E. Smith and H. B. Cribbs, III, "Is a learning classifier a type of neutal network?" Evolutionary Computation, vol. 2, no. 1, pp. 19-36, 1994.
65. D. P. Greene and S. F. Smith, "Using coverage as a model building constraint in learning classifier systems," Evolutionary Computation, vol. 2, no. 1, pp. 67-91, 1994.
66. J. E. Baker, "Reducing bias and inefficiency in the selection algorithm," in Proc. 2nd Int. Conf. Genetic Algorithms. San Francisco: Morgan Kaufmann, 1987, pp. 14-21.
67. M. C. Mackey and L. Glass, "Oscillation and chaos in physiological control systems," Sci., vol. 197, pp. 287-289, 1977.
68. A. Lapedes and R. Farber, "How neural nets work," in Neural Information Processing Systems. New York: Amer. Inst. Physics, 1988, pp. 442-456.
69. J. D. Farmer and J. J. Sidorowich, "Predicting chaotic time series," Phys. Rev. Lett., vol. 59, no. 8, pp. 845-848, 1987.
70. M. F. Tenorio, "Self-organizing network for optimum supervised learning," IEEE Trans. Neural Networks, vol. 1, no. 1, pp. 100-110, 1990.
71. J. Platt, "A resource-allocating network for function interpolation," Neural Computa., vol. 3, pp. 213-225, 1991.
72. E. Hartman and J. D. Keeler, "Predicting the future: Advantages of semilocal units," Neural Computa., vol. 3, no. 4, pp. 566-578, 1991.
73. A. M. Logar, E. M. Corwin, and W. J. Oldham, "A comparison of recurrent neural network learning algorithms," in IEEE Int. Conf. Neural Networks, IEEE, Piscataway, NJ, 1993, pp. 1129-1134.
74. R. A. Fisher, "The use of multiple measurements in taxonomic problems," Annu. Eugenics, vol. 7, pt. II, pp. 179-188, 1936.
75. T. D. Sanger, "A tree-structured adaptive network for function approximation in high-dimensional spaces," IEEE Trans. Neural Networks, vol. 2. no. 2, pp. 285-293, 1991.
76. W. C. Mead, R. D. Jones, Y. C. Lee, C. W. Barnes, G. W. Flake, L. A. Lee, and M. K. O'Rourke, "Using CNLS-net to predict the Mackey-Glass chaotic time series," in IJCNN: Int. Joint Conf. Neural Networks, IEEE, Piscataway, NJ, July 1991, pp. II-485-II-490.