Combinatorial evolution of regression nodes in feedforward neural networks

Neural Networks

Volumn 12, Issue 1, 1999, Pages 175-189

  Schmitz, Gregor P J ^a   Aldrich, Chris ^a  

^a STELLENBOSCH UNIVERSITY   (South Africa)

Author keywords

Combinatorial optimisation; Combinatorial search; Evolutionary algorithms; Genetic algorithms; Multilayer perceptrons; Neural network design; Orientated ellipsoidal radial basis functions

Indexed keywords

BACKPROPAGATION; COMPUTER ARCHITECTURE; GENETIC ALGORITHMS; OPTIMIZATION; REGRESSION ANALYSIS;

EVOLUTIONARY ALGORITHMS; RADIAL BASIS FUNCTION NETWORKS;

FEEDFORWARD NEURAL NETWORKS;

ACCURACY; ALGORITHM; ARTIFICIAL NEURAL NETWORK; COMPUTER AIDED DESIGN; COMPUTER MODEL; INFORMATION PROCESSING; INTERMETHOD COMPARISON; PRIORITY JOURNAL; REVIEW; VALIDATION PROCESS;

EID: 0032981443     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(98)00104-X     Document Type: Article

References (61)

1. Angeline, P.J., Sauders, G.M., & Pollack, J.B. (1994). An evolutionary algorithm that constructs recurrent neural networks. IEEE Transaction on Neural Networks, 5, 54-65.
2. Beiu, V. (1996). Optimal implementation of neural networks: VLSIfriendly learning algorithms. In J.G. Taylor (Ed.), Neural networks and their applications (Chapter 18, pp. 255-276). Chichester: Wiley.
3. Billings, S.A., & Zheng, G.L. (1995). Radial basis function network configuration using genetic algorithms. Neural Networks, 8 (6), 877-890.
4. Bornhold, S., & Graudenz, D. (1992). General asymmetric neural networks and structure design by genetic algorithms. Neural Networks, 5 (1), 327-334.
5. Broomhead, D.S., & Lowe, D. (1988). Multivariable functional interpolation and adaptive networks. Complex Systems, 2, 321-355.
6. Chen, S., Billings, S.A., Cowan, C.F.N., & Grant, P.M. (1990). Practical identification of narmax models using radial basis functions. International Journal of Control, 52 (6), 1327-1350.
7. Chen, S., Cowan, C.F.N., & Grant, P.M. (1991). Orthogonal least squares learning algorithm for radial basis function networks. IEEE Transaction on Neural Networks, 2, 302-303.
8. Crowder, R.S. (1990). Predicting the Mackey-Glass time series with cascade correlation learning. In Proceedings 1990 Connectionist Models Summer School (pp. 117-123). San Mateo, CA: Morgan Kaufmann.
9. Deb, K., & Goldberg, D.E. (1989). An investigation of niche and species formation in genetic function optimisation. In Proceedings of the Third International Conference on Genetic Algorithms (pp. 42-50). San Fransisco, CA: Morgan Kaufmann.
10. De Jong, K.A. (1975). An analysis of the behaviour of a class of genetic adaptive algorithms. Ph.D. dissertation, University of Michigan, Ann Arbor, Diss. Abstr. Int. 36(10), 5140B, Univ. Microfilm #76-9381.
11. Duda, R.O., & Hart, E. (1973). Pattern classification and scene analysis. New York: Wiley.
12. Fahlman, S.E., & Lebiere, C. (1990). The cascade-correlation learning architecture. In D.S. Touretzky (Ed.), Advances in neural information processing systems 2 (pp. 524-532). San Mateo, CA: Morgan Kaufmann.
13. Fiesler, E. (1994). Comparative bibliography of ontogenetic neural networks. In M. Marinaro & P.O. Morasso (Eds.), Artificial neural networks 4 (ICANN'94, Sorrento, Italy) (pp. 793-795). Berlin: Springer-Verlag.
14. Fogel, D.B. (1993). Using evolutionary programming to create neural networks that are capable of playing tic-tac-toe. In Proceedings 1993 International Joint Conference on Neural Networks (IJCNN'93) (pp. 875-880). New York: IEEE Press.
15. Fogel, D.B., Fogel, L.J., & Porto, V.W. (1990). Evolving neural networks. Biological Cybernetics, 64, 487-493.
16. Fritzke, B. (1994). Growing cell structures: a self-organising network for unsupervised and supervised learning. Neural Networks, 7 (9), 14411460.
17. Furnival, G.M., & Wilson, W.W. Jr. (1974). Regression by leaps and bounds. Technometrics, 16 (4), 499-511.
18. Hancock, P.J. (1992). Genetic algorithms and permutation problems: A comparison of recombination operators for neural net structure specifications. In L.D. Whitley & J.D. Schaffer (Eds.), Combinations of genetic algorithms and neural networks (pp. 108-122). Los Alamitos, CA: IEEE Computer Society Press.
19. Haykin, S. (1994). Neural networks: a comprehensive foundation. New York: Macmillan College Publishing Company.
20. Hertz, J., Krogh, A., & Palmer, R.G. (1991). Introduction to the theory of neural computation. Redwood City, CA: Addison-Wesley.
21. Holland, J.H. (1975). Adaptation in natural and artificial systems. Ann Arbor, MI: University Michigan Press.
22. Hornik, K., Stinchcombe, M., & White, H. (1989). Multilayer feedforward networks are universal approximators. Neural Networks, 2, 359-366.
23. Hrycej, T. (1992). Modular learning in neural networks: a modularised approach to neural network classification. New York: Wiley.
24. Jacobs, R.A. (1988). Increased rates of convergence through learning rate adaptation. Neural Networks, I, 295-307.
25. Kitano, H. (1990). Designing neural networks using genetic algorithms with graph generation systems. In Complex Systems (Vol. 4).
26. Koza, J.R. (1992). Genetic programming: on the programming of computers by means of natural selection. Cambridge, MA: The MIT Press.
27. Lange, J.M., Voigt, H.-M., & Wolf, D. (1994). Task decomposition and correlations in growing artificial neural networks. In M. Mariano & P.G. Morasso (Eds.), Proceedings of the International Conference on Artificial Neural Networks (ICANN 94) (pp. 735-738). London: Springer Verlag.
28. Littmann, E., & Ritter, H. (1992). Cascade network architectures. In Proceedings International Joint Conference on Neural Network (Baltimore, Section II, pp. 398-404). New York: IEEE Press.
29. Littmann, E., & Ritter, H. (1993). Generalisation abilities of cascade network architectures. In J.S. Hanson, J.D. Cowan & C.L. Giles (Eds.), Advances in neural information processing systems (pp. 188195). San Mateo, CA: Morgan Kaufmann.
30. Mackey, M.C., & Glass, L. (1977). Oscillations and chaos in physiological control systems. Science, 197, 287-289.
31. Maniezzo, V. (1994). Genetic evolution of the topology and weight distribution of neural networks. IEEE Transactions on Neural Network, 5 (I), 39-53.
32. Martinez, D. (1992). Dealing with continuous inputs for the offset algorithm. In Proceedings of the International Conference on Signal Processing Appl. and Tech. SPAT'92 (Boston, pp. 1167-1175).
33. McDonnel, J.R., & Waagen, D. (1993). Neural network structure designed by evolutionary programming. In D.B. Fogel & W. Atmar (Eds.), Proceedings of the Second Annual Conference on Evolutionary Programming (pp. 79-89). La Jolla, CA: Evolutionary Programming Society.
34. Mézard, M., & Nadal, J.-P. (1989). Learning feedforward layered networks: the tiling algorithm. Journal of Physics A, 22, 2191-2204.
35. Moody, J. (1989). Fast learning in multi-resolution hierarchies. In D.S. Touretzky (Ed.), Advances in neural information processing systems I (pp. 29-39). San Mateo, CA: Morgan Kaufmann.
36. Moody, J., & Darken, C.J. (1989). Fast learning in networks of locallytuned processing units. Neural Computation, I (4), 281-294.
37. Nash, W.J., Sellers, T.L., Talbot, S.R., Cawthorn, A.J., & Ford, W.B. (1994). The population biology of abalone (Haliotis species) in Tasmania. I. Blacklip abalone (H. rubra) from the North Coast and Islands of Bass Strait, Sea Fisheries Division, Technical Report No. 48 (ISSN 1034-3288).
38. Odri, S.V., Petrovacki, O.P., & Krstonosic, O.A. (1993). Evolutional development of a multilevel neural network. Neural Networks, 6 (4), 583-595.
39. Pietruschka, U., & Kinder, M. (1995). Ellipsoidal basis functions for higher-dimensional approximation problems. In F. Fogelman-Soulie & P. Gallinari (Eds.), ICANN '95. International Conference on
40. Artificial Neural Networks. Neuronimes '95 Scientific Conference (Vol. 2, pp. 81-85). Paris, France: EC2 and Cie.
41. Platt, J. (1991). A resource allocating network for function interpolation. Neural Computation, 3, 213-225.
42. Powel, M.J.D. (1987). Radial basis functions for multivariable interpolation: A review. In J.C. Mason & M.G. Cox (Eds.), IMA Conference on Algorithms for the Approximation of Functions and Data (Royal Military College of Science, pp. 143-167). Oxford: University Press.
43. Rissanen, J. (1978). Modelling by shortest data description. Automatica, 14, 465-471.
44. Rumelhart, D.E., & McClelland, J.L. (Eds.) (1986). Parallel distribution processing: exploration in the microstructure of cognition (Vol. 1). Cambridge, MA: MIT Press.
45. Saffrey, J., & Thornton, C. (1991). Using stereographic projection as a preprocessing technique for upstar. In Proceedings 1991 International Joint Conference on Neural Networks (IJCNN'91) (Seattle, Vol. 2, pp. 441-446). New York: IEEE Press.
46. Saha, A., Christian, J., Tang, D.S., & Wu, C.-L. (1991). Orientated nonradial basis functions for image coding and analysis. In R.P. Lippmann, J.E. Moody & D.S. Touretzky (Eds.), Advances in neural information processing systems 3 (pp. 728-734). San Mateo, CA: Morgan Kaufmann.
47. Schaffer, J.D., Whitley, D., & Eshelman, L.J. (1992). Combinations of genetic algorithms and neural networks: A survey of the state of the art. In D. Whitley & J.D. Schaffer (Eds.), Combinations of genetic algorithms and neural networks (pp. 1 -37). Los Alamitos, CA: IEEE Computer Society Press.
48. Schiffmann, W., Joost, M., & Werner, R. (1991). Performance evaluation of evolutionary created neural network topologies. In G. Goos & J. Hartmanis (Eds.), Lecture Notes in Computer Science 496: Parallel Problem Solving From Nature (pp. 274-283). Berlin: Springer Verlag.
49. Seber, G.A.F. (1977). Linear regression analysis. New York: Wiley (Wiley series in probability and mathematical statistics).
50. Sherstinsky, A., & Picard, R.W. (1996). On the efficiency of the orthogonal least squares training method for radial basis function networks. IEEE Transactions on Neural Networks, 7 (1), 195-201.
51. Simpson, P.K. (1992). Fuzzy min-max neural networks:-part 1: classification. IEEE Transactions on Neural Networks, 3 (5), 776-786.
52. Smalz, R., & Conrad, M. (1994). Combining evolution with credit apportionment: A new learning algorithm for neural nets. Neural Networks, 7 (2), 341-351.
53. Storn, R., & Price, K. (1995). Differential evolution-a simple and efficient adaptive scheme for global optimisation over continuous spaces. Technical Report TR-95-012, ICSI, Berkeley, USA.
54. Watkins, D.S. (1991). Fundamentals of matrix computations. New York: Wiley.
55. Werbos, P.J. (1974). Beyond regression: New Tools for prediction and analysis in the behavioural science. Ph.D. Thesis Harvard University, Cambridge, MA.
56. Whitehead, B.A., & Choate, T.D. (1994). Evolving space filling curves to distribute radial basis function networks over an input space. IEEE Transactions on Neural Networks, 5 (1), 15-23.
57. Whitehead, B.A., & Choate, T.D. (1996). Cooperative-competitive genetic evolution of radial basis function centers and widths for time series prediction. IEEE Transactions on Neural Networks, 7 (4), 869-880.
58. Whitley, D., Starkweather, T., & Bogart, C. (1990). Genetic algorithms and neural networks: optimising connections and connectivity. Parallel Computing, 14, 347-361.
59. Yao, X. (1993). A review of evolutionary artificial neural networks. International Journal of Intelligent Systems, 8 (4), 539-567.
60. Yao, X., & Liu, Y. (1997). A new evolutionary system for evolving artificial neural networks. IEEE Transactions on Neural Networks, 8 (3), 694-713.
61. Yingwei, Lu, Sundararajan, N., & Saratchandran, P. (1997). A sequential learning scheme for function approximation using minimal radial basis function neural networks. Neural Computation, 9 (2), 461-478.