Learning polynomial feedforward neural networks by genetic programming and backpropagation

IEEE Transactions on Neural Networks

Volumn 14, Issue 2, 2003, Pages 337-350

  Nikolaev, Nikolay Y ^a   Iba, Hitoshi ^b  

^a UNIVERSITY OF LONDON   (United Kingdom)

^b UNIVERSITY OF TOKYO   (Japan)

Author keywords

Backpropagation; Polynomial feedforward neural networks (PFNNs); Time series prediction; Volterra models

Indexed keywords

BACKPROPAGATION; FUNCTIONS; GENETIC ALGORITHMS; LEARNING ALGORITHMS; LEARNING SYSTEMS; MATHEMATICAL MODELS; POLYNOMIAL APPROXIMATION; TIME SERIES ANALYSIS;

GENETIC PROGRAMMING; POLYNOMIAL ACTIVATION FUNCTIONS; POLYNOMIAL FEEDFORWARD NEURAL NETWORKS; POLYNOMIAL NETWORK ALGORITHMS; VOLTERRA MODELS;

FEEDFORWARD NEURAL NETWORKS;

EID: 0037361327     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2003.809405     Document Type: Article

References (44)

1. A. M. Abdelbar, "Achieving superior generalization with a high order neural network," Neural Comput. Applicat., vol. 7, pp. 141-146, 1998.
2. H. Akaike, "Power spectrum estimation through autoregression model fitting," Ann. Inst. Statist. Math., vol. 21, pp. 407-419, 1969.
3. G. K. Alataris, T. W. Berger, and V. Z. Marmarelis, "A novel network for nonlinear modeling of neural systems with arbitrary point-process inputs," Neural Networks, vol. 13, no. 2, pp. 1645-1657, 2000.
4. M. Anthony and S. B. Holden, "Quantifying generalization in linearly weighted neural networks," Complex Syst., vol. 18, pp. 91-114, 1994.
5. A. R. Barron and R. L. Barron, "Statistical learning networks: A unifying view," in Proc. 20th Symp. Interface: Comput. Sci. Statist., M. D. Fraser, Ed., 1988, pp. 192-203.
6. C. Bishop, Neural Networks for Pattern Recognition. Oxford, U.K.: Oxford Univ. Press, 1995.
7. M. S. Chen and M. T. Manry, "Conventional modeling of the multi-layer perceptron using polynomial basis functions," IEEE Trans. Neural Networks, vol. 4, pp. 164-166, Jan. 1993.
8. C. Chang and J. Y. Cheung, "Backpropagation algorithm for higher order neural network," in Proc. Int. Joint Conf. Neural Networks, Baltimore, MD, 1992, pp. 164-166.
9. N. E. Cotter, "The Stone-Weierstrass theorem and its application to neural networks," IEEE Trans. Neural Networks, vol. 1, pp. 290-295, July 1990.
10. R. Durbin and R. Rumelhart, "Product units: A computationally powerful and biologically plausible extension to backpropagation networks," Neural Comput., vol. 1, pp. 133-142, 1989.
11. M. Ebner, "On the search space of genetic programming and its relation to nature's search space," in Proc. Congr. Evolutionary Computation, Washington, DC, 1999, pp. 1357-1361.
12. J. F. Elder, IV and D. E. Brown, "Induction and polynomial networks," in Network Models for Control and Processing, M. D. Fraser, Ed. Exeter, U.K.: Intellect Books, 2000, pp. 143-198.
13. D. Gabor, W. Wildes, and R. Woodcock, "A universal nonlinear filter, predictor and simulator which optimizes itself by a learning process," Proc. Inst. Elect. Eng., vol. 108B, pp. 422-438, 1961.
14. M. Gallagher and T. Downs, "Weight space learning trajectory vizualization," in Proc. 8th Australian Conf. Neural Networks, M. Dale, Ed., 1997, pp. 55-59.
15. D. G. Green, R. E. Reichelt, and R. H. Bradbury, "Statistical behavior of the GMDH algorithm," Biometrics, vol. 44, pp. 49-69, 1988.
16. J. Hertz, A. Krogh, and R. G. Palmer, Introduction to the Theory of Neural Computation. Reading, MA: Addison-Wesley, 1991.
17. M. Heywood and P. Noakes, "A framework for improved training of Sigma-Pi networks," IEEE Trans. Neural Networks, vol. 6, pp. 893-903, July 1996.
18. U. Hübner, C.-O. Weiss, N. B. Abraham, and D. Tang, "Lorenz-like chaos in NH3-FIR lasers," in Time Series Prediction: Forecasting the Future and Understanding the Past, A. S. Weigend and N. A. Gershenfeld, Eds. Reading, MA: Addison-Wesley, 1994, pp. 73-104.
19. H. Iba, H. de Garis, and T. Sato, "Temporal data processing using genetic programming," in Proc. 6th Int. Conf. on Genetic Algorithms, ICGA-95, L. Eshelman, Ed. San Mateo, CA: Morgan Kaufmann, 1995, pp. 279-286.
20. ____, "Numerical approach to genetic programming for system identification," Evol. Comput., vol. 3, no. 4, pp. 417-452, 1996.
21. A. G. Ivakhnenko, "Polynomial theory of complex systems," IEEE Trans. Syst, Man, Cybern., vol. SMC-1, pp. 364-378, Aug. 1971.
22. I. T. Jolliffe, Principal Component Analysis. New York: Springer-Verlag, 1986.
23. T. C. Jones and S. Forrest, "Fitness distance correlation as a measure of problem difficulty for genetic algorithms," in Proc. 6th Int. Conf. Genetic Algorithms, L. J. Eshelman, Ed. San Mateo, CA: Morgan Kaufmann, 1995, pp. 184-192.
24. H. Kargupta and R. E. Smith, "System identification with evolving polynomial networks," in Proc. 4th Int. Conf. Genetic Algorithms, R. K. Belew and L. B. Booker, Eds. San Mateo, CA: Morgan Kaufmann, 1991, pp. 370-376.
25. A. N. Kolmogorov and S. V. Fomin, Elements of the Theory of Functions and Functional Analysis. New York: Dover, 1999.
26. T. Kondo, A. S. Pandya, and J. M. Zurada, "GMDH-type neural networks with a feedback loop and their application to nonlinear system identification," in Proc. Smart Eng. Syst. Design Conf., St. Louis, MO, 1999.
27. J. R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection. Cambridge, MA: MIT Press, 1992.
28. A. Lapedes and R. Farber, "Nonlinear signal processing using neural networks: predictions and system modeling," Los Alamos Nat. Lab., Los Alamos, NM, Tech. Rep. LA-UR-87-2662, 1987.
29. G. P. Liu, V. Kadirkamanathan, and S. A. Billings, "On-line identification of nonlinear systems using volterra polynomial basis function neural networks," Neural Networks, vol. 11, no. 9, pp. 1645-1657, 1998.
30. H. R. Madala and A. G. Ivakhnenko, Inductive Learning Algorithms for Complex Systems Modeling, Boca Raton, FL: CRC, 1994.
31. V. Z. Marmarelis and X. Zhao, "Volterra models and three-layer perceptrons," IEEE Trans. Neural Networks, vol. 8, pp. 1421-1433, Nov. 1997.
32. N. Nikolaev and H. Iba, "Regularization approach to inductive genetic programming," IEEE Trans. Evol. Comput., vol. 5, pp. 359-375, Dec. 2001.
33. Y. H. Pao, Adaptive Pattern Recognition and Neural Networks. Reading, MA: Addison-Wesley, 1989.
34. T. Pham and X. Liu, "Modeling and prediction using GMDH networks of adalines with nonlinear preprocessors," Int. J. Syst. Sci., vol. 25, no. 11, pp. 1743-1759, 1994.
35. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning internal representations by error propagation," in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge, MA: MIT Press, 1986, vol. 1, pp. 318-362.
36. T. D. Sanger, "A tree-structured adaptive network for function approximation in high-dimensional spaces," IEEE Trans. Neural Networks, vol. 2, pp. 285-293, Mar. 1991.
37. M. Schetzen, The Volterra and Wiener Theories of Nonlinear Systems. New York: Wiley, 1980.
38. M. F. Tenorio and W.-T. Lee, "Self-organizing network for optimum supervised learning," IEEE Trans. Neural Networks, vol. 1, pp. 100-110, Mar. 1990.
39. Z. Wang, C. Di Massimo, M. T. Tham, and A. J. Morris, "A procedure for determining the topology of multilayer feedforward neural networks," Neural Networks, vol. 7, no. 2, pp. 291-300, 1994.
40. A. S. Weigend, B. A. Huberman, and D. E. Rumelhart, "Predicting sunspots and exchange rates with connectionist networks," in Nonlinear Modeling and Forecasting, S. Eubank and M. Casdagli, Eds. Redwood City, CA: Addison-Wesley, 1992, pp. 395-432.
41. J. Wesley Hines, "A logarithmic neural network architecture for unbounded nonlinear function approximation," in Proc. Int. Conf. Neural Networks, vol. 2, 1996, pp. 1245-1250.
42. J. Wray and G. G. R. Green, "Calculation of the volterra kernels of non-linear dynamic systems using an artificial neural networks," Biol. Cybern., vol. 71, no. 3, pp. 187-195, 1994.
43. ____, "Neural networks, approximation theory, and finite precision computation," Neural Networks, vol. 8, no. 1, pp. 31-37, 1995.
44. B.-T. Zhang, P. Ohm, and H. Mühlenbein, "Evolutionary induction of sparse neural trees," Evol. Comput., vol. 5, no. 2, pp. 213-236, 1997.