Orthogonal least squares algorithm for training cascade neural networks

IEEE Transactions on Circuits and Systems I: Regular Papers

Volumn 59, Issue 11, 2012, Pages 2629-2637

  Huang, Gao ^a   Song, Shiji ^a   Wu, Cheng ^a  

^a TSINGHUA UNIVERSITY   (China)

Author keywords

Cascade correlation; constructive neural networks; Newton's method; orthogonal least squares

Indexed keywords

LEAST SQUARES APPROXIMATIONS; NEWTON-RAPHSON METHOD; ORTHOGONAL FUNCTIONS;

CASCADE CORRELATION; CASCADE NEURAL NETWORKS; CONSTRUCTIVE NEURAL NETWORK; CONSTRUCTIVE TRAINING; GENERALIZATION PERFORMANCE; NEWTON'S METHODS; ORTHOGONAL LEAST SQUARES; ORTHOGONAL LEAST SQUARES ALGORITHM;

NEURAL NETWORKS;

EID: 84869499003     PISSN: 15498328     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSI.2012.2189060     Document Type: Article

References (35)

1. Y. N. Zhang, W.M.Ma, and B. H. Cai, "From Zhang neural network to Newton iteration for matrix inversion," IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 56, no. 7, pp. 1405-1415, Jul. 2009.
2. K. Li, J. X. Peng, and E. W. Bai, "Two-stage mixed discrete-continuous identification of radial basis function (RBF) neural models for nonlinear systems," IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 56, no. 3, pp. 630-643, Mar. 2009.
3. M. D. Marco, M. Forti, M. Grazzini, and L. Pancioni, "Limit set dichotomy and convergence of cooperative piecewise linear neural networks," IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 58, no. 5, pp. 1052-1062, May 2011.
4. C. Feng, R. Plamondon, and C. O'Reilly, "Analyzing oscillations for an-node recurrent neural networks model with time delays and general activation functions," IEEETrans.Circuits Syst. I, Reg. Papers, no. 99, Aug. 2011.
5. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning representations by back-propagation errors," Nature, vol. 323, pp. 533-536, 1986.
6. B. M. Wilamowski, "Neural network architectures and learning algorithms," IEEE Ind. Electron. Mag., vol. 3, no. 4, pp. 56-63, Dec. 2009.
7. B. M. Wilamowski, "Challenges in applications of computational intelligence in industrial electronics," in Proc. IEEE Int. Symp. Ind. Electron., Jul. 2010, pp. 15-22.
8. B. Wilamowski, D. Hunter, and A. Malinowski, "Solving parityproblems with feedforward neural networks," in Proc. Int. Joint Conf. Neural Netw., Jul. 2003, vol. 4, pp. 2546-2551.
9. J. Park and I. W. Sandberg, "Universal approximation using radialbasis-function networks," Neural Comput., vol. 3, pp. 246-257, 1991.
10. T. Kohonen, "The self-organizing map," Proc. IEEE, vol. 78, no. 9, pp. 1464-1480, Sep. 1990.
11. N. K. Treadgold and T. D. Gedeon, "A cascade network algorithm employing progressive rprop," in Int. Work Conf. Artif. Natural Neural Netw., 1997, pp. 733-742. (Pubitemid 127080538)
12. B. W. Wah and M. Qian, "Constrained formulations for neural networks training and their applications to solve the two-spiral problem," in Proc. 5th Int. Conf. Comput. Sci. Informatics, 2000, vol. 1, pp. 598-601.
13. B. M. Wilamowski and Y. Hao, "Improved computation for Levenberg-Marquardt training," IEEE Trans. Neural Netw., vol. 21, no. 6, pp. 930-937, Jun. 2010.
14. B. M. Wilamowski and H. Yu, "Neural network learning without backpropagation," IEEE Trans. Neural Netw., vol. 21, no. 11, pp. 1793-1803, Nov. 2010.
15. K. J. Lang and M. J. Witbrock, "Learning to tell two spirals apart," in Proc. 1988 Connectionist Models Summer School, D. S. Touretzky, G. E. Hinton, and T. J. Sejnowski, Eds. San Mateo, CA: Morgan Kaufmann, 1988, pp. 52-59.
16. T. Y. Kwok and D. Y. Yeung, "Constructive algorithms for structure learning in feedforward neural networks for regression problems," IEEE Trans. Neural Netw., vol. 8, no. 3, pp. 630-645, May 1997.
17. R. Reed, "Pruning algorithms-a survey," IEEE Trans. Neural Netw., vol. 4, no. 5, pp. 740-747, Sep. 1993.
18. S. E. Fahlman and C. Lebiere, "The cascade-correlation learning architecture," in Advances in Neural Information Processing Systems 2, D. S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1991, pp. 524-532.
19. S. Chen, S.A. Billings, and W. Luo, "Orthogonal least squaresmethods and their applications to non-linear system identification," Int. J. Control, vol. 50, pp. 1873-1896, 1989. (Pubitemid 20636409)
20. S. Chen, C. Cowan, and P. M. Grant, "Orthogonal least squares learning algorithm for radial basis function networks," IEEE Trans. Neural Netw., vol. 2, no. 2, pp. 302-309, Mar. 1991.
21. K. Li, J. X. Peng, and G. W. Irwin, "A fast nonlinear model identification method," IEEE Trans. Autom. Control, vol. 50, no. 8, pp. 1211-1216, Aug. 2005. (Pubitemid 41410121)
22. K. Li and J. X. Peng, "Neural input selection-A fast model-based approach," Neurocomput., vol. 70, no. 4-6, pp. 762-769, Oct. 2007. (Pubitemid 46043972)
23. J. X. Peng, K. Li, and D. S. Huang, "A hybrid forward algorithm for rbf neural network construction," IEEE Trans. Neural Netw., vol. 17, no. 6, pp. 1439-1451, Nov. 2006. (Pubitemid 44824258)
24. S. Chen, X. Hong, C. Harris, and X.Wang, "Identification of nonlinear systems using generalized kernel models," IEEE Trans. Control Syst. Technol., vol. 13, no. 3, pp. 401-411, May 2005. (Pubitemid 40679595)
25. S. Chen, X. Hong, B. L. Luk, and C. J. Harris, "Construction of tunable radial basis function networks using orthogonal forward selection," IEEE Trans. Syst., Man, Cybern. B, vol. 39, no. 2, pp. 457-466, Apr. 2009.
26. S. E. Fahlman, "Faster learning variations on backpropagation: An empirical study," in Proc. 1988 Connectionist Models Summer School, D. S. Touretzky, G. E. Hinton, and T. J. Sejnowski, Eds. SanMateo, CA: Morgan Kaufmann, 1988, pp. 38-51.
27. J. L. Yuan and T. L. Fine, "Forecasting demand for electric power," in Advances in Neural Information Processing Systems 5, S. J. Hanson, J. Cowan, and C. L. Giles, Eds. San Mateo, CA:Morgan Kaufmann, 1993, pp. 739-746.
28. J. N. Hwang, S. R. Lay, M. Maechler, R. D. Martin, and J. Schimert, "Regression modeling in back-propagation and projection pursuit learning," IEEE Trans. Neural Netw., vol. 5, no. 3, pp. 342-353, May 1994.
29. O. Fujita, "Optimization of the hidden unit function in feedforward neural networks," Neural Netw., vol. 5, no. 5, pp. 755-764, 1992. (Pubitemid 23564796)
30. P. Courrieu, "A convergent generator of neural networks," Neural Netw., vol. 6, no. 6, pp. 835-844, 1993.
31. T. Y. Kwok and D. Y. Yeung, "Objective functions for training new hidden units in constructive neural networks," IEEE Trans. Neural Netw., vol. 8, no. 5, pp. 1131-1148, Sep. 1997.
32. G. Golub and C. F. V. Loan, Matrix Computations. Baltimore, MD: Johns Hopkins Univ. Press, 1993.
33. X. Z. Gao, X.Wang, and S. J. Ovaska, "Fusion of clonal selection algorithm and differential evolution method in training cascade-correlation neural network," Neurocomputing, vol. 72, pp. 2483-2490, Jun. 2009.
34. A. Frank and A. Asuncion, UCI Machine Learning Repository 2011 [Online]. Available: http://archive.ics.uci.edu/ml
35. D. Michie, D. J. Spiegelhalter, and C. Taylor, Machine Learning, Neural and Statistical Classification. New York: Ellis Horwood, 1994.