Systemical convergence rate analysis of convex incremental feedforward neural networks

Neurocomputing

Volumn 72, Issue 10-12, 2009, Pages 2627-2635

  Chen, Lei ^a   Huang, Guang Bin ^a   Pung, Hung Keng ^a  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

Convergence rate; Feedforward neural networks; Random hidden neurons; Universal approximation

Indexed keywords

ALTERNATIVE ALGORITHMS; CONVERGENCE RATE; DIFFERENTIAL FUNCTIONS; FASTER CONVERGENCES; GENERALIZATION PERFORMANCE; INCREMENTAL EXTREME LEARNING MACHINES; LOCAL MINIMUMS; PARAMETER SELECTIONS; RANDOM HIDDEN NEURONS; REGRESSION PROBLEMS; RESIDUAL ERRORS; UNIVERSAL APPROXIMATION;

EDUCATION; GRADIENT METHODS; LEARNING ALGORITHMS; LEARNING SYSTEMS; NEURONS;

FEEDFORWARD NEURAL NETWORKS;

ALGORITHM; ANALYTICAL ERROR; ARTICLE; ARTIFICIAL NEURAL NETWORK; CONTROLLED STUDY; INTERMETHOD COMPARISON; MACHINE LEARNING; MATHEMATICAL ANALYSIS; MATHEMATICAL COMPUTING; MATHEMATICAL PARAMETERS; NERVE CELL; PRIORITY JOURNAL; RANDOMIZATION; REGRESSION ANALYSIS;

EID: 67349131281     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2008.10.016     Document Type: Article

References (19)

1. Cybenko G. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems 2 (1989) 303-314
2. Hornik K., Stinchcombe M., and White H. Multilayer feedforward networks are universal approximators. Neural Networks 2 (1989) 359-366
3. Ito Y. Approximation of functions on a compact set by finite sums of a sigmoid function without scaling. Neural Networks 4 (1991) 817-826
4. Hornik K. Approximation capabilities of multilayer feedforward networks. Neural Networks 4 (1991) 251-257
5. d by linear combinations of shifted rotations of a sigmoid function with and without scaling. Neural Networks 5 (1992) 105-115
6. Leshno M., Lin V.Y., Pinkus A., and Schocken S. Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural Networks 6 (1993) 861-867
7. Jones L.K. A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural networks. The Annals of Statistics 20 1 (1992) 608-613
8. Barron A.R. Universal approximation bounds for superpositions of a sigmoid function. IEEE Transactions on Information Theory 39 3 (1993) 930-945
9. Lee W.S., Bartlett P.L., and Williamson R.C. Efficient agnostic learning of neural networks with bounded fan-in. IEEE Transactions on Information Theory 42 6 (1996) 2118-2132
10. Huang G.-B., Zhu Q.-Y., and Siew C.-K. Extreme learning machine: theory and applications. Neurocomputing 70 (2006) 489-501
11. Huang G.-B., Chen L., and Siew C.-K. Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Transactions on Neural Networks 17 4 (2006) 879-892
12. Huang G.-B., and Chen L. Convex incremental extreme learning machine. Neurocomputing 70 (2007) 3056-3062
13. Voxman W.L., Roy J., and Goetschel H. Advanced Calculus: An Introduction to Modern Analysis (1981), Marcel Dekker, Inc.
14. Kwok T.-Y., and Yeung D.-Y. Objective functions for training new hidden units in constructive neural networks. IEEE Transactions on Neural Networks 8 5 (1997) 1131-1148
15. Y. LeCun, L. Bottou, G.B. Orr, K.-R. Müller, Efficient BackProp, in: Lecture Notes in Computer Science, vol. 1524, 1998, pp. 9-50.
16. Drucker H., Burges C.J.C., Kaufman L., Smola A., and Vapnik V. Support vector regression machines. In: Mozer M.C., Jordan M.I., and Petsche T. (Eds). Advances in Neural Information Processing Systems vol. 9 (1997), The MIT Press, Cambridge 155
17. Platt J. A resource-allocating network for function interpolation. Neural Computation 3 (1991) 213-225
18. Yingwei L., Saratchandran P., and Sundararajan N. A sequential learning scheme for function approximation using minimal radial basis function (rbf) neural networks. Neural Computation 9 (1997) 461-478
19. C. Blake, C. Merz, UCI repository of machine learning databases, Department of Information and Computer Sciences, University of California, Irvine, USA, 1998 〈http://www.ics.uci.edu/∼mlearn/MLRepository.html〉.