Dynamic extreme learning machine and its approximation capability

IEEE Transactions on Cybernetics

Volumn 43, Issue 6, 2013, Pages 2054-2065

  Zhang, Rui ^a,b   Lan, Yuan ^a   Huang, Guang Bin ^a   Xu, Zong Ben ^c   Soh, Yeng Chai ^a  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

^b NORTHWEST UNIVERSITY   (China)

^c XI'AN JIAOTONG UNIVERSITY   (China)

Author keywords

Dynamic learning; extreme learning machine (ELM); feedforward neural networks; universal approximation

Indexed keywords

ACTIVATION FUNCTIONS; APPROXIMATION CAPABILITIES; DYNAMIC LEARNING; EXTREME LEARNING MACHINE; FEED-FORWARD NETWORK; GENERALIZATION PERFORMANCE; NETWORK SIZE; UNIVERSAL APPROXIMATION;

FEEDFORWARD NEURAL NETWORKS; LEARNING SYSTEMS; NETWORK ARCHITECTURE; NETWORK LAYERS;

KNOWLEDGE ACQUISITION;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; DECISION SUPPORT SYSTEM; METHODOLOGY; THEORETICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; DECISION SUPPORT TECHNIQUES; MODELS, THEORETICAL; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED;

EID: 84890110727     PISSN: 21682267     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCYB.2013.2239987     Document Type: Article

References (36)

1. A. R. Barron, "Universal approximation bounds for superpositions of a sigmoidal functions," IEEE Trans. Inf. Theory, vol. 39, no. 3, pp. 930-945, May 1993.
2. E. B. Baum, "On the capabilities of multilayer perceptrons," J. Complexity, vol. 4, no. 3, pp. 193-215, Sep. 1998.
3. C. M. Bishop, Neural Networks for Pattern Recognition. New York: Oxford Univ. Press, 1995.
4. C. L. Black and C. J. Merz, "UCI repository of machine learning databases," Univ. California Irvine, Dept. Inf. Comput. Sci., irvine, CA, 1998.
5. C.-C. Chang and C.-J. Lin, "LIBSVM: A library for support vector machines," 2001.
6. G. Feng, G.-B. Huang, Q. Lin, and R. Gay, "Error minimized extreme learning machine with growth of hidden nodes and incremental learning," IEEE Trans. Neural Netw., vol. 20, no. 8, pp. 1352-1357, Aug. 2009.
7. M. T. Hagan, H. B. Demuth, and M. H. Beale, Neural Network Design. Boston, MA: PWS-Kent, 1996.
8. S. Haykin, Neural Networks: A Comprehensive Foundation. NewYork: Macmillan, 1994.
9. K. Hornik, "Approximation capabilities of multilayer feedforward networks," Neural Netw., vol. 4, no. 2, pp. 251-257, 1991.
10. G.-B. Huang, "Learning capability and storage capacity of two-hidden layer feedforward networks," IEEE Trans. Neural Netw., vol. 14, no. 2, pp. 274-281, Mar. 2003.
11. G.-B. Huang and L. Chen, "Convex incremental extreme learning machine," Neurocomputing, vol. 70, no. 16-18, pp. 3056-3062, Oct. 2007. (Pubitemid 47308612)
12. G.-B. Huang and L. Chen, "Enhanced random search based incremental extreme learning machine," Neurocomputing, vol. 71, no. 16-18, pp. 3460-3468, Oct. 2008.
13. G.-B. Huang, L. Chen, and C.-K. Siew, "Universal approximation using incremental constructive feedforward networks with random hidden nodes," IEEE Trans. Neural Netw., vol. 17, no. 4, pp. 879-892, Jul. 2006. (Pubitemid 44194149)
14. G.-B. Huang, X. Ding, and H. Zhou, "Optimization method based extreme learning machine for classification," Neurocomputing, vol. 74, no. 1-3, pp. 155-163, Dec. 2010.
15. G.-B. Huang, N.-Y. Liang, H.-J. Rong, P. Saratchandran, and N. Sundararajan, "On-line sequential extreme learning machine," in Proc. IASTED Int. Conf. Comput. Intell., Calgary, AB, Canada, 2005.
16. G.-B. Huang and C.-K. Siew, "Extreme learning machine: RBF network case," in Proc. 8th Int. Conf. Control Autom. Robot. Vis., 2004, vol. 2, pp. 1029-1036. (Pubitemid 40893698)
17. G.-B. Huang and C.-K. Siew, "Extreme learning machine with randomly assigned RBF kernels," Int. J. Technol., vol. 11, no. 1, pp. 16-24, 2005.
18. G.-B. Huang, D. Wang, and L. Yuan, "Extreme learning machines: A survey," Int. J. Mach. Learn. Cybern., vol. 2, no. 2, pp. 107-122, Jun. 2011.
19. G.-B. Huang, H. Zhou, X. Ding, and R. Zhang, "Extreme learning machines for regression and multi-class classification," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 42, no. 2, pp. 513-529, Apr. 2012.
20. G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, "Extreme learning machine: A new learning scheme of feedforward neural networks," in Proc. Int. Joint Conf. Neural Netw., 2004, vol. 2, pp. 985-990. (Pubitemid 40011594)
21. G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, "Extreme learning machine: Theory and applications," Neurocomputing, vol. 70, no. 1-3, pp. 489-501, Dec. 2006. (Pubitemid 44615772)
22. G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, "Real-time learning capability of neural networks," IEEE Trans. Neural Netw., vol. 17, no. 4, pp. 863-878, Jul. 2006. (Pubitemid 44194148)
23. G.-B. Huang, Q.-Y. Zhu, M.-K. Zhi, C.-K. Siew, P. Saratchandran, and N. Sundararajan, "Can threshold networks be trained directly?" IEEE Trans. Circuits Syst. II, Exp. Briefs, vol. 53, no. 3, pp. 187-191, Mar. 2006.
24. T.-Y. Kwok and D.-Y. Yeung, "Objective functions for training new hidden units in constructive neural networks," IEEE Trans. Neural Netw., vol. 8, no. 5, pp. 1131-1148, Sep. 1997. (Pubitemid 127763344)
25. Y. Lan, Y. C. Soh, and G.-B. Huang, "Ensemble of online sequential extreme learning machine," Neurocomputing, vol. 72, no. 13-15, pp. 3391-3395, Aug. 2009.
26. M. Leshno, V. Y. Lin, A. Pinkus, and S. Schocken, "Multilayer feedforward networks with a nonpolynomial activation function can approximate any function," Neural Netw., vol. 6, no. 6, pp. 861-867, 1993. (Pubitemid 23713273)
27. M.-B. Li, G.-B. Huang, P. Saratchandran, and N. Sundararajan, "Fully complex extreme learning machine," Neurocomputing, vol. 68, pp. 306-314, Oct. 2005. (Pubitemid 41235768)
28. N.-Y. Liang, G.-B. Huang, P. Saratchandran, and N. Sundararajan, "A fast and accurate online sequential learning algorithm for feedforward networks," IEEE Trans. Neural Netw., vol. 17, no. 6, pp. 1411-1423, Nov. 2006. (Pubitemid 44824256)
29. C. G. Looney, Pattern Recognition Using Neural Networks: Theory and Algorithms for Engineers and Scientists. New York: Oxford Univ. Press, 1997.
30. R. Meir and V. E.Maiorov, "On the optimality of neural-network approximation using incremental algorithms," IEEE Trans. Neural Netw., vol. 11, no. 2, pp. 323-337, Mar. 2000.
31. J. Park and I. W. Sandberg, "Universal approximation using radialbasis- function networks," Neural Comput., vol. 3, no. 2, pp. 246-257, Jun. 1991.
32. C. R. Rao and S. K. Mitra, Generalized Inverse of Matrices and Its Applications. New York: Wiley, 1971.
33. H.-J. Rong, G.-B. Huang, N. Sundararajan, and P. Saratchandran, "Online sequential fuzzy learning machine for function approximation and classfication problem," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 39, no. 4, pp. 1067-1072, Aug. 2009.
34. A. J. Smola and B. Scholkopf, "A tutorial on supprot vector regression," Stat. Comput., vol. 14, no. 3, pp. 199-222, Aug. 2004.
35. R. Zhang, Y. Lan, G.-B. Huang, and Y. C. Soh, "Extreme learning machine with adaptive growth of hidden nodes and incremental updating of output weights," in Proc. 2nd Int. Conf. Auton. Intell. Syst., 2011, vol. 6752, pp. 253-262, Lecture Notes in Artificial Intelligence.
36. R. Zhang, Y. Lan, G.-B. Huang, and Z.-B. Xu, "Universal approximation of extreme learning machine with adaptive growth of hidden nodes," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 2, pp. 365-371, Feb. 2012.