Is extreme learning machine feasible? A theoretical assessment (Part II)

IEEE Transactions on Neural Networks and Learning Systems

Volumn 26, Issue 1, 2015, Pages 21-34

  Lin, Shaobo ^a,b   Liu, Xia ^b   Fang, Jian ^b   Xu, Zongben ^b  

^a WENZHOU UNIVERSITY   (China)

^b XI'AN JIAOTONG UNIVERSITY   (China)

Author keywords

Extreme learning machine (ELM); Gaussian kernel; generalization capability; neural networks

Indexed keywords

CHEMICAL ACTIVATION; COMPUTATION THEORY; GAUSSIAN DISTRIBUTION; KNOWLEDGE ACQUISITION; NEURAL NETWORKS; RANDOM PROCESSES;

ACTIVATION FUNCTIONS; COEFFICIENT REGULARIZATIONS; COMPUTATIONAL BURDEN; ESSENTIAL CHARACTERISTIC; EXTREME LEARNING MACHINE; GAUSSIAN KERNELS; GENERALIZATION CAPABILITY; UNCERTAINTY PROBLEMS;

LEARNING SYSTEMS;

ALGORITHM; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; BIOLOGICAL MODEL; COMPUTER SIMULATION; HUMAN; LEARNING; NERVE CELL; PHYSIOLOGY; PROBABILITY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; HUMANS; LEARNING; MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER); NEURONS; PROBABILITY;

EID: 84919660197     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2014.2336665     Document Type: Article

References (37)

1. R. F. Bass and K. Gröchenig, "Random sampling of multivariate trigonometric polynomials," SIAM J. Math. Anal., vol. 36, no. 3, pp. 773-795, 2005.
2. P. B. Borwein and T. Erdélyi, Polynomials and Polynomial Inequalities. New York, NY, USA: Springer-Verlag, 1995.
3. F. Cucker and S. Smale, "On the mathematical foundations of learning," Bull. Amer. Math. Soc., vol. 39, no. 1, pp. 1-49, 2001.
4. W. Deng, Q. Zheng, and L. Chen, "Regularized extreme learning machine," in Proc. IEEE Symp. Comput. Inell. Data Mining (CIDM), Mar./Apr. 2009, pp. 389-395.
5. R. DeVore and G. Lorentz, Constructive Approximation. Berlin, Germany: Springer-Verlag, 1993.
6. M. Eberts and I. Steinwart, "Optimal learning rates for least squares SVMs using Gaussian kernels," in Advances in Neural Information Processing Systems, vol. 24. Dec. 2011, pp. 1539-1547.
7. T. Erdélyi, "Bernstein-type inequalities for linear combinations of shifted Gaussian," Bull. London Math. Soc., vol. 38, no. 1, pp. 124-138, 2006.
8. S. Ferrari et al., "Illuminance prediction through extreme learning machines," in Proc. IEEE Workshop Environ. Energy Struct. Monitor. Syst. (EESMS), Sep. 2012, pp. 97-103.
9. L. Györfy, M. Kohler, A. Krzyzak, and H. Walk, A Distribution-Free Theory of Nonparametric Regression. Berlin, Germany: Springer-Verlag, 2002.
10. M. Hagan, M. Beale, and H. Demuth, Neural Network Design. Boston, MA, USA: PWS-Kent, 1996.
11. G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, "Extreme learning machine: Theory and applications," Neurocomputing, vol. 70, nos. 1-3, pp. 489-501, 2006.
12. G.-B. Huang, L. Chen, and C.-K. Siew, "Universal approximation using incremental constructive feedforward networks with random hidden nodes," IEEE Trans. Neural Netw., vol. 17, no. 4, pp. 879-892, Jul. 2006.
13. G.-B. Huang, Q.-Y. Zhu, K. Z. Mao, C.-K. Siew, P. Saratchandran, and N. Sundararajan, "Can threshold networks be trained directly?" IEEE Trans. Circuits Syst. II, Exp. Briefs, vol. 53, no. 3, pp. 187-191, Mar. 2006.
14. G.-B. Huang and L. Chen, "Convex incremental extreme learning machine," Neurocomputing, vol. 70, nos. 16-18, pp. 3056-3062, 2007.
15. G.-B. Huang, H. Zhou, X. Ding, and R. Zhang, "Extreme learning machine for regression and multiclass classification," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 42, no. 2, pp. 513-529, Apr. 2012.
16. G.-B. Huang, D. H. Wang, and Y. Lan, "Extreme learning machines: A survey," Int. J. Mach. Learn. Cybern., vol. 2, no. 2, pp. 107-122, 2011.
17. S. Lin, C. Xu, J. Zeng, and J. Fang, "Does generalization performance of lq regularization learning depend on q? A negative example," ArXiv preprint, arXiv:1307.6616, 2013.
18. S. Lin, J. Zeng, J. Fang, and Z. Xu, "Learning rate of lq coefficient regularization learning with Gaussian kernel," Neural Comput., doi: 10.1162/NECO-a-00641.
19. S. Lin, X. Liu, Y. Rong, and Z. Xu, "Almost optimal estimates for approximation and learning by radial basis function networks," Mach. Learn., vol. 95, no. 2, pp. 147-164, 2014.
20. X. Liu, S. B. Lin, J. Fang, and Z. B. Xu, "Is extreme learning machine feasible? A theoretical assessment (Part I)," IEEE Trans. Neural Netw. Learn. Syst., doi: 10.1109/TNNLS.2014.2335212.
21. H. N. Mhaskar, F. J. Narcowich, and J. D. Ward, "Approximation properties of zonal function networks using scattered data on the sphere," Adv. Comput. Math., vol. 11, nos. 2-3, pp. 121-137, 1999.
22. H. N. Mhaskar, F. J. Narcowich, and J. D. Ward, "Spherical Marcinkiewicz-Zygmund inequalities and positive quadrature," Math. Comput., vol. 70, no. 235, pp. 1113-1130, 2000.
23. I. Marques and M. Granã, "Face recognition with lattice independent component analysis and extreme learning machines," Soft Comput., vol. 16, no. 9, pp. 1525-1537, 2012.
24. C. R. Rao and S. K. Mitra, Generalized Inverse of Matrices and Its Application. New York, NY, USA: Wiley, 1971.
25. D. Serre, Matrices: Theory and Applications. New York, NY, USA: Springer-Verlag, 2002.
26. L. Shi, Y.-L. Feng, and D.-X. Zhou, "Concentration estimates for learning with l1-regularizer and data dependent hypothesis spaces," Appl. Comput. Harmon. Anal., vol. 31, no. 2, pp. 286-302, 2011.
27. E. Soria-Olivas et al., "BELM: Bayesian extreme learning machine," IEEE Trans. Neural Netw., vol. 22, no. 3, pp. 505-509, Mar. 2011.
28. I. Steinwart and C. Scovel, "Fast rates for support vector machines using Gaussian kernels," Ann. Statist., vol. 35, no. 2, pp. 575-607, 2007.
29. H. Sun and Q. Wu, "Least square regression with indefinite kernels and coefficient regularization," Appl. Comput. Harmon. Anal., vol. 30, no. 1, pp. 96-109, 2011.
30. J. Tang, D. Wang, and T. Chai, "Predicting mill load using partial least squares and extreme learning machines," Soft Comput., vol. 16, no. 9, pp. 1585-1594, 2012.
31. H. Tong, D.-R. Chen, and F. Yang, "Least square regression with l p-coefficient regularization," Neural Comput., vol. 22, no. 12, pp. 3221-3235, 2010.
32. Q. Wu and D.-X. Zhou, "Learning with sample dependent hypothesis spaces," Comput. Math. Appl., vol. 56, no. 11, pp. 2896-2907, 2008.
33. T. F. Xie and F. L. Cao, "The rate of approximation of Gaussian radial basis neural networks in continuous function space," Acta Math. Sinica, English Ser., vol. 29, no. 2, pp. 295-302, 2013.
34. J. Xu, H. Zhou, and G.-B. Huang, "Extreme learning machine based fast object recognition," in Proc. 15th Int. Conf. Inform. Fusion (FUSION), Jul. 2012, pp. 1490-1496.
35. R. Zhang, Y. Lan, G.-B. Huang, and Z.-B. Xu, "Universal approximation of extreme learning machine with adaptive growth of hidden nodes," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 2, pp. 365-371, Feb. 2012.
36. D.-X. Zhou and K. Jetter, "Approximation with polynomial kernels and SVM classifiers," Adv. Comput. Math., vol. 25, nos. 1-3, pp. 323-344, 2006.
37. Q.-Y. Zhu, A. K. Qin, P. N. Suganthan, and G.-B. Huang, "Evolutionary extreme learning machine," Pattern Recognit., vol. 38, no. 10, pp. 1759-1763, 2005.