Small number of hidden units for ELM with two-stage linear model

IEICE Transactions on Information and Systems

Volumn E91-D, Issue 4, 2008, Pages 1042-1049

  Huynh, Hieu Trung ^a   Won, Yonggwan ^a  

^a Chonnam National University   (South Korea)

Author keywords

Extreme learning machine; Least squares scheme; Linear model; Neural networks; Single hidden layer feedforward neural networks

Indexed keywords

FEEDFORWARD NEURAL NETWORKS; INVERSE PROBLEMS; KNOWLEDGE ACQUISITION; LEARNING SYSTEMS; NETWORK LAYERS; NEURAL NETWORKS; RADIAL BASIS FUNCTION NETWORKS; TRANSFER MATRIX METHOD;

ACTIVATION FUNCTIONS; CLASSIFICATION ACCURACY; EXTREME LEARNING MACHINE; GENERALIZATION PERFORMANCE; LEAST SQUARES SCHEMES; LINEAR MODELING; MATRIX INVERSE OPERATION; SINGLE-HIDDEN LAYER FEEDFORWARD NEURAL NETWORKS;

LEARNING ALGORITHMS;

EID: 55749110008     PISSN: 09168532     EISSN: 17451361     Source Type: Journal    
DOI: 10.1093/ietisy/e91-d.4.1042     Document Type: Article

References (31)

1. D. Nguyen and B.Widrow, "Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights," Int'l Joint Conf. Neural Neural Networks, vol.3, pp.21-26, 1990.
2. J.Y.F. Yam and T.W.S. Chow, "Feedforward networks training speed enhancement by optimal initialization of the synaptic coefficients," IEEE Trans. Neural Netw., vol.12, no.2, pp.430-434, 2001.
3. N.B. Karayiannis and A.N. Venetsanopoulos, Artificial neural networks: Learning algorithms, performance evaluation, and applications, Kluwer Academic, Boston, MA, 1993.
4. Y. LeCun, L. Bottou, G.B. Orr, and K.R. Müler, "Efficient backprop," Lecture Notes in Computer Science, vol.1524, pp.9-50, 1998.
5. S.M.A. Burney, T.A. Jilani, and C. Ardil, "A comparison of first and second order training algorithms for artificial neural networks," International Journal of Computational Intelligence, vol.1, no.2, pp.218-224, 2004.
6. L.S. Nghia, J. Sjöerg, and M. Viberg, "Adaptive neural nets filter using a recursive levenberg-marquardt search direction," Proc. Asilomar Conf. Signals, Syst., Comput., vol.1-4, pp.697-701, Nov. 1998.
7. V.S. Asirvadam, S.F. McLoone, and G. Irwin, "Parallel and separable recursive levenberg-marquardt training algorithm," Proc. 12th IEEE Workshop Neural Net. Signal Process., pp.129-138, Sept. 2002.
8. S. Lawrence and C.L. Giles, "Overfitting and neural networks: Conjugate gradient and backpropagation," Proc. IEEE-INNS-ENNS Int'l Joint Conf. on Neural networks, vol.1, pp.114-119, July 2000.
9. Z.P. Liu and J.P. Castagna, "Avoiding overfitting caused by noise using a uniform training mode," Int'l Joint Conf. on Neural Networks, vol.3, pp.1788-1793, July 1999.
10. S. Dumais, J. Platt, and D. Heckerman, "Inductive learning algorithms and representations for text categorization," Proc. Int'l Conf. Information and Knowledge Management, vol.3, pp.148-154, 1998.
11. C.W. Hsu and C.J. Lin, "A comparison of methods for multiclass support vector machines," IEEE Trans. Neural Netw., vol.13, no.2, pp.415-425, March 2002.
12. S.X. Lu and X.Z. Wang, "A comparison among four svm classification methods: Lsvm, nlsvm, ssvm and nsvm," Proc. Int'l Conf. on Machine Learning and Cybernetics, pp.4277-4282, Aug. 2004.
13. T. Joachims, "Text categorization with support vector machines: Learning with many relevant features," Proc. Eur. Conf. Machine Learning, pp.137-142, 1998.
14. O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukherjee, "Choosing multiple parameters for support vector machines," Mach. Learn., vol.46, no.1, pp.131-159, 2002.
15. D. Decoste and K. Wagstaff, "Alpha seeding for support vector machines," Proc. Intl Conf. Knowledge Discovery and Data Mining, pp.345-349, 2000.
16. S.S. Keerthi, "Efficient tuning of svm hyperparameters using radius/margin bound and iterative algorithms," IEEE Trans. Neural Netw., vol.13, no.5, pp.1225-1229, Sept. 2002.
17. G.B. Huang, Y.Q. Chen, and H.A. Babri, "Classification ability of single hidden layer feedforward neural networks," IEEE Trans. Neural Netw., vol.11, no.3, pp.799-801, May 2000.
18. G.B. Huang, Q.Y. Zhu, and C.K. Siew, "Extreme learning machine: Theory and applications," Neurocomputing, vol.70, pp.489-501, 2006.
19. G.B. Huang, Q.Y. Zhu, and C.K. Siew, "Extreme learning machine: A new learning scheme for feedforward neural networks," Proc. Int'l Joint Conf. on Neural Networks, July 2004.
20. D. Serre, Matrices: Theory and Applications, Springer, New York, 2002.
21. N.Y. Liang, G.B. Huang, P. Saratchandran, and N. Sundararajan, "A fast and accurate online sequential learning algorithm for feedforward networks," IEEE Trans. Neural Netw., vol.17, no.6, pp.1411-1423, 2006.
22. P.L. Bartlett, "The sample complexity of pattern classification with neural networks: The size of the weights is more important than the size of the network," IEEE Trans. Inf. Theory, vol.44, no.2, pp.525-536, March 1998.
23. G.H. Golub and C.F.V. Loan, Matrix computations, 3rd ed., Johns Hopkins University Press, Baltimore, MD, USA, 1996.
24. T.O.G. Räsch and K.R. Müler, "An improvement of adaboost to avoid overfitting," Proc. 5th Int'l Conf. on Neural Information Processing, 1998.
25. E. Romero and R. Alquézar, "A new incremental method for function approximation using feedforward neural networks," Proc. INNS-IEEE Int'l Joint Conf. on Neural Networks, pp.1968-1973, 2002.
26. Y. Freund and R.E. Schapire, "Experiments with a new boosting algorithm," Proc. 13th Int'l Conf. on Machine Learning, pp.148-156, 1996.
27. D.R. Wilson and T.R. Martinez, "Heterogeneous radial basis functionnetworks," Proc. Int'l Conf. on Neural Networks, pp.1263-1267, June 1996.
28. C. Constantinopoulos and A. Likas, "An incremental training method for probabilistic rbf network," IEEE Trans. Neural Netw., vol.17, no.4, pp.966-974, July 2006.
29. G.B. Huang, P. Saratchandran, and N. Sundararajan, "An efficient sequential learning algorithm for growing and pruning rbf (gap-rbf) networks," IEEE Trans. Syst., Man Cybern. B, Cybern., vol.34, no.6, pp.2284-2292, 2004.
30. D. Newman, S. Hettich, C. Blake, and C. Merz, "UCI repository of machine learning databases," University of California, Irvine, 1998, http://www.ics.edu/̃mlearn/MLRepository.html
31. L. Yingwei, N. Sundararajan, and P. Saratchandran, "Performance evaluation of a sequential minimal radial basis function (rbf) neural network learning algorithm," IEEE Trans. Neural Netw., vol.9, no.2, pp.308-318, March 1998.