TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization

Neurocomputing

Volumn 74, Issue 16, 2011, Pages 2413-2421

  Miche, Yoan ^a,b   van Heeswijk, Mark ^a   Bas, Patrick ^b   Simula, Olli ^a   Lendasse, Amaury ^a  

^a AALTO UNIVERSITY   (Finland)

^b GIPSA LAB   (France)

Author keywords

ELM; LARS; OP ELM; Regularization; Ridge regression; Tikhonov regularization

Indexed keywords

ELM; LARS; OP-ELM; REGULARIZATION; RIDGE REGRESSION; TIKHONOV REGULARIZATION;

LEARNING SYSTEMS; REGRESSION ANALYSIS;

PROBLEM SOLVING;

ARTICLE; CONTROLLED STUDY; INTERMETHOD COMPARISON; KERNEL METHOD; MACHINE LEARNING; MATHEMATICAL COMPUTING; OPTIMALLY PRUNED EXTREME LEARNING MACHINE; PERCEPTRON; PRIORITY JOURNAL; RELIABILITY; SENSITIVITY ANALYSIS; SUPPORT VECTOR MACHINE;

EID: 80051671932     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2010.12.042     Document Type: Article

References (35)

1. David M.A. The relationship between variable selection and data augmentation and a method for prediction. Technometrics 1974, 16(1):125-127.
2. Berger J. Minimax estimation of a multivariate normal mean under arbitrary quadratic loss. Journal of Multivariate Analysis 1976, 6(2):256-264.
3. Bishop C.M. Neural Networks for Pattern Recognition 1996, Oxford University Press, USA.
4. C.-C. Chang, C.-J. Lin, LIBSVM: a library for support vector machines, Software available at 〈〉, 2001. http://www.csie.ntu.edu.tw/~cjlin/libsvm.
5. W. Deng, Q. Zheng, L. Chen, Regularized extreme learning machine, in: IEEE Symposium on Computational Intelligence and Data Mining, CIDM '09, March 30th-April 2nd 2009, pp. 389-395.
6. Efron B., Hastie T., Johnstone I., Tibshirani R. Least angle regression. Annals of Statistics 2004, 32:407-499.
7. Feng G., Huang G.-B., Lin Q., Gay R. Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Transactions on Neural Networks 2009, 20(8):1352-1357.
8. A. Frank, A. Asuncion, UCI machine learning repository, 2010. http://archive.ics.uci.edu/ml.
9. Golub G.H., Heath M., Wahba G. Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 1979, 21(2):215-223.
10. EIML Group. The op-elm toolbox, Available online at, 2009. http://www.cis.hut.fi/projects/eiml/research/downloads/op-elm-toolbox.
11. Hastie T., Tibshirani R., Friedman J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction 2009, Springer. second ed.
12. Haykin S. Neural Networks: A Comprehensive Foundation 1998, Prentice Hall, ISBN 0132733501. second ed.
13. Hoerl A.E. Application of ridge analysis to regression problems. Chemical Engineering Progress 1962, 58:54-59.
14. Huang G.-B., Chen L. Convex incremental extreme learning machine. Neurocomputing 2007, 70(16-18):3056-3062. ISSN 0925-2312.
15. Huang G.-B., Chen L. Enhanced random search based incremental extreme learning machine. Neurocomputing 2008, 71(16-18):3460-3468. ISSN 0925-2312.
16. Huang G.-B., Siew C.-K. Extreme learning machine with randomly assigned rbf kernels. International Journal of Information Technology 2005, 11(1):16-24.
17. Huang G.-B., Chen L., Siew C.-K. Universal approximation using incremental constructive feedforward networks with. IEEE Transactions on Neural Networks 2005, 17:879-892.
18. Huang G.-B., Zhu Q.-Y., Mao K.Z., Siew C.-K., Saratchandran P., Sundararajan N. Can threshold networks be trained directly?. IEEE Transactions on Circuits and Systems II: Express Briefs 2006, 53(3):187-191. ISSN 1549-7747.
19. Huang G.-B., Zhu Q.-Y., Siew C.-K. Extreme learning machine: theory and applications. Neurocomputing 2006, 70(1-3):489-501.
20. Lan Y., Soh Y.C., Huang G.-B. Constructive hidden nodes selection of extreme learning machine for regression. Neurocomputing 2010, 73(16-18):3191-3199.
21. Li M.-B., Huang G.-B., Saratchandran P., Sundararajan N. Fully complex extreme learning machine. Neurocomputing 2005, 68:306-314. ISSN 0925-2312.
22. Y. Miche, P. Bas, C. Jutten, O. Simula, A. Lendasse, A methodology for building regression models using extreme learning machine: OP-ELM, in: M. Verleysen (Ed.), ESANN 2008, European Symposium on Artificial Neural Networks, Bruges, Belgium, d-side publ, (Evere, Belgium), 23-25 April 2008, pp. 247-252.
23. Y. Miche, A. Sorjamaa, A. Lendasse, OP-ELM: theory, experiments and a toolbox, in: R. Neruda, V. Kurková, J. Koutník (Eds.), LNCS-Artificial Neural Networks-ICANN 2008-Part I, Lecture Notes in Computer Science, vol. 5163, Springer Berlin/Heidelberg, September 2008, pp.145-154.
24. Miche Y., Sorjamaa A., Bas P., Simula O., Jutten C., Lendasse A. OP-ELM: optimally pruned extreme learning machine. IEEE Transactions on Neural Networks 2010, 21(1):158-162.
25. Nelder J.A., Mead R. A simplex method for function minimization. The Computer Journal 1965, 7(4):308-313.
26. A.B. Owen, A robust hybrid of lasso and ridge regression, Technical Report, Stanford University, 2006.
27. Radhakrishna Rao C., Mitra S.K. Generalized Inverse of Matrices and Its Applications 1971, John Wiley & Sons Inc.
28. Rasmussen C.E., Williams C.K.I. Gaussian Processes for Machine Learning 2006, The MIT Press.
29. T. Similä, J. Tikka, Multiresponse sparse regression with application to multidimensional scaling, in: International Conference on Artificial Neural Networks (ICANN), Lecture Notes in Computer Science, vol. 3697, Warsaw, Poland, 11-15 September 2005, pp. 97-102.
30. R.A. Thisted, Ridge regression, minimax estimation, and empirical bayes methods, Technical Report 28, Division of Biostatistics, Stanford University, 1976.
31. Tibshirani R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B 1994, 58:267-288.
32. Tychonoff A.N. Solution of incorrectly formulated problems and the regularization method. Soviet Mathematics 1963, 4:1035-1038.
33. L. Yuan, S. Yeng Chai, G.-B. Huang, Random search enhancement of error minimized extreme learning machine, in: M. Verleysen (Ed.), European Symposium on Artificial Neural Networks (ESANN) 2010, d-side Publications, Bruges, Belgium, April 28-30th 2010, pp. 327-332.
34. Zhao P., Rocha G.V., Yu B. Grouped and hierarchical model selection through composite absolute penalties. Annals of Statistics 2009, 37(6A):3468-3497.
35. Zou H., Hastie T. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society Series B 2005, 67(2):301-320.