Feature selection for nonlinear models with extreme learning machines

Neurocomputing

Volumn 102, Issue , 2013, Pages 111-124

  Benoît, Frénay ^a,b   van Heeswijk, Mark ^b   Miche, Yoan ^b   Verleysen, Michel ^a   Lendasse, Amaury ^b,c,d  

^a UNIVERSITÉ CATHOLIQUE DE LOUVAIN   (Belgium)

^b AALTO UNIVERSITY   (Finland)

^c BASQUE FOUNDATION FOR SCIENCE   (Spain)

^d UNIVERSITY OF THE BASQUE COUNTRY UPV EHU   (Spain)

Author keywords

Extreme learning machines; Feature selection; Regression; Regularisation

Indexed keywords

DOMAIN KNOWLEDGE; EXTREME LEARNING MACHINE; FEATURE SUBSET; GENERALISATION; GRAPHICAL TOOLS; NON-LINEAR MODEL; REGRESSION; REGULARISATION; TRADE-OFF CURVES;

ERROR STATISTICS; FEATURE EXTRACTION; SET THEORY;

KNOWLEDGE ACQUISITION;

ARTICLE; CLINICAL PRACTICE; CONTROLLED STUDY; DIABETES MELLITUS; ELECTRICITY; EXPERIMENTAL STUDY; EXTREME LEARNING MACHINE; FEATURE SELECTION; GENERALIZATION ERROR; LASER; LEARNING ALGORITHM; MACHINE LEARNING; NONLINEAR SYSTEM; POLAND; PRIORITY JOURNAL; SPARSITY ERROR TRADE OFF CURVE; STATISTICAL MODEL; STATISTICAL PARAMETERS; SYSTEMATIC ERROR; THEORETICAL STUDY; VALIDATION PROCESS;

EID: 84870253587     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2011.12.055     Document Type: Article

References (28)

1. Hastie T., Tibshirani R., Friedman J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction Springer Series in Statistics 2009, Springer. 2nd ed.
2. Efron B., Hastie T., Johnstone L., Tibshirani R. Least angle regression. Ann. Stat. 2004, 32:407-499.
3. Kraskov A., Stögbauer H., Grassberger P. Estimating mutual information. Phys. Rev. E 2004, 69(6):066138. 10.1103/PhysRevE.69.066138.
4. Rossi F., Lendasse A., François D., Wertz V., Verleysen M. Mutual information for the selection of relevant variables in spectrometric nonlinear modelling. Chemometr. Intell. Lab. Syst. 2006, 80(2):215-226. 10.1016/j.chemolab.2005.06.010.
5. Huang G.-B., Zhu Q.-Y., Siew C.-K. Extreme learning machine: theory and applications. Neurocomputing 2006, 70(1-3):489-501.
6. Huang G.-B., Wang D., Lan Y. Extreme learning machines: a survey. Int. J. Mach. Learn. Cyb. 2011, 2:107-122.
7. Huang G.-B., Wang D. Advances in extreme learning machines (elm2010). Neurocomputing 2011, 74(16):2411-2412.
8. G.-B. Huang, H. Zhou, X. Ding, R. Zhang, Extreme learning machine for regression and multiclass classification, IEEE Trans. Syst. Man Cyb. B Cyb. 99 (2011) 1-17.
9. Huang G.-B., Chen L. Enhanced random search based incremental extreme learning machine. Neurocomputing 2008, 71(16-18):3460-3468. advances in Neural Information Processing (ICONIP 2006)/ Brazilian Symposium on Neural Networks (SBRN 2006).
10. Miche Y., Sorjamaa A., Bas P., Simula O., Jutten C., Lendasse A. OP-ELM: optimally-pruned extreme learning machine. IEEE Trans. Neural Netw. 2010, 21(1):158-162. 10.1109/TNN.2009.2036259.
11. R. Kohavi, A study of cross-validation and bootstrap for accuracy estimation and model selection, in: Proceedings of the 14th international joint conference on Artificial intelligence, vol. 2, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1995, pp. 1137-1143.
12. Efron B., Tibshirani R. An Introduction to the Bootstrap, Monographs on Statistics and Applied Probability 1993, Chapman & Hall.
13. Allen D.M. The relationship between variable selection and data augmentation and a method for prediction. Technometrics 1974, 16(1):125-127.
14. Tibshirani R. Regression shrinkage and selection via the lasso. J. R. Stat. Soc. B 1994, 58:267-288.
15. Bach J.M.F., Jenatton R., Obozinski G. Convex optimization with sparsity-inducing norms. Optimization for Machine Learning 2011, MIT Press. S.J.W.S. Sra, S. Nowozin (Eds.).
16. P. Bradley, O. L. Mangasarian, Feature selection via concave minimization and support vector machines, in: Machine Learning Proceedings of the Fifteenth International Conference (ICML 98), Morgan Kaufmann, 1998, pp. 82-90.
17. Zhu J., Rosset S., Hastie T., Tibshirani R. 1-norm support vector machines. Advances in Neural Information Processing Systems 2004, vol. 16. MIT Press.
18. Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques 2006, Springer. E.K. Burke, G. Kendall (Eds.).
19. Bishop C.M. Pattern Recognition and Machine Learning (Information Science and Statistics) 2007, Springer. 1st ed.
20. Haykin S. Neural Networks: A Comprehensive Foundation 1998, Prentice Hall.
21. Huang G.-B., Siew C.-K. Extreme learning machine with randomly assigned RBF kernels. Int. J. Inf. Technol. 2005, 11(1):16-24.
22. Rao C., Mitra S. Generalized Inverse of Matrices and its Applications 1971, Wiley.
23. Huang G.-B., Chen L., Siew C.-K. Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans. Neural Netw. 2006, 17(4):879-892.
24. Verleysen M., Rossi F., François D. Advances in feature selection with mutual information. Similarity-Based Clustering, Lecture Notes in Computer Science 2009, vol. 5400:52-69. Springer, Berlin/Heidelberg.
25. Environmental and industrial machine learning group, http://research.ics.tkk.fi/eiml/datasets.shtml.
26. Lendasse A., Lee J.A., Wertz V., Verleysen M. Forecasting electricity consumption using nonlinear projection and self-organizing maps. Neurocomputing 2002, 48(1-4):299-311.
27. A.S. Weigend, N.A. Gershenfeld, Results of the time series prediction competition at the Santa Fe Institute, in: International Symposium on Neural Networks, 1993.
28. Guillén A., Sovilj D., Mateo F., Rojas I., Lendasse A. Minimizing the delta test for variable selection in regression problems. Int. J. High Perform. Syst. Archit. 2008, 1(4):269-281.