A novel automatic two-stage locally regularized classifier construction method using the extreme learning machine

Neurocomputing

Volumn 102, Issue , 2013, Pages 10-22

  Du, Dajun ^a,b   Li, Kang ^b   Irwin, George W ^b   Deng, Jing ^b  

^a SHANGHAI UNIVERSITY   (China)

^b QUEEN'S UNIVERSITY BELFAST   (United Kingdom)

Author keywords

Classification; Extreme learning machine; Leave one out (LOO) misclassification rate; Linear in the parameters model; Regularization; Two stage stepwise selection

Indexed keywords

COMPUTATIONAL EXPENSE; CONSTRUCTION METHOD; EXTREME LEARNING MACHINE; FAST IMPLEMENTATION; FINAL PREDICTION ERRORS; GAUSSIAN FUNCTIONS; GENERALIZATION CAPABILITY; LEAVE-ONE-OUT; MISCLASSIFICATION RATES; NON-LINEAR PARAMETERS; REGULARIZATION; STOPPING CRITERION; TRAINING DATA; TWO-STAGE STEPWISE SELECTION;

ALGORITHMS; FACE RECOGNITION; KNOWLEDGE ACQUISITION; LEARNING SYSTEMS;

CLASSIFICATION (OF INFORMATION);

ARTICLE; AUTOMATIC TWO STAGE LOCALLY REGULARIZED CLASSIFIER CONSTRUCTION ALGORITHM; BREAST CANCER; CLASSIFIER; COMPARATIVE STUDY; COMPUTATIONAL COMPLEXITY ANALYSIS; CONTROLLED STUDY; DATA ANALYSIS; DIABETES MELLITUS; EXTREME LEARNING MACHINE; GAUSSIAN FUNCTION WIDTH; GENERALIZATION CAPABILITY; HEART; LEARNING ALGORITHM; LEAVE ONE OUT MISCLASSIFICATION RATE; LINEAR IN THE PARAMETER CLASSIFIER; NONLINEAR SYSTEM; POLYNOMIAL TERM FUNCTION; PRIORITY JOURNAL; SIMULATION; SPARSITY; STATISTICAL ANALYSIS; STATISTICAL PARAMETERS;

EID: 84870239889     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2011.12.052     Document Type: Article

References (36)

1. Chen W., Metz C.E., Giger M.L., Drukker K. A novel hybrid linear/nonlinear classifier for two-class classification: theory, algorithm and applications. IEEE Trans. Med. Imaging 2010, 29(2):428-441.
2. Zhang R., Huang G.-B., Sundararajan N., Saratchandran P. Improved GAP-RBF network for classification problems. Neurocomputing 2007, 70:3011-3018.
3. Twala B. Multiple classifier application to credit risk assessment. Expert Syst. Appl. 2010, 37:3326-3336.
4. S.R. Gunn, Support Vector Machines for Classification and Regression, Technical Report, University of Southampton, 1998.
5. Ulas A., Semerci M., Y1ld1z O.T., Alpayd1n E. Incremental construction of classifier and discriminant ensembles. Inf. Sci. 2009, 179:1298-1318.
6. Elanayar S., Shin Y.C. Radial basis function neural network for approximation and estimation of nonlinear stochastic dynamic systems. IEEE Trans. Neural Networks 1994, 5(4):594-603.
7. Xu L., Krzyzak A., Yuille A. On radial basis function nets and kernel regression: statistical consistency, convergence rates, and receptive field size. Neural Networks 1994, 7(4):609-628.
8. Wang X., Chen A., Feng H. Upper integral network with extreme learning mechanism. Neurocomputing 2011, 74:2520-2525.
9. Huang G.-B., Wang D.H., Lan Y. Extreme learning machines: a survey. Int. J. Mach. Learn. Cybern. 2011, 2:107-122.
10. Huang G.-B., Zhu Q.-Y., Siew C.-K. Extreme learning machine: theory and applications. Neurocomputing 2006, 70:489-501.
11. Huang G.-B., Zhou H., Ding X., Zhang R. Extreme learning machine for regression and multi-class classification. IEEE Trans. Syst. Man Cybern. B: Cybern. 2012, 42(2):513-529.
12. Hong X., Sharkey P.M., Warwick K. Automatic nonlinear predictive model-construction algorithm using forward regression and the PRESS static. IEE Proc. Control Theory Appl. 2003, 150(3):245-254.
13. Mao K.Z. RBF neural network center selection based on Fisher ratio class separability measure. IEEE Trans. Neural Netw. 2002, 13(5):1211-1217.
14. Li K., Peng J., Irwin G.W. A fast nonlinear model identification method. IEEE Trans. Autom. Control 2005, 50(8):1211-1216.
15. Peng J., Li K., Irwin G.W. A novel continuous forward algorithm for RBF neural modelling. IEEE Trans. Autom. Control 2007, 52(1):117-122.
16. Xiao J., Hen C., Jiang X., Liu D. A dynamic classifier ensemble selection approach for noise data. Inf. Sci. 2010, 180:3402-3421.
17. Poggio T., Torre V., Koch C. Computational vision and regularization theory. Nature 1985, 317:314-319.
18. Hoerl A.E., Kennard R.W. Ridge regression: biased estimation for nonorthogonal problems. Technometrics 1970, 12(1):55-67.
19. MacKay D.J.C. Bayesian interpolation. Neural Comput. 1992, 4(3):415-447.
20. Tipping M.T. Sparse Bayesian learning and the relevance vector machine. J. Mach. Learn. Res. 2001, 1:211-244.
21. Chen S., Hong X., Harris C.J., Sharkey P.M. Sparse modelling using orthogonal forward regression with PRESS statistic and regularization. IEEE Trans. Syst. Man Cybern. B: Cybern. 2004, 34(2):898-911.
22. D. Du, X. Li, M.R. Fei, G.W. Irwin, A novel locally regularized automatic construction method for RBF neural models, Neurocomputing, , in press. http://dx.doi.org/10.1016/j.neucom.2011.05.045.
23. Akaike H. Fitting autoregressive models for prediction. Ann. Inst. Stat. Math. 1969, 21(1):243-247.
24. Stone M. Cross validatory choice and assessment of statistical predictions. J. R. Stat. Soc. 1974, 36(1):117-147.
25. Hong X., Mitchell R.J. Backward elimination model construction for regression and classification using leave one out criteria. Int. J. Syst. Sci. 2007, 38(2):101-113.
26. Li K., Peng J., Bai E.W. A two-stage algorithm for identification of nonlinear dynamic systems. Automatica 2006, 42:1189-1197.
27. Deng J., Li K., Irwin G.W. Fast automatic two-stage nonlinear model identification based on the extreme learning machine. Neurocomputing 2011, 74(16):2422-2429.
28. Li K., Peng J., Bai E.W. Two-stage mixed discrete-continuous identification of radial basis function (RBF) neural models for nonlinear systems. IEEE Trans. Circuits Syst. I 2009, 56(3):630-643.
29. Hong X., Chen S., Harris C.J. A fast linear-in-the-parameters classifier construction algorithm using orthogonal forward selection to minimize leave-one-out misclassification rate. Int. J. Syst. Sci. 2008, 39(2):119-125.
30. Hong X., Mitchell R.J., Chen S., Harris C.J., Li K., Irwin G.W. Model selection approaches for non-linear system identification: a review. Int. J. Syst. Sci. 2008, 39(10):925-946.
31. Rao C.R., Mitra S.K. Generalized Inverse of Matrices and Its Applications 1971, Wiley, New York.
32. Myers R.H. Classical and Modern Regression with Applications 1990, PWS-KENT, Boston.
33. Rätsch G., Onoda T., Müller K.R. Soft margins for AdaBoost. Mach. Learn. 2001, 42:287-320.
34. University of East Anglia Computational Biology Laboratory, URL 〈〉. http://theoval.cmp.uea.ac.uk/gcc/matlab/.
35. Freund Y., Schapire R.E. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 1997, 55(1):119-139.
36. Q.-Y. Zhu, G.-B. Huang, MATLAB Codes of ELM Algorithm. URL 〈〉. http://www.ntu.edu.sg/home/egbhuang/ELM-Codes.htm.