Direct Kernel Perceptron (DKP): Ultra-fast kernel ELM-based classification with non-iterative closed-form weight calculation

Neural Networks

Volumn 50, Issue , 2014, Pages 60-71

  Fernández Delgado, Manuel ^a   Cernadas, Eva ^a   Barro, Senén ^a   Ribeiro, Jorge ^c   Neves, José ^b  

^a UNIVERSITY OF SANTIAGO DE COMPOSTELA   (Spain)

^b UNIVERSITY OF MINHO   (Portugal)

^c INSTITUTO POLITÉCNICO DE VIANA DO CASTELO   (Portugal)

Author keywords

Analytical weight calculation; Closed form solution; Extreme learning machine; Kernel based classification; Margin maximization; Parallel Delta rule; Support vector machine

Indexed keywords

ADAPTIVE BOOSTING; DECISION TREES; IMAGE RETRIEVAL; ITERATIVE METHODS; KNOWLEDGE ACQUISITION; NEURAL NETWORKS; PARAMETERIZATION; VECTOR SPACES;

CLOSED FORM SOLUTIONS; EXTREME LEARNING MACHINE; LINEAR DISCRIMINANT ANALYSIS; PARALLEL DELTA RULE; PROBABILISTIC NEURAL NETWORKS; RADIAL BASIS FUNCTION NEURAL NETWORKS; REGULARIZATION PARAMETERS; WEIGHT CALCULATION;

SUPPORT VECTOR MACHINES;

ACCURACY; ALPHA COEFFICIENT; ARTICLE; ARTIFICIAL NEURAL NETWORK; CALCULATION; CLASSIFIER; CONTROLLED STUDY; DECISION TREE; ERROR; EXTREME LEARNING MACHINE; K NEAREST NEIGHBOR; KERNEL METHOD; LINEAR DISCRIMINANT ANALYSIS; LINEAR SYSTEM; MACHINE LEARNING; PRIORITY JOURNAL; PROBABILISTIC NEURAL NETWORK; QUALITY CONTROL; RADIAL BASIS FUNCTION NEURAL NETWORK; SUPPORT VECTOR MACHINE;

ANALYTICAL WEIGHT CALCULATION; CLOSED-FORM SOLUTION; EXTREME LEARNING MACHINE; KERNEL-BASED CLASSIFICATION; MARGIN MAXIMIZATION; PARALLEL DELTA RULE; SUPPORT VECTOR MACHINE;

ALGORITHMS; CLASSIFICATION; COMPUTER SIMULATION; DISCRIMINANT ANALYSIS; HUMANS; LINEAR MODELS; NEURAL NETWORKS (COMPUTER); SUPPORT VECTOR MACHINES;

EID: 84888254825     PISSN: 08936080     EISSN: 18792782     Source Type: Journal    
DOI: 10.1016/j.neunet.2013.11.002     Document Type: Article

References (39)

1. Asuncion, A., & Newman, D. (2007). UCI machine learning repository. URL . http://www.ics.uci.edu/~mlearn/MLRepository.html.
2. Auer P., Burgsteiner H., Maass W. A learning rule for very simple universal approximators consisting of a single layer of perceptrons. Neural Networks 2008, 1(21):786-795.
3. Breiman L. Bagging predictors. Machine Learning 1996, 24(2):123-140.
4. Breiman L. Random forests. Machine Learning 2001, 45(1):5-32.
5. Carrión P., Cernadas E., Gálvez J.F., Sá-Otero P. Classification of honeybee pollen using a multiscale texture filtering scheme. Machine Vision and Applications 2004, 15:186-193.
6. Chang, C.-C., & Lin, C.-J. (2008). LIBSVM: a library for Support Vector Machines. URL . http://www.csie.ntu.edu.tw/~cjlin/libsvm.
7. Cortes C., Vapnik V. Support-vector networks. Machine Learning 1995, 20(3):273-297.
8. Dacal-Nieto A., Formella A., Carrión P., Vázquez-Fernández E., Fernández-Delgado M. Common scab detection on potatoes using an infrared hyperspectral imaging. LNCS 2011, Vol. 6979:303-312. Springer.
9. Delogu R., Fanni A., Montisci A. Geometrical synthesis of MLP neural networks. Neurocomputing 2008, 71:919-930.
10. Fernández-Delgado, M., Ribeiro, J., Cernadas, E., & Barro, S. (2010). Fast weight calculation for kernel-based perceptron in two-class classification problems. In Proc. 2010 IEEE int. joint conf. on neural netw., vol. 1 (pp. 1-6).
11. Fernández-Delgado M., Ribeiro J., Cernadas E., Barro S. Direct parallel perceptrons (DPPs): fast analytical calculation of the parallel perceptrons weights with margin control for classification tasks. IEEE Transactions on Neural Networks 2011, 22:1837-1848.
12. Freund, Y., & Schapire, R. (1995). A decision-theoretic generalization of on-line learning and an application to boosting. In Proc. of the 2nd European conf. on computational learning theory (pp. 23-37).
13. González-Rufino E., Carrión P., Cernadas E., Fernández-Delgado M., Domínguez-Petit R. Exhaustive comparison of colour texture features and classification methods to discriminate cells categories in histological images of fish ovary. Pattern Recognition 2013, 46:2391-2407.
14. Haykin S. Neural networks: a comprehensive foundation 1994, Macmillan Coll Div.
15. Hsu C.-W., Lin C.-J. A comparison of methods for multiclass support vector machines. IEEE Transactions on Neural Networks 2002, 13(2):415-425.
16. Huang, G.-B. (2010). Extreme learning machines: learning without iterative tuning. URL . http://www.extreme-learning-machines.org.
17. Huang G.-B., Chen L. Enhanced random search based incremental extreme learning machine. Neurocomputing 2008, 71:3460-3468.
18. Huang G.-B., Ding X., Zhou H. Optimization method based extreme learning machine for classification. Neurocomputing 2010, 74:155-163.
19. Huang G.-B., Wang D., Lan Y. Extreme learning machines: a survey. International Journal of Machine Learning and Cybernetics 2011, 2:107-122.
20. Huang G.-B., Zhou H., Ding X., Zhang R. Extreme learning machine for regression and multiclass classification. IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) 2012, 42:513-529.
21. Iglesias R., Fernández-Delgado M., Barciela J., Peña R., García-Martín S., Herrero C. Comparison of several chemometric techniques for the classification of Orujo distillate alcoholic samples from Galicia (NW Spain) according to their certified brand of origin. Analytical and Bioanalytical Chemistry 2010, 397(6):2603-2614.
22. Kernel Fisher discriminant analysis. (2013). URL . http://en.wikipedia.org/wiki/Kernel_Fisher_discriminant_analysis#Multi-class_KFD.
23. Liu, B. (2003). A kernel-based nonlinear discriminator with closed-form solution. In Proc. int. conf. on neural netw. and signal proc., vol. 1 (pp. 41-44).
24. Martinetz T., Labusch K., Schneegaß D. SoftDoubleMaxMinOver: perceptron-like training of Support Vector Machines. IEEE Transactions on Neural Networks 2009, 20(7):1061-1072.
25. Miche Y., van Heeswijk M., Bas P., Simula O., Lendasse A. TROP-ELM: a double-regularized ELM using LARS and Tikhonov regularization. Neurocomputing 2011, 74(16):2413-2421.
26. Mika, S., Rätsch, G., Weston, J., Schölkopf, B., & Müller, K.-R. (1999). Fisher discriminant analysis with kernels. In Proc. IX IEEE int. conf. on neural networks for signal processing (pp. 41-48).
27. Panagiotakopoulos C., Tsampouka P. The margitron: a generalized perceptron with margin. IEEE Transactions on Neural Networks 2011, 22(3):395-407.
28. Peng X., Wang Y. Geometric algorithms to large margin classifier based on affine hulls. IEEE Transactions on Neural Networks 2012, 23(2):236-246.
29. Rodríguez-Damián M., Cernadas E., Formella A., Fernández-Delgado M., Sá-Otero P. Automatic detection and classification of grains of pollen based on shape and texture. IEEE Transactions on Systems, Man and Cybernetics, Part C 2006, 36:531-542.
30. Sheskin D. Handbook of parametric and nonparametric statistical procedures 2006, CRC Press.
31. Specht D. Probabilistic neural networks. Neural Networks 1990, 3(1):109-118.
32. Talukder A., Casasent D. A closed-form neural network for discriminatory feature extraction from high-dimensional data. Neural Networks 2001, 14:1201-1218.
33. The R project for statistical computing. (2013). URL . http://www.r-project.org.
34. Toh K.-A. An error-counting network for pattern classification. Neurocomputing 2008, 71:1680-1693.
35. Venables W., Ripley B. Modern applied statistics with S 2002, Springer.
36. Wang X., Shao Q., Qing M., Zhai J. Architecture selection for networks trained with extreme learning machine using localized generalization error model. Neurocomputing 2013, 102:3-9.
37. Yap K.S., Lim C.P., Abidin I.Z. A hybrid ART-GRNN online learning neural network with an ε-insensitive loss function. IEEE Transactions on Neural Networks 2008, 19(9):1641-1646.
38. Yujian L., Bo L., Xinwu Y., Yaozong F., Houjun L. Multiconlitron: a general piecewise linear classifier. IEEE Transactions on Neural Networks 2011, 22(2):276-289.
39. Zhai J., Xu H., Wang X. Dynamic ensemble extreme learning machine based on sample entropy. Soft Computing 2012, 16(9):1493-1502.