Optimizing extreme learning machines via ridge regression and batch intrinsic plasticity

Neurocomputing

Volumn 102, Issue , 2013, Pages 23-30

  Neumann, Klaus ^a   Steil, Jochen J ^a  

^a BIELEFELD UNIVERSITY   (Germany)

Author keywords

Batch intrinsic plasticity; Extreme learning machine; Learning; Neural network; Regularization; Ridge regression

Indexed keywords

EXTREME LEARNING MACHINE; INTRINSIC PLASTICITY; LEARNING; REGULARIZATION; RIDGE REGRESSION;

NEURAL NETWORKS; OPTIMIZATION; REGRESSION ANALYSIS;

KNOWLEDGE ACQUISITION;

ANALYTICAL PARAMETERS; ARTICLE; ARTIFICIAL NEURAL NETWORK; BATCH INTRINSIC PLASTICITY; CONTROLLED STUDY; EXTREME LEARNING MACHINE; GENERALIZATION ABILITY; INPUT DISTRIBUTION; LINEAR REGRESSION ANALYSIS; MACHINE LEARNING; OUTPUT DISTRIBUTION; PRIORITY JOURNAL; PROCESS OPTIMIZATION; QUALITY CONTROL; RIDGE REGRESSION; ROBOTICS; ROBOTICS EXTRAPOLATION TASK; ROBOTICS INTERPOLATION TASK; SINGLE HIDDEN LAYER FEED FORWARD NEURAL NETWORK; STATISTICAL ANALYSIS;

EID: 84870236270     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2012.01.041     Document Type: Article

References (26)

1. G.-B. Huang, Q.-Y. Zhu, C.-K. Siew, Extreme learning machine: a new learning scheme of feedforward neural networks, in: Proceedings, IEEE International Joint Conference on Neural Networks, vol. 2, 2004, pp. 985-990.
2. Huang G.-B., Wang D., Lan Y. Extreme learning machine. a survey. Int. J. Mach. Learn. Cybern. 2011, 2(2):107-122.
3. Pao Y.-H., Park G.-H., Sobajic D.J. Learning and generalization characteristics of the random vector functional-link net. Neurocomputing 1994, 6(2):163-180.
4. Broomhead D.S., Lowe D. Multivariable functional interpolation and adaptive networks. Complex Syst. 1988, 2:321-355.
5. Huang G.-B., Li M.-B., Chen L., Siew C.-K. Incremental extreme learning machine with fully complex hidden nodes. Neurocomputing 2008, 71:576-583.
6. Wang L., Wan C. Comments on the extreme learning machine. IEEE Trans. Neural Networks 2008, 19(8):1494-1495.
7. Huang G.-B. Reply to comments on the extreme learning machine. IEEE Trans. Neural Networks 2008, 19(8):1495-1496.
8. Huang G.-B., Zhu Q.-Y., Siew C.-K. Extreme learning machine. theory and applications. Neurocomputing 2006, 70(1-3):489-501.
9. Guang-Bin Huang, Hongming Zhou, Xiaojian Ding, Rui Zhang, Extreme learning machine for regression and multiclass classification, IEEE Trans. Sys. Man Cybernetics B 42 (2) (2012) 513-529.
10. Vapnik V.N. The Nature of Statistical Learning Theory 1995, Springer-Verlag New York, Inc., New York, NY, USA.
11. Huang G.-B., Chen L., Siew C.-K. Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans. Neural Networks 2006, 17(4):879-892.
12. Y. Miche, A. Sorjamaa, A. Lendasse, OP-ELM: theory, experiments and a toolbox, in: Artificial Neural Networks-ICANN 2008, vol. 5163, 2008, pp. 145-154.
13. Feng G., Huang G.-B., Lin Q., Gay R. Error minimized extreme learning machine with growth of hidden nodes and incremental learning. Trans. Neural Networks 2009, 20:1352-1357.
14. W. Deng, Q. Zheng, L. Chen, Regularized extreme learning machine, in: IEEE Symposium on Computational Intelligence and Data Mining, 2009, pp. 389-395.
15. Miche Y., van Heeswijk M., Bas P., Simula O., Lendasse A. Trop-elm: a double-regularized elm using lars and tikhonov regularization. Neurocomputing 2011, 74(16):2413-2421.
16. Tikhonov A.N., Arsenin V.Y. Solutions of ill-posed problems. Soviet Math. Dokl. 1963, (4):1035-1038.
17. X. Dutoit, B. Schrauwen, J.M.V. Campenhout, D. Stroobandt, H.V. Brussel, M. Nuttin, Pruning and regularisation in reservoir computing: a first insight, in: ESANN, 2008, pp. 1-6.
18. Bishop C.M. Training with noise is equivalent to Tikhonov regularization. Neural Comput. 1995, 7(1):108-116.
19. J. Triesch, A gradient rule for the plasticity of a neuron's intrinsic excitability, in: Proceedings of the ICANN, 2005, pp. 65-79.
20. J.J. Steil, Online reservoir adaptation by intrinsic plasticity for backpropagation decorrelation and echo state learning, Neural Networks, Special Issue on Echo State and Liquid State Networks, 2007, pp. 353-364.
21. K. Neumann, J.J. Steil, Batch intrinsic plasticity for extreme learning machines, in: Proceedings, International Conference on Artificial Neural Networks, 2011, pp. 339-346.
22. Kullback S., Leibler A. On information and sufficiency. Ann. Math. Statist. 1951, 22(1):79-86.
23. R. Penrose, A generalized inverse for matrices, in: Mathematical Proceedings of the Cambridge Philosophical Society, 1955, pp. 406-413.
24. A. Frank, A. Asuncion, UCI machine learning repository, 2010.
25. Huang G.-B., Chen L. Convex incremental extreme learning machine. Neurocomputing 2007, 70(16-18):3056-3062.
26. J. Triesch, Synergies beween intrinsic and synaptic plasticity in individual model neurons, in: NIPS, 2005.