Binary/ternary extreme learning machines

Neurocomputing

Volumn 149, Issue Part A, 2015, Pages 187-197

  van Heeswijk, Mark ^a   Miche, Yoan ^a  

^a AALTO UNIVERSITY   (Finland)

Author keywords

Binary features; Extreme learning machine; Hidden layer initialization; Intrinsic plasticity; Random projection; Ternary features

Indexed keywords

KNOWLEDGE ACQUISITION;

BINARY FEATURES; EXTREME LEARNING MACHINE; HIDDEN LAYERS; INTRINSIC PLASTICITY; RANDOM PROJECTIONS; TERNARY FEATURES;

LEARNING SYSTEMS;

ACCURACY; ARTICLE; ARTIFICIAL NEURAL NETWORK; CLASSIFICATION; EXTREME LEARNING MACHINE; INFORMATION PROCESSING; MACHINE LEARNING; MATHEMATICAL COMPUTING; REGRESSION ANALYSIS; TIME;

EID: 84922032836     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2014.01.072     Document Type: Article

References (25)

1. G.-B. Huang, Q.-Y. Zhu, C.-K. Siew, Extreme learning machine: a new learning scheme of feedforward neural networks, in: Proceedings of the International Joint Conference on Neural Networks, vol. 2, 2004, pp. 985-990. doi:10.1109/IJCNN.2004.1380068.
2. Huang G.-B., Zhu Q.-Y., Siew C.-K. Extreme learning machine: theory and applications. Neurocomputing 2006, 70(1-3):489-501. 10.1016/j.neucom.2005.12.126.
3. Miche Y., Sorjamaa A., Bas P., Simula O., Jutten C., Lendasse A. OP-ELM: optimally pruned extreme learning machine. IEEE Trans. Neural Netw. 2010, 21(1):158-162.
4. E. Parviainen, J. Riihimäki, Y. Miche, A. Lendasse, Interpreting extreme learning machine as an approximation to an infinite neural network, in: KDIR 2010-Proceedings of the International Conference on Knowledge Discovery and Information Retrieval, 2010, pp. 65-73.
5. K. Neumann, J.J. Steil, Batch intrinsic plasticity for extreme learning machines, in: Artificial Neural Networks and Machine Learning-ICANN 2011, vol. 6791, Lecture Notes in Computer Science, Springer, Berlin, Heidelberg, 2011, pp. 339-346. http://dx.doi.org/10.1007/978-3-642-21735-7_42.
6. Neumann K., Emmerich C., Steil J.J. Regularization by intrinsic plasticity and its synergies with recurrence for random projection methods. J. Intell. Learn. Syst. Appl. 2012, 4(3):230-246.
7. Neumann K., Steil J.J. Optimizing extreme learning machines via ridge regression and batch intrinsic plasticity. Neurocomputing 2013, 102:555-560. 〈http://dx.doi.org/10.1016/j.neucom.2012.01.041.
8. Steil J.J. Online reservoir adaptation by intrinsic plasticity for backpropagation-decorrelation and echo state learning. Neural Netw.: Off. J. Int. Neural Netw, Soc. 2007, 20(3):353-364.
9. Bartlett P.L. The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. IEEE Trans. Inf. Theory 1998, 44(2):525-536. 10.1109/18.661502.
10. Huang G.-B., Chen L. Convex incremental extreme learning machine. Neurocomputing 2007, 70(16-18):3056-3062. 10.1016/j.neucom.2007.10.008.
11. Feng G., Huang G.-B., Lin Q., Gay R. Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Trans. Neural Netw. 2009, 20(8):1352-1357. 10.1109/TNN.2009.2024147.
12. Huang G.-B., Chen L., Siew C.-K. Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans. Neural Netw. 2006, 17(4):879-892. 10.1109/TNN.2006.875977.
13. T. Raiko, H. Valpola, Deep learning made easier by linear transformations in perceptrons, in: Proceedings of the 15th International Conference on Artificial Intelligence and Statistics, 2012, pp. 924-932.
14. Miche Y., van Heeswijk M., Bas P., Simula O., Lendasse A. TROP-ELM: a double-regularized ELM using LARS and Tikhonov regularization. Neurocomputing 2011, 74(16):2413-2421. 10.1016/j.neucom.2010.12.042.
15. Bishop C.M. Pattern Recognition and Machine Learning 2006, Springer, http://www.springer.com/computer/image+processing/book/978-0-387-31073-2, http://books.google.fi/books?id=kTNoQgAACAAJ.
16. Myers R.H. Classical and Modern Regression with Applications 1990, Duxbury, http://books.google.fi/books?id=LOHHKQAACAAJ. 2nd ed.
17. van Heeswijk M., Miche Y., Oja E., Lendasse A. GPU-accelerated and parallelized ELM ensembles for large-scale regression. Neurocomputing 2011, 74(16):2430-2437. 10.1016/j.neucom.2010.11.034.
18. W.-Y. Deng, Q.-H. Zheng, L. Chen, Regularized extreme learning machine, in: IEEE Symposium on Computational Intelligence and Data Mining, CIDM'09, IEEE, 2009, pp. 389-395. doi:10.1109/CIDM.2009.4938676.
19. Nelder J., Mead R. A simplex method for function minimization. Comput. J. 1965, 7(4):308-313.
20. J.C. Lagarias, J.A. Reeds, M.H. Wright, P.E. Wright, Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions (1998). doi:10.1137/S1052623496303470.
21. Triesch J. A gradient rule for the plasticity of a neuron's intrinsic excitability. Artificial Neural Networks: Biological Inspirations-ICANN 2005 2005, vol. 3696:65-70. Springer, Berlin/Heidelberg. 10.1007/11550822_11.
22. Triesch J. Synergies between intrinsic and synaptic plasticity in individual model neurons. Advances in Neural Information Processing Systems 2005, vol. 17:1417-1424. MIT Press. 10.1162/neco.2007.19.4.885.
23. Verstraeten D., Schrauwen B., D'Haene M., Stroobandt D. An experimental unification of reservoir computing methods. Neural Netw. 2007, 20(3):391-403.
24. K. Neumann, J.J. Steil, Intrinsic plasticity via natural gradient descent, in: ESANN 2012: 20th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, 2012, pp. 555-560.
25. A. Asuncion, D.J. Newman, UCI Machine Learning Repository, 2007.